Watch
Script (read-only demo)
#!/usr/bin/env bash
# ==========================================================
# Azure Data Factory (ADF) - CLI Discovery (Pipeline Focus)
# Read-only, lecture-style script
# ==========================================================
# ---- variables ----
RESOURCE_GROUP="covid-reporting-rg"
FACTORY_NAME="covid-reporting-adf-mike123"
echo "=========================================================="
echo " Azure CLI - Azure Data Factory (Pipeline Discovery)"
echo "=========================================================="
echo ""
# ----------------------------------------------------------
echo "STEP 0: Show current Azure subscription context"
echo "This confirms which subscription and tenant the CLI is"
echo "currently using. No changes are made."
echo ""
echo "Command:"
echo " az account show -o table"
echo ""
az account show -o table
# ----------------------------------------------------------
echo ""
echo "STEP 1: List all Azure Data Factories"
echo "This shows every Data Factory in the current subscription."
echo "Use this to discover available factories and resource groups."
echo ""
echo "Command:"
echo " az datafactory list -o table"
echo ""
az datafactory list -o table
# ----------------------------------------------------------
echo ""
echo "STEP 2: What is a pipeline? (conceptual pause)"
echo "A pipeline defines a sequence of activities that move or"
echo "transform data. Pipelines reference datasets (data) and"
echo "linked services (connections and credentials)."
echo ""
echo "No command runs in this step."
echo ""
# ----------------------------------------------------------
echo "STEP 3: List pipelines in the selected Data Factory"
echo "This shows the pipelines that exist in the factory."
echo "Each pipeline represents a data workflow."
echo ""
echo "Command:"
echo " az datafactory pipeline list \\"
echo " --factory-name $FACTORY_NAME \\"
echo " --resource-group $RESOURCE_GROUP"
echo ""
az datafactory pipeline list \
--factory-name "$FACTORY_NAME" \
--resource-group "$RESOURCE_GROUP" \
-o table
# ----------------------------------------------------------
echo ""
echo "STEP 4: List datasets used by the Data Factory"
echo "Datasets describe the structure and location of data used"
echo "by pipelines. These are the inputs and outputs of workflows."
echo ""
echo "Command:"
echo " az datafactory dataset list \\"
echo " --factory-name $FACTORY_NAME \\"
echo " --resource-group $RESOURCE_GROUP"
echo ""
az datafactory dataset list \
--factory-name "$FACTORY_NAME" \
--resource-group "$RESOURCE_GROUP" \
-o table
# ----------------------------------------------------------
echo ""
echo "STEP 5: Reading a pipeline definition (next step)"
echo "To understand how pipelines use datasets and linked"
echo "services, inspect a pipeline's JSON definition."
echo ""
echo "Example command:"
echo " az datafactory pipeline show \\"
echo " --factory-name $FACTORY_NAME \\"
echo " --resource-group $RESOURCE_GROUP \\"
echo " --name <pipeline-name>"
echo ""
# ----------------------------------------------------------
echo "=========================================================="
echo " Discovery complete."
echo " Focus: understanding pipelines and datasets via CLI."
echo "=========================================================="Full transcript (from the video)
This is going to be a tutorial on Azure CLIs focusing on Azure data factories with reads. Right here I have on my screen a bash script that I'm about to run that essentially performs a bunch of read commands. So let's just let finish running. It's finished running and we can look at step zero. So step zero all it does is do a account show o table and that just shows our subscription ID our home tenant or tenant ID things like that. Step one essentially actually lists the data factory. So this is important. Let's say your company has tons of data factories and um you you just want to quickly look through it right and here you go. So in my current subscription I have two data factories and we can see that right here. So these are my two current data factories that I have. So now let's use something useful. Now I want to go into these data factories. Maybe someone has created a resource that I want to look into. So like okay so for instance in this one we use pipeline. So let's go ahead and I already have it open another window but you could just simply go co launch studio but I already have it open here. And here you go. So here's the pipeline that we are looking for right there. Now step four, we can also look at your data sets and um and just to enumerate it out loud, it would be like Azure data factory data set list and then your factory name. So literally the factory name that we pulled from the other command up here and then resource group which is again something that you could pull from up here all the way to the right there.