Custom Distributed Task Execution on Azure Pipelines
Using Nx Agents is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. With an enterprise license, you can set up distributed task execution on your own CI provider using the recipe below.
Run Custom Agents on Azure Pipelines
Run agents directly on Azure Pipelines with the workflow below:
1trigger:
2 - main
3pr:
4 - main
5
6variables:
7 CI: 'true'
8 NX_CLOUD_DISTRIBUTED_EXECUTION_AGENT_COUNT: 3 # expected number of agents
9 ${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
10 NX_BRANCH: $(System.PullRequest.PullRequestNumber)
11 TARGET_BRANCH: $[replace(variables['System.PullRequest.TargetBranch'],'refs/heads/','origin/')]
12 BASE_SHA: $(git merge-base $(TARGET_BRANCH) HEAD)
13 ${{ if ne(variables['Build.Reason'], 'PullRequest') }}:
14 NX_BRANCH: $(Build.SourceBranchName)
15 BASE_SHA: $(git rev-parse HEAD~1)
16 HEAD_SHA: $(git rev-parse HEAD)
17
18jobs:
19 - job: agents
20 strategy:
21 parallel: 3
22 displayName: Nx Cloud Agent
23 pool:
24 vmImage: 'ubuntu-latest'
25 steps:
26 - checkout: self
27 fetchDepth: 0
28 persistCredentials: true
29
30 - script: npm ci
31 - script: npx nx-cloud start-agent
32
33 - job: main
34 displayName: Nx Cloud Main
35 pool:
36 vmImage: 'ubuntu-latest'
37 steps:
38 # Get last successfull commit from Azure Devops CLI
39 - bash: |
40 LAST_SHA=$(az pipelines build list --branch $(Build.SourceBranchName) --definition-ids $(System.DefinitionId) --result succeeded --top 1 --query "[0].triggerInfo.\"ci.sourceSha\"")
41 if [ -z "$LAST_SHA" ]
42 then
43 echo "Last successful commit not found. Using fallback 'HEAD~1': $BASE_SHA"
44 else
45 echo "Last successful commit SHA: $LAST_SHA"
46 echo "##vso[task.setvariable variable=BASE_SHA]$LAST_SHA"
47 fi
48 displayName: 'Get last successful commit SHA'
49 condition: ne(variables['Build.Reason'], 'PullRequest')
50 env:
51 AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
52
53 - script: git branch --track main origin/main
54 - script: npm ci
55 - script: npx nx-cloud start-ci-run --distribute-on="manual" --stop-agents-after="e2e-ci"
56 - script: npx nx-cloud record -- nx format:check --base=$(BASE_SHA) --head=$(HEAD_SHA)
57 - script: npx nx affected --base=$(BASE_SHA) --head=$(HEAD_SHA) -t lint,test,build,e2e-ci --parallel=2 --configuration=ci
58This configuration is setting up two types of jobs - a main job and three agent jobs.
The main job tells Nx Cloud to use DTE and then runs normal Nx commands as if this were a single pipeline set up. Once the commands are done, it notifies Nx Cloud to stop the agent jobs.
The agent jobs set up the repo and then wait for Nx Cloud to assign them tasks.
Two Types of ParallelizationThe agent strategy of parallel: 3 and the nx affected --parallel=2 flag both parallelize tasks, but in different ways. The way this workflow is written, there will be 3 agents running tasks and each agent will try to run 2 tasks at once. If a particular CI run only has 2 tasks, only one agent will be used.