Skip to main content
Jenkins · lesson 3 of 5

Beginner

2

Intermediate

2

Advanced

1

Also part of Jenkins

1
Open the Jenkins page →
Jenkins intermediate Lesson 3 of 5

Jenkins: Multibranch Pipelines & Parallel Stages

Level up Jenkins by running builds for every branch automatically, parallelising slow stages, and controlling flow with post conditions and input gates.

By SkillByExample · Updated

After you understand basic Jenkins pipelines, the next step is scaling them: automating per-branch builds, running slow test suites in parallel, and adding promotion gates for production deploys.

Learning outcomes

By the end you can:

  • configure a Multibranch Pipeline project
  • run stages in parallel
  • use post conditions for notifications and cleanup
  • add manual input gates before production releases

1) Multibranch Pipeline

A Multibranch Pipeline automatically discovers branches and PRs in your repo. Each branch’s Jenkinsfile defines its own pipeline.

How to set it up

  1. In Jenkins, click New Item → Multibranch Pipeline
  2. Under Branch Sources, add your Git repo URL and credentials
  3. Set scan interval (e.g., every 5 minutes) or use a webhook
  4. Jenkins creates and runs a job for each branch with a Jenkinsfile

Branch-aware Jenkinsfile

pipeline {
  agent any

  stages {
    stage('Build') {
      steps {
        sh 'npm ci'
        sh 'npm run build'
      }
    }

    stage('Test') {
      steps {
        sh 'npm test'
      }
    }

    stage('Deploy to Staging') {
      // Only run on the main branch
      when {
        branch 'main'
      }
      steps {
        sh './scripts/deploy.sh staging'
      }
    }

    stage('Deploy to Production') {
      when {
        branch 'main'
      }
      input {
        message "Deploy to production?"
        ok "Deploy Now"
        submitter "team-lead,devops-team"
      }
      steps {
        sh './scripts/deploy.sh production'
      }
    }
  }
}

2) Parallel stages

Run independent stages at the same time to reduce pipeline duration.

pipeline {
  agent any

  stages {
    stage('Build') {
      steps {
        sh 'npm ci'
        sh 'npm run build'
      }
    }

    stage('Test Suite') {
      parallel {
        stage('Unit Tests') {
          steps {
            sh 'npm run test:unit'
          }
        }

        stage('Integration Tests') {
          agent { label 'integration-runner' }
          steps {
            sh 'npm run test:integration'
          }
        }

        stage('Lint') {
          steps {
            sh 'npm run lint'
          }
        }
      }
    }

    stage('Security Scan') {
      steps {
        sh 'npm audit --audit-level=high'
      }
    }
  }
}

failFast true on the stage that holds parallel {} aborts the sibling branches as soon as one fails—useful when you want fast feedback. It goes beside parallel, not inside it; inside, the pipeline fails to compile with Expected a stage:

stage('Tests') {
  failFast true
  parallel {
    stage('Unit') { steps { sh 'npm run test:unit' } }
    stage('E2E')  { steps { sh 'npm run test:e2e' } }
  }
}

3) Post conditions

post blocks run after stages complete, regardless of result.

pipeline {
  agent any

  stages {
    stage('Build') {
      steps {
        sh 'npm ci && npm run build'
      }
    }

    stage('Test') {
      steps {
        sh 'npm test'
      }
    }
  }

  post {
    always {
      // Always clean up workspace
      cleanWs()
    }

    success {
      // Notify on success
      echo "Build ${env.BUILD_NUMBER} succeeded on branch ${env.BRANCH_NAME}"
    }

    failure {
      // Alert the team on failure
      emailext(
        to: 'team@example.com',
        subject: "Build Failed: ${env.JOB_NAME} #${env.BUILD_NUMBER}",
        body: "Check ${env.BUILD_URL} for details."
      )
    }

    unstable {
      // Tests ran but some failed (non-fatal)
      echo "Build is unstable—check test results"
    }
  }
}

4) Environment variables and build parameters

pipeline {
  agent any

  parameters {
    string(name: 'APP_VERSION', defaultValue: '1.0.0', description: 'Version to deploy')
    choice(name: 'TARGET_ENV', choices: ['staging', 'production'], description: 'Target environment')
    booleanParam(name: 'SKIP_TESTS', defaultValue: false, description: 'Skip test stage')
  }

  environment {
    // Static values
    REGISTRY = 'registry.example.com'
    IMAGE_TAG = "${params.APP_VERSION}-${env.BUILD_NUMBER}"
  }

  stages {
    stage('Test') {
      when {
        expression { !params.SKIP_TESTS }
      }
      steps {
        sh 'npm test'
      }
    }

    stage('Build Docker Image') {
      steps {
        sh "docker build -t ${REGISTRY}/myapp:${IMAGE_TAG} ."
        sh "docker push ${REGISTRY}/myapp:${IMAGE_TAG}"
      }
    }

    stage('Deploy') {
      steps {
        sh "./scripts/deploy.sh ${params.TARGET_ENV} ${IMAGE_TAG}"
      }
    }
  }
}

5) Stashing files between stages on different agents

If your stages run on different agents, use stash/unstash to pass files:

stage('Build') {
  agent { label 'build-server' }
  steps {
    sh 'npm ci && npm run build'
    stash name: 'dist', includes: 'dist/**'
  }
}

stage('Test') {
  agent { label 'test-server' }
  steps {
    unstash 'dist'
    sh 'npm test'
  }
}

Practice

These solutions ran on a jenkins/jenkins:lts container with the Pipeline, Git and Workspace Cleanup plugins. Task 1 uses a Multibranch Pipeline over a local git repository. The other tasks are plain Pipeline jobs with the Jenkinsfile pasted as their script, run with jenkins-cli.jar build -s -v, which waits for the build and streams its log. Where npm appears in the lesson, the solutions use echo and exit so that the pipeline behaviour is the only thing under test.

  1. Make a git repo whose main branch holds the branch-aware Jenkinsfile, without the production stage and with every sh replaced by an echo of $BRANCH_NAME. Create a feature/login branch from it, then point a Multibranch Pipeline called app at the repo. What does each branch’s build do?

    Solution 
    pipeline {
  agent any

  stages {
    stage('Build') {
      steps {
        sh 'echo "building $BRANCH_NAME"'
      }
    }

    stage('Test') {
      steps {
        sh 'echo "testing $BRANCH_NAME"'
      }
    }

    stage('Deploy to Staging') {
      // Only run on the main branch
      when {
        branch 'main'
      }
      steps {
        sh 'echo "deploying $BRANCH_NAME to staging"'
      }
    }
  }
}
    $ curl -s http://localhost:8080/job/app/indexing/consoleText | grep -E 'Checking branch|found|Scheduled|Processed'
Checking branches...
  Checking branch feature/login
      ‘Jenkinsfile’ found
Scheduled build for branch: feature/login
  Checking branch main
      ‘Jenkinsfile’ found
Scheduled build for branch: main
Processed 2 branches
$ curl -s http://localhost:8080/job/app/job/main/1/consoleText | grep -E '^(Checking out|\+ |Stage |Finished)'
Checking out Revision 598f4b8c2b129c48aa37ba3bf5e393592d80586c (main)
+ echo building main
+ echo testing main
+ echo deploying main to staging
Finished: SUCCESS
$ curl -s http://localhost:8080/job/app/job/feature%252Flogin/1/consoleText | grep -E '^(Checking out|\+ |Stage |Finished)'
Checking out Revision 25af6a05193b879fb66aad0fb4582bb2753ab170 (feature/login)
+ echo building feature/login
+ echo testing feature/login
Stage "Deploy to Staging" skipped due to when conditional
Finished: SUCCESS

    Branch indexing found a Jenkinsfile on each branch and created one job per branch. Both ran the same file, but when { branch 'main' } skipped the deploy on the feature branch. The / in feature/login is encoded twice in the job URL (%252F).

  2. Build a Tests stage with two parallel branches: Unit runs sh 'sleep 10; echo "unit passed"' and E2E runs sh 'sleep 2; echo "e2e failed"; exit 1'. Run it without failFast, then with failFast true.

    Solution

    Without failFast:

    $ java -jar jenkins-cli.jar -s http://localhost:8080/ build parallel-nofailfast -s -v
Started parallel-nofailfast #1
Started from command line by anonymous
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/parallel-nofailfast
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Tests)
[Pipeline] parallel
[Pipeline] { (Branch: Unit)
[Pipeline] { (Branch: E2E)
[Pipeline] stage
[Pipeline] { (Unit)
[Pipeline] stage
[Pipeline] { (E2E)
[Pipeline] sh
[Pipeline] sh
+ sleep 10
+ sleep 2
+ echo e2e failed
e2e failed
+ exit 1
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Failed in branch E2E
+ echo unit passed
unit passed
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE
Completed parallel-nofailfast #1 : FAILURE

    With failFast true placed beside parallel:

    pipeline {
  agent any

  stages {
    stage('Tests') {
      failFast true
      parallel {
        stage('Unit') { steps { sh 'sleep 10; echo "unit passed"' } }
        stage('E2E')  { steps { sh 'sleep 2; echo "e2e failed"; exit 1' } }
      }
    }
  }
}
    $ java -jar jenkins-cli.jar -s http://localhost:8080/ build failfast -s -v
Started failfast #1
Started from command line by anonymous
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/failfast
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Tests)
[Pipeline] parallel
[Pipeline] { (Branch: Unit)
[Pipeline] { (Branch: E2E)
[Pipeline] stage
[Pipeline] { (Unit)
[Pipeline] stage
[Pipeline] { (E2E)
[Pipeline] sh
[Pipeline] sh
+ sleep 10
+ sleep 2
+ echo e2e failed
e2e failed
+ exit 1
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Failed in branch E2E
Sending interrupt signal to process
Terminated
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Failed in branch Unit
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE
Completed failfast #1 : FAILURE

    Both builds fail. Without failFast, Unit kept running for its full 10 seconds after E2E had already doomed the build. With it, Jenkins killed Unit’s sleep the moment E2E failed, so “unit passed” never prints.

  3. Take the “Post conditions” pipeline. Make Build sh 'mkdir -p dist && echo "app" > dist/app.txt && ls', make Test fail with sh 'echo "1 test failed"; exit 1', and replace emailext(...) with echo "ALERT: ${env.JOB_NAME} #${env.BUILD_NUMBER} failed, see ${env.BUILD_URL}". Which post blocks run?

    Solution 
    $ java -jar jenkins-cli.jar -s http://localhost:8080/ build post-demo -s -v
Started post-demo #1
Started from command line by anonymous
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/post-demo
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Build)
[Pipeline] sh
+ mkdir -p dist
+ echo app
+ ls
dist
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] sh
+ echo 1 test failed
1 test failed
+ exit 1
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] cleanWs
[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Deferred wipeout is used...
[WS-CLEANUP] done
[Pipeline] echo
ALERT: post-demo #1 failed, see http://localhost:8080/job/post-demo/1/
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE
Completed post-demo #1 : FAILURE

    Only always and failure ran, in that order, inside a synthetic “Declarative: Post Actions” stage. success and unstable were silently passed over. post reacts to the build’s result, but it does not change it: the build still finishes as FAILURE.

  4. Use the “Environment variables and build parameters” pipeline, with the Docker and deploy-script steps replaced by sh "echo deploying ${REGISTRY}/myapp:${IMAGE_TAG} to ${params.TARGET_ENV}" and the test step by sh 'echo "running tests"'. Run it once with the defaults. Then run it with SKIP_TESTS=true, TARGET_ENV=production and APP_VERSION=2.4.0.

    Solution 
    $ java -jar jenkins-cli.jar -s http://localhost:8080/ build params-demo -s -v
Started params-demo #1
Started from command line by anonymous
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/params-demo
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] sh
+ echo running tests
running tests
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Deploy)
[Pipeline] sh
+ echo deploying registry.example.com/myapp:1.0.0-1 to staging
deploying registry.example.com/myapp:1.0.0-1 to staging
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
Completed params-demo #1 : SUCCESS
$ java -jar jenkins-cli.jar -s http://localhost:8080/ build params-demo -s -v -p SKIP_TESTS=true -p TARGET_ENV=production -p APP_VERSION=2.4.0
Started params-demo #2
Started from command line by anonymous
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/params-demo
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Test)
Stage "Test" skipped due to when conditional
[Pipeline] getContext
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Deploy)
[Pipeline] sh
+ echo deploying registry.example.com/myapp:2.4.0-2 to production
deploying registry.example.com/myapp:2.4.0-2 to production
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
Completed params-demo #2 : SUCCESS

    The first run used each parameter’s defaultValue. The second skipped Test through the expression condition. IMAGE_TAG combines both inputs: 2.4.0 from the parameter and 2 from the build number.

  5. Give the production gate an ID by adding id 'Prod' to its input block. Put it in a one-stage pipeline whose step is sh 'echo "deploying to production"', and leave out submitter. Start the build, look at its log, then approve it.

    Solution 
    pipeline {
  agent any

  stages {
    stage('Deploy to Production') {
      input {
        id 'Prod'
        message "Deploy to production?"
        ok "Deploy Now"
      }
      steps {
        sh 'echo "deploying to production"'
      }
    }
  }
}
    $ curl -s http://localhost:8080/job/input-gate/1/consoleText
Started from command line by anonymous
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/input-gate
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Deploy to Production)
[Pipeline] input
Deploy to production?
Deploy Now or Abort

    The build sits there until someone clicks Deploy Now on the build page (or POSTs to /job/input-gate/1/input/Prod/proceedEmpty with a CSRF crumb). After approval:

    $ curl -s http://localhost:8080/job/input-gate/1/consoleText
Started from command line by anonymous
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/input-gate
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Deploy to Production)
[Pipeline] input
Deploy to production?
Deploy Now or Abort
[Pipeline] sh
+ echo deploying to production
deploying to production
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

    The input line comes after [Pipeline] node. With agent any at the top, the build held an executor for the whole time it waited for a human.

Next steps

  • Jenkins Shared Libraries: reusable pipeline code across teams
  • Jenkins + Docker: containerised build agents
  • Integrate with Terraform and Ansible for infrastructure deployments

Frequently Asked Questions

What is a Multibranch Pipeline in Jenkins?
A Multibranch Pipeline project scans your repository and automatically creates a Jenkins pipeline job for each branch (or PR) that contains a Jenkinsfile. You don't need to manually create a job per branch.
How do parallel stages affect build time?
Parallel stages run simultaneously on available executors/agents. If you have three independent test suites each taking 5 minutes, running them in parallel reduces total test time from 15 minutes to ~5 minutes.