Utilize newContainerPerStage

In this lesson, you will learn why the newContainerPerStage directive is essential when using a Dockerfile agent in a Jenkins Declarative Pipeline. By default, Jenkins runs all stages in a single container, sharing the workspace and any generated artifacts. We’ll explore both behaviors and show how to enforce stage-level isolation.

Single-Container Pipeline

When you configure a pipeline with a global Dockerfile agent, every stage executes inside the same container. Artifacts created in one stage remain available in the next:

pipeline {
  agent {
    dockerfile {
      filename 'Dockerfile.cowsay'
      label 'ubuntu-docker-jdk17-node20'
    }
  }
  stages {
    stage('Stage-1') {
      steps {
        sh 'cat /etc/os-release'
        sh 'node -v'
        sh 'npm -v'
        echo '#############################'
        sh "echo $((RANDOM)) > /tmp/imp-file-$BUILD_ID"
        sh 'ls -ltr /tmp/imp-file-$BUILD_ID'
        sh 'cat /tmp/imp-file-$BUILD_ID'
        echo '#############################'
      }
    }
    stage('Stage-2') {
      steps {
        sh 'cat /etc/os-release'
        sh 'node -v'
        sh 'npm -v'
        echo 'Reading file generated in Stage-1:'
        sh 'cat /tmp/imp-file-$BUILD_ID'
      }
    }
    stage('Stage-3') {
      steps {
        sh 'cat /etc/os-release'
        sh 'node -v'
        sh 'npm -v'
      }
    }
    stage('Stage-4') {
      steps {
        sh 'node -v'
        sh 'npm -v'
        sh 'cowsay -f dragon This is running on Docker Container'
        echo 'Final file check:'
        sh 'cat /tmp/imp-file-$BUILD_ID'
        sh 'sleep 120s'
      }
    }
  }
}

Execution Logs

# Dockerfile build and container start
$ docker build -t pipeline-external-agent -f Dockerfile.cowsay .
...
$ docker run ...
# Stage-1
+ cat /etc/os-release
NAME="Alpine Linux" ...
+ node -v
v18.20.4
+ npm -v
10.7.0
+ echo $((RANDOM)) > /tmp/imp-file-7
+ ls -ltr /tmp/imp-file-7
-rw-r--r-- 1 root root 5 Nov 10 07:42 /tmp/imp-file-7
+ cat /tmp/imp-file-7
7577
# Stage-2, Stage-3, Stage-4 reuse the same container
+ cat /tmp/imp-file-7
7577

When the pipeline finishes, Jenkins stops and removes that single container.

Enforcing Stage Isolation with newContainerPerStage

To run each stage in its own container—so artifacts from one stage aren’t carried over—enable the newContainerPerStage() option.

Updated Pipeline Configuration

Insert an options block with newContainerPerStage():

pipeline {
  agent {
    dockerfile {
      filename 'Dockerfile.cowsay'
      label 'ubuntu-docker-jdk17-node20'
    }
  }
  options {
    newContainerPerStage()
  }
  stages {
    stage('Stage-1') {
      steps {
        sh 'cat /etc/os-release'
        sh 'node -v'
        sh 'npm -v'
        echo '********************'
        sh "echo $((RANDOM)) > /tmp/imp-file-$BUILD_ID"
        sh 'ls -ltr /tmp/imp-file-$BUILD_ID'
        sh 'cat /tmp/imp-file-$BUILD_ID'
        echo '********************'
      }
    }
    stage('Stage-2') {
      steps {
        sh 'cat /etc/os-release'
        sh 'node -v'
        sh 'npm -v'
        echo 'Trying to read file from Stage-1:'
        sh 'ls -ltr /tmp/imp-file-$BUILD_ID'
        sh 'cat /tmp/imp-file-$BUILD_ID'
      }
    }
  }
}

Now Jenkins rebuilds the Docker image and starts a fresh container for every stage. Stage-2 will fail because the file from Stage-1 no longer exists:

# Stage-2
$ docker build -t pipeline-external-agent -f Dockerfile.cowsay .
...
# Inside new container
$ ls -ltr /tmp/imp-file-9
ls: cannot access '/tmp/imp-file-9': No such file or directory

Comparison of Pipeline Behaviors

Links and References

• Jenkins Pipeline Syntax
• Jenkins Docker Pipeline
• Pipeline Options: newContainerPerStage

That’s all for now—thank you for reading!