Demo Controller Failure Pipeline Project

A controller restart typically disrupts Freestyle jobs, but Jenkins Pipelines are checkpointed and can resume after a reboot. In this guide, you’ll modify a simple “Hello World” Pipeline to include a long-running loop, then simulate a controller failure mid-build and observe Jenkins resume from the last checkpoint.

Prerequisites

1. Add a Long-Running Loop to the Pipeline

Open your Pipeline’s Jenkinsfile and locate the Unit Test stage. Here’s the original Declarative Pipeline:

pipeline {
    agent any
    tools {
        maven 'M3'
    }
    stages {
        stage('Echo Version') {
            steps {
                sh 'echo Print Maven Version'
                sh 'mvn -version'
            }
        }
        stage('Build') {
            steps {
                sh 'mvn clean package -DskipTests=true'
            }
        }
        stage('Unit Test') {
            steps {
                sh 'mvn test'
            }
        }
    }
}

1.1 Why a Direct Loop Fails

If you insert a Groovy for loop directly under steps, Declarative syntax rejects it:

stage('Unit Test') {
    steps {
        // This will error out
        for (int i = 0; i < 60; i++) {
            echo "${i + 1}"
            sleep 1
        }
        sh 'mvn test'
    }
}

1.2 Wrapping the Loop in a script Block

Correct the syntax by enclosing the loop inside a script {} step:

pipeline {
    agent any
    tools {
        maven 'M3'
    }
    stages {
        stage('Echo Version') {
            steps {
                sh 'echo Print Maven Version'
                sh 'mvn -version'
            }
        }
        stage('Build') {
            steps {
                sh 'mvn clean package -DskipTests=true'
            }
        }
        stage('Unit Test') {
            steps {
                script {
                    for (int i = 0; i < 60; i++) {
                        echo "${i + 1}"
                        sleep 1
                    }
                }
                sh 'mvn test'
            }
        }
    }
}

Commit and push to main, then trigger the build in Jenkins.

2. Trigger the Build and Observe the Loop

In Pipeline Steps, you’ll see logs like:

[Pipeline] script
[Pipeline] { (Unit Test)
[Pipeline] echo
1
[Pipeline] sleep
Sleeping for 1 sec
...
[Pipeline] echo
15
[Pipeline] sleep
Sleeping for 1 sec

Allow the loop to run to around step 37 before interrupting the controller.

3. Stop the Jenkins Controller Mid-Loop

SSH into your Jenkins controller and halt the service:

sudo systemctl stop jenkins
sudo systemctl status jenkins

The Pipeline log will freeze at the last echo (e.g., 38).

4. Restart Jenkins and Refresh the Console

Bring Jenkins back online:

sudo systemctl restart jenkins

After a few seconds, refresh the Console Output for your running job:

You should see Jenkins resume:

Resuming build at Mon Aug 19 18:00:22 UTC 2024 after Jenkins restart
No need to sleep any longer
Ready to run at Mon Aug 19 18:00:23 UTC 2024
[Pipeline] sleep
Sleeping for 1 sec
[Pipeline] echo
39
...
[Pipeline] echo
60
[Pipeline] sleep
Sleeping for 1 sec
[Pipeline] sh
+ mvn test
[INFO] --- surefire:3.2.5:test (default-test) @ hello-demo ---

The build continues from the last checkpoint and completes successfully.

5. Disabling Resume After Controller Restart

By default, Pipelines pick up after a restart. To enforce an abort-on-restart policy:

1. In Jenkins, select Configure on your Pipeline project.
2. Under Build Triggers, check Do not allow the pipeline to resume if the controller restarts.

Conclusion

Jenkins Pipelines are designed for resilience. By checkpointing the pipeline state, Jenkins can survive controller restarts and resume where it left off. Use this feature for long-running stages, or disable it when an immediate abort is required on controller failure.

References

• Jenkins Pipeline Syntax
• Declarative Pipeline
• Pipeline: Groovy