Demo Refactor existing Jenkinsfile

This lesson refactors an existing Jenkinsfile so the same pipeline can be reused across upcoming demos. It builds on concepts from the Jenkins Pipelines course. If you haven’t completed that course, review it first:

Jenkins Pipelines course

This demo shows a practical Jenkinsfile refactor to:

Reduce and focus stages for CI-focused demos.
Keep Slack notifications and critical scans.
Move report generation into the stage that produces the artifacts (Trivy).

Prerequisites

Familiarity with Declarative Jenkins Pipelines.
Basic Git usage (branching, committing, pushing).
Basic knowledge of Trivy for container vulnerability scanning.
The repository used in this demo (details below).

Repository and branch being used This repo lives under the dasher organization as solar-system. In the previous course we used the feature/enabling-slack branch. For the advanced demos we create a new branch from that branch so we can experiment without affecting the original.

A dark-themed Gitea organization page for "dasher-org" showing a list of repositories (like "solar-system", "solar-system-gitops-argocd", etc.) and a right-hand sidebar with Members and Teams. The top has navigation and buttons for "New Repository" and "New Migration."

Open the repository in your editor or terminal. Example shell prompt:

root@jenkins-controller-1 in ~ on ☁ (us-east-2)
◯>

Goals of the refactor

Retain Slack notification logic to keep CI visibility.
Keep Docker build and Trivy vulnerability scan stages.
Keep npm install, unit tests, and code coverage stages.
Comment out long-running or environment-specific stages we won’t use (SonarQube, OWASP dependency-check, pushing to registries, EC2/K8s deployments).
Move Trivy convert and report publishing (publishHTML, junit) into the Trivy stage so artifacts are grouped with their generating stage.

Slack notification helper The Slack helper function is a small Groovy method used inside the post blocks. It remains unchanged and will be reused by the simplified top-level post block.

def slackNotificationMethod(String buildStatus = 'STARTED') {
    buildStatus = buildStatus ?: 'SUCCESS'

    def color
    if (buildStatus == 'SUCCESS') {
        color = '#47ec05'
    } else if (buildStatus == 'UNSTABLE') {
        color = '#d5ee0d'
    } else {
        color = '#ec2805'
    }

    def msg = "${buildStatus}: `${env.JOB_NAME}` #${env.BUILD_NUMBER}:\n${env.BUILD_URL}"
    slackSend(color: color, message: msg)
}

Top of the Jenkinsfile — agent, tools, environment Example snippet from the top of the Jenkinsfile showing agent, tools, environment, and the simplified post block. This remains the structural basis for the pipeline:

pipeline {
    agent any

    tools {
        // tool installations, e.g. nodejs, maven etc.
        // ...
    }

    environment {
        MONGO_URI = "mongodb+srv://supercluster.d83jj.mongodb.net/superData"
        MONGO_DB_CREDS = credentials('mongo_db_credentials')
        GITEA_TOKEN = credentials('gitea-api-token')
    }

    options {
        // pipeline/options ...
    }

    stages {
        // stages defined below
    }

    post {
        always {
            slackNotificationMethod("${currentBuild.result}")

            // If we still want to clean up repo copies that the pipeline clones:
            script {
                if (fileExists('solar-system-gitops-argocd')) {
                    sh 'rm -rf solar-system-gitops-argocd'
                }
            }
        }
    }
}

Which stages we keep vs. comment out To make the pipeline concise and focused for demos, we kept the core CI stages and commented out most environment-dependent or long-running stages.

Kept (active)	Commented out / Disabled
Installing Dependencies (`npm install`)	`Push Docker Image`
Dependency Scanning (`npm audit`)	`Deploy - AWS EC2`
Build Docker Image	`Integration Testing - AWS EC2`
Trivy Vulnerability Scanner (with local post)	`K8S - Update Image Tag`
Unit Tests + Code Coverage	`K8S - Raise PR`
	`App Deployed?`
	OWASP ZAP, SonarQube, S3 uploads, Lambda deploys, etc.

For readability the disabled stages are preserved in the file as commented placeholders to show they exist but are not executed in this demo:

stage('Build Docker Image') {
    // kept
}

stage('Trivy Vulnerability Scanner') {
    // kept
}

stage('Push Docker Image') {
    // commented out for these demos
}

stage('Deploy - AWS EC2') {
    // commented out
}

stage('Integration Testing - AWS EC2') {
    // commented out
}

stage('K8S - Update Image Tag') {
    // commented out
}

stage('K8S - Raise PR') {
    // commented out
}

stage('App Deployed?') {
    // commented out
}

Trivy stage: run, convert, and publish reports locally within the stage Rather than publishing Trivy’s HTML/XML reports from a global post block, move the conversion and publishing into the Trivy stage’s own post so that report generation stays close to its source. The workflow in the Trivy stage typically:

Run Trivy scans for different severity sets.
Output JSON results.
Convert JSON to HTML and JUnit XML using trivy convert.
Publish JUnit and HTML reports using junit and publishHTML.

Example Trivy scan commands used in the Trivy stage (shell commands executed by the stage):

# MEDIUM severity scan (exit code 0 keeps pipeline going)
trivy image siddharth67/solar-system:$GIT_COMMIT \
  --severity LOW,MEDIUM,HIGH \
  --exit-code 0 \
  --quiet \
  --format json -o trivy-image-MEDIUM-results.json

# CRITICAL severity scan (exit code 1 on finding CRITICAL issues)
trivy image siddharth67/solar-system:$GIT_COMMIT \
  --severity CRITICAL \
  --exit-code 1 \
  --quiet \
  --format json -o trivy-image-CRITICAL-results.json

Convert the JSON results to HTML and JUnit XML:

trivy convert \
  --format template --template "@/usr/local/share/trivy/templates/html.tpl" \
  --output trivy-image-MEDIUM-results.html trivy-image-MEDIUM-results.json

trivy convert \
  --format template --template "@/usr/local/share/trivy/templates/html.tpl" \
  --output trivy-image-CRITICAL-results.html trivy-image-CRITICAL-results.json

trivy convert \
  --format template --template "@/usr/local/share/trivy/templates/junit.tpl" \
  --output trivy-image-MEDIUM-results.xml trivy-image-MEDIUM-results.json

trivy convert \
  --format template --template "@/usr/local/share/trivy/templates/junit.tpl" \
  --output trivy-image-CRITICAL-results.xml trivy-image-CRITICAL-results.json

Publish JUnit/XML and HTML reports inside the Trivy stage post block:

post {
    always {
        sh '''
          # trivy convert commands above
        '''

        // Publish JUnit results converted from Trivy JSON -> JUnit
        junit allowEmptyResults: true, testResults: 'trivy-image-MEDIUM-results.xml'
        junit allowEmptyResults: true, testResults: 'trivy-image-CRITICAL-results.xml'

        // Publish HTML reports
        publishHTML([allowMissing: true,
                     alwaysLinkToLastBuild: true,
                     keepAll: true,
                     reportDir: './',
                     reportFiles: 'trivy-image-MEDIUM-results.html',
                     reportName: 'Trivy MEDIUM Report'])

        publishHTML([allowMissing: true,
                     alwaysLinkToLastBuild: true,
                     keepAll: true,
                     reportDir: './',
                     reportFiles: 'trivy-image-CRITICAL-results.html',
                     reportName: 'Trivy CRITICAL Report'])
    }
}

Cleaning up the global post section To keep the pipeline concise and visually clear in the Jenkins UI, we removed or commented out global junit and publishHTML calls and instead publish results in the specific stage that produces them. The top-level post block now contains only notifications and optional workspace cleanup:

post {
    always {
        slackNotificationMethod("${currentBuild.result}")

        script {
            if (fileExists('solar-system-gitops-argocd')) {
                sh 'rm -rf solar-system-gitops-argocd'
            }
        }
    }
}

Creating and pushing a new feature branch Rather than editing the original feature/enabling-slack branch directly, create a new branch feature/advanced-demo. Example git flow:

root@jenkins-controller-1 in solar-system on feature/enabling-slack via v20.16.0 on (us-east-2)
# create & switch to new branch
git checkout -b feature/advanced-demo

# edit Jenkinsfile, save changes, then commit and push
git add Jenkinsfile
git commit -m "refactored Jenkinsfile for advanced demos"
git push -u origin feature/advanced-demo

As soon as the branch is pushed, the multibranch pipeline (Git Organization job) detects the new branch and triggers a build when it finds the Jenkinsfile. Build execution and handling expected failures The refactor reduces stage count (from ~20 to ~4–5), which is ideal for demos. During early runs you may encounter Trivy failures that originate outside your environment (e.g., rate-limiting when Trivy downloads its vulnerability DB). These failures are external and should be handled separately (e.g., use a cached local DB, adjust Trivy exit codes, or re-run). Sample console excerpt showing a Trivy DB download rate-limit error:

+ trivy image siddharth67/solar-system:23ac3d5c61f666daf3f8795cc229693d3e3af78e \
  --severity LOW,MEDIUM,HIGH --exit-code 0 --quiet --format json -o trivy-image-MEDIUM-results.json
2024-11-10T03:03:22Z        FATAL    Fatal error   init error: DB error: failed to download vulnerability DB: database download error: oci download error: failed to fetch the layer: GET https://ghcr.io/v2/aquasecurity/trivy-db/blobs/sha256:...: TOOMANYREQUESTS: retry-after: 160.753µs, allowed: 44000/minute
script returned exit code 1

Trivy failures like the example above are usually due to external rate-limiting of Trivy DB downloads. Consider these mitigations:

Use a local Trivy DB cache (mirror) for CI.
Adjust --exit-code thresholds to avoid failing builds on lower-severity issues.
Add retry logic or fallback behavior for DB downloads.

Summary of the refactor

Focused and reduced the Jenkinsfile to key CI stages for demos: dependencies, unit tests/coverage, Docker build, and Trivy scans.
Preserved and reused Slack notification logic; simplified the global post block to notifications and cleanup.
Moved Trivy report conversion and publishing into the Trivy stage post block for logical grouping of artifacts and easier debugging.
Disabled long-running/deployment-specific stages in-place (commented) so they can be re-enabled later if needed.
Created feature/advanced-demo branch to test the refactor without modifying the original branch.

Next steps

Implement Trivy DB caching or retry strategies to avoid rate-limit issues.
Consider moving shared helper functions (like slackNotificationMethod) to a Jenkins shared library if they will be reused across multiple pipelines.
Re-enable additional stages gradually when demonstration requirements expand (for example, SonarQube or K8s deployment stages).

Links and references

Shared Libraries in Jenkins

Agents and Nodes in Jenkins

Pipeline Enhancement and Caching

Pipeline Structure and Scripted vs Declarative

Jenkins Administration and Monitoring

Backup and Configuration Management

Introduction

Demo Refactor existing Jenkinsfile

Watch Video