Skip to main content
In this lesson we focus on making software releases secure and resilient. Security is not only about what features you ship — it’s about ensuring the release process itself does not introduce risk. Developers write code, security teams define controls, and SREs operationalize and enforce those controls. When ownership gets assumed by others, SREs often become the last line of defense; many postmortems note “this should have been caught at deployment.”
A presentation slide titled "Security in Software Releases Is Essential" showing an "Expensive Hall of Fame" of three major incidents. It lists SolarWinds hack (2020), Log4Shell/Log4j (2021), and the Target breach (2013) with short notes on their impacts.
Real incidents demonstrate how the release path can amplify risk: the SolarWinds supply-chain compromise (2020) propagated a poisoned build broadly; Log4Shell (2021) showed how a single vulnerable library can cascade across ecosystems; and the Target breach (2013) involved stolen deployment credentials with costly business impact. SREs validate deployments, enforce security policies, and stop misconfigurations before they become incidents.
A presentation slide titled "Security in Software Releases Is Essential" stating "SREs are the last line of defense." It lists three points showing developers write code (SREs ensure secure deployment), security teams set policies (SREs enforce them), and everyone assumes someone else checks (SREs actually check).
Common release-path risks and mitigations A small misstep can expose everything. For example:
A slide titled "Common Risks in the Release Path" showing a "Vulnerability Domino Effect" graphic (a cracked shield on a monitor). It lists five numbered risks: widely used library, vulnerability found in production, no automated dependency scanning, manual audits of many services, and resulting panic/ executive escalations.
Build security into CI/CD (shift left)
  • Add automated checks in CI so vulnerabilities are detected before deployment.
  • SAST (Static Application Security Testing) inspects source code for issues (SQL injection, XSS, hard-coded credentials, buffer overflows, weak crypto) without running programs.
The image is a presentation slide titled "Security Verification in CI/CD" explaining Static Application Security Testing (SAST) which scans source code for security issues without running the program. A sidebar lists common findings: SQL injection, cross-site scripting (XSS), buffer overflow, and hardcoded credentials.
Example: GitHub CodeQL can be used as a SAST step in GitHub Actions. Keep CI configured to fail or require remediation for critical issues where appropriate.
Dependencies and SBOMs
  • Most modern apps include large third-party dependency trees. Automated dependency scanning prevents known vulnerable packages from reaching production.
  • Tools: pip-audit, npm audit, GitHub Dependabot, SCA (Software Composition Analysis) solutions.
A Software Bill of Materials (SBOM) is an “ingredients list” for your software. With an SBOM you can rapidly identify which services are affected by a vulnerability (for example, during Log4Shell), rather than manually tracing dependency trees service by service.
A presentation slide titled "Security Verification in CI/CD" explaining Software Bill of Materials (SBOM) as a complete inventory of software components. It notes why SBOMs matter — they helped teams quickly identify at-risk apps during Log4Shell, whereas without them teams spent days or weeks manually checking services.
Core principles and common practices
  • Principle of Least Privilege: grant only the minimum permissions required.
  • Automate security checks: dependency scanning, secret detection, SAST/DAST, container image scanning, and infrastructure-as-code (IaC) checks should run in CI.
  • Review and sign artifacts: build-time signing ensures the artifact you deploy is the artifact you built. cosign and sigstore are industry-standard tools for container/image signing and verification.
  • Continuous monitoring: subscribe to advisories, configure automated CVE alerts, and periodically re-scan deployed images and running systems.
An infographic titled "Best Practices for Secure Releases" that lists four principles: Principle of Least Privilege, Automate Security Checks, Review and Sign Artifacts, and Monitor for New Vulnerabilities. A checklist below recommends subscribing to security advisories, setting up automated CVE alerts, and regularly reviewing infrastructure and apps.
Progressive security pipeline — step by step Evolve pipelines incrementally to reduce risk without blocking delivery:
  1. Start with a basic build/deploy pipeline.
  2. Add dependency vulnerability scanning so bad packages are blocked early.
  3. Generate SBOMs so you can quickly identify affected apps when vulnerabilities surface.
  4. Improve authentication and restrict permissions (least privilege).
  5. Add container image scanning (Trivy, Grype, etc.).
  6. Enforce environment separation and promotion gates (staging → production).
A presentation slide titled "Progressive Security Pipeline Evolution" that lists five steps: The Problem (Basic Insecure Pipeline), Dependency Vulnerability Scanning, Software Bill of Materials (SBOM), Secure Authentication and Permissions, and Container Security Scanning. A side note reads "Scan image for vulnerabilities" and the slide is copyrighted by KodeKloud.
Working with the provided repository
  • Fork and clone the repository locally to iterate on release workflows and test changes.
Design a consolidated, secure GitHub Actions workflow with:
  • Explicit triggers (push branches and workflow_dispatch for manual runs).
  • Limited permissions using the top-level permissions block.
  • Jobs: build-and-test, build-container (secure registry login, image build, SBOM generation, container scan), deploy-staging, and deploy-production (with manual approval).
Example workflow header and permissions:
Build-and-test (important parts only):
Authentication to container registry — avoid hardcoded credentials
  • Never store plaintext credentials in workflows or source code.
Never store plaintext credentials in workflows or in source code. Use the provided runtime tokens and secrets.
Do not commit static credentials. Hardcoded passwords in CI can be exposed in logs, forks, or via leaked access. Rotate any credentials that were committed immediately.
Bad example (do not use):
Good example (use GitHub token at runtime):
Install Docker Compose and build/push images (use fixed URL for reproducibility):
Generate SBOM and upload as an artifact (example using syft):
Container scanning (example using Grype):
Promotion gates and deploy jobs
  • Deploy to staging automatically if scans pass.
  • Require manual approval for production promotion (explicit human gate).
Troubleshooting — failed authentication due to insecure login
  • Common error when using hardcoded login:
Use runtime tokens and secrets as shown above; replace insecure credentials and re-run the workflow. When the pipeline runs successfully you will see GitHub Actions output indicating build, scans, SBOM upload, and staged deployment succeeded.
A screenshot of a GitHub repository's Actions page showing a workflow run titled "run secure pipeline #76." The right pane shows the "deploy-staging" job and its steps (Set up job, Checkout code, Install Docker Compose, Create Staging Environment, Deploy to Staging, etc.).
Summary — what changed and why it matters
  • Replaced insecure patterns with managed, auditable controls:
    • Token-based authentication instead of hard-coded secrets
    • Automated dependency and container scanning in CI
    • SBOM generation and artifact signing to ensure provenance
    • Principle of least privilege and environment-based promotion gates
Each layer reduces the chance that a single mistake or vulnerable dependency becomes a full-scale breach. Use the pipeline pattern shown here as a starting point: customize scanners, severity thresholds, and promotion gates to your organization’s risk profile.
A presentation slide titled "Progressive Security Pipeline Evolution" showing a Before vs After comparison: Before lists issues like hardcoded credentials, no scanning/SBOM, overly broad permissions, and no deployment controls; After lists fixes like token auth, automated/SBOM/container scanning, least privilege, and environment controls.
Further reading and references Next, we’ll wrap up this section with a concise checklist of best practices and additional resources for hardening CI/CD pipelines.

Watch Video