CICD on Kubernetes Pipelines Tasks Artifacts and Flow

This lesson explains how to run Continuous Integration (CI) natively on Kubernetes: building images, running tests, scanning for vulnerabilities, and producing artifacts that a GitOps system deploys. We contrast the traditional external-CI model with Kubernetes-native CI, and introduce two CNCF projects that implement Kubernetes-native pipelines: Tekton and Argo Workflows. We also provide examples and a comparison to help you choose the right tool for common CI/CD patterns. Traditionally, CI runs outside the cluster using systems like Jenkins, GitHub Actions, or GitLab CI. That works, but it has operational and security trade-offs which we’ll cover below. This lesson shows why running CI inside Kubernetes often results in better security posture, simpler credential management, and more efficient resource usage. A company ran Jenkins on a dedicated EC2 instance to manage deployments to three Kubernetes clusters. Jenkins had cluster-admin access to all three clusters, write access to the container registry, and access to deployment secrets across seven environments. Everything worked—until a Jenkins plugin vulnerability was disclosed.

A diagram titled "CI Systems Outside Kubernetes" showing Jenkins running on an EC2 instance. Jenkins connects to three external services: 3 Kubernetes clusters, a container registry, and a secrets store (7 envs).

Centralized CI servers with broad privileges create a single blast radius. If the external CI control plane is compromised, attackers can access clusters, registries, and secrets. Plan for least privilege and minimize credential exposure.

If Jenkins had been compromised, attackers could have gained full access to production systems. The emergency remediation (rotating cluster credentials, regenerating registry tokens, replacing secrets) required two weeks. The root cause was architectural: one external system held privileged access across environments. Common problems with the external-CI/server model:

Separate infrastructure to manage: another server/VM or cluster to patch, secure, and scale.
Credential sprawl: CI pipelines require kubectl credentials, registry push tokens, and environment secrets. A single compromised host exposes them all.
Resource inefficiency: static runners or always-on VMs often remain idle between builds.
Limited native integration: external CI cannot easily leverage Kubernetes RBAC, service accounts, pod-level resource limits, or pod security policies.

The Kubernetes-native approach Run pipelines inside Kubernetes. Each build step maps to a pod (or a container within a pod), which unlocks Kubernetes primitives for CI:

Pods as on-demand runners: each task runs in an isolated pod that Kubernetes schedules and scales—no need to manage long-lived runner VMs.
Native RBAC and least privilege: pipelines authenticate using Kubernetes service accounts and can be limited by fine-grained RBAC.
Secrets integration: mount Secrets directly into build pods (no copying or storing credentials externally).
Declarative pipelines: pipelines are CRDs (CustomResourceDefinitions) that can be stored in Git and managed via GitOps.

Kubernetes-native CI turns pipeline steps into Kubernetes objects. That makes pipelines observable, auditable, and manageable with the same tooling you already use for workloads (kubectl, RBAC, NetworkPolicies, ResourceQuotas).

Two CNCF projects implement Kubernetes-native CI/CD patterns: Tekton and Argo Workflows. They both run on Kubernetes but differ in design and execution model.

A slide titled "Kubernetes-Native CI/CD" showing two CNCF projects: Tekton on the left (CI/CD building blocks) and Argo Workflows on the right (general-purpose DAG-based workflow engine). Each project is shown with a mascot icon and a short description of its features.

Tekton (CI/CD building blocks) Tekton is purpose-built for CI/CD; it models pipelines as reusable building blocks and exposes them as CRDs. Core Tekton resources:

Task: a sequence of steps that run in the same pod. Each step is a container. Steps share the pod filesystem and can use the same workspace.
Pipeline: a collection of Tasks with ordering and dependencies (so you can express “run tests after build”).
TaskRun / PipelineRun: runtime instantiations of Task or Pipeline with concrete parameters and resources.
Workspace: a PVC-backed volume that allows different pods (tasks) to share files like compiled artifacts or test reports.

Example behavior (Tekton):

A pipeline with three tasks — FetchSource (clone repo), BuildImage (run Kaniko to build and push), and Test (execute test suites).
runAfter or explicit dependency ordering controls execution. Tasks that have no dependencies run in parallel.
Because all steps in a Task share a pod filesystem, passing build artifacts between steps is simple (no external artifact store required).

Minimal Tekton Task + Pipeline example (illustrative):

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: build-image
spec:
  steps:
  - name: kaniko
    image: gcr.io/kaniko-project/executor:latest
    command: ["/kaniko/executor"]
    args: ["--dockerfile=/workspace/source/Dockerfile", "--destination=registry.example.com/myapp:latest"]
  workspaces:
  - name: source

---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: ci-pipeline
spec:
  workspaces:
  - name: shared-workspace
  tasks:
  - name: fetch
    taskRef:
      name: git-clone
    workspaces:
    - name: output
      workspace: shared-workspace
  - name: build
    taskRef:
      name: build-image
    runAfter: [fetch]
    workspaces:
    - name: source
      workspace: shared-workspace

Argo Workflows (DAG-based workflow engine) Argo Workflows is a general-purpose workflow engine designed for DAGs, batch jobs, ML pipelines, and CI/CD. Primary concepts:

Workflow: top-level resource that describes the full job.
Template: reusable step definitions (container, script, DAG, or resource).
DAG (Directed Acyclic Graph): express dependencies between steps as a graph.
Artifacts: file passing between steps typically uses object stores (S3, GCS) or artifact repositories because each step runs in a separate pod.

The key difference from Tekton: Argo runs each step in its own pod, improving isolation at the cost of explicit artifact handling between steps.

An infographic titled "Argo Workflows: DAG Engine" explaining directed acyclic graph workflows with artifact passing. It shows colorful semicircular segments for core concepts — Workflow, Template, DAG, and Artifacts — with brief descriptions.

Argo Workflow example (DAG with build, test, deploy):

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: ci-pipeline-
spec:
  entrypoint: ci-pipeline
  templates:
  - name: ci-pipeline
    dag:
      tasks:
      - name: build
        template: build-image

      - name: test
        template: run-tests

      - name: deploy
        template: deploy-app
        dependencies:
        - build
        - test

The semantics are equivalent to Tekton’s runAfter/dependency behavior, but Argo’s per-step pod model requires artifact passing (e.g., upload build artifacts to an S3-compatible store, then download in downstream steps). Comparing Tekton and Argo Workflows Use this comparison to pick the right tool for your CI/CD needs:

Aspect	Tekton	Argo Workflows
Primary focus	CI/CD building blocks, pipelines as CRDs	General DAG engine for CI, CD, ML, batch
Execution model	Multiple steps in the same `Task` pod; shared filesystem via `Workspace` (PVC)	Each step runs in its own pod; artifacts passed via object storage
Data passing	`Workspace`/PVC (shared file system)	Artifacts (S3/GCS or artifact repo)
Use cases	Standard CI pipelines where steps benefit from a shared filesystem	Complex DAGs, ML/data pipelines, strong per-step isolation
Ecosystem & UI	Tekton Hub for reusable tasks; dashboard is lightweight	Rich built-in UI; extensive Argo ecosystem (ArgoCD, Argo Rollouts, Argo Events)
Best fit examples	Container builds, tests, and pushes that share a workspace	Parallel DAG tasks, ML pipelines, long-running batch jobs

When to choose which:

Choose Tekton if you want a CI-first solution where tasks share filesystem state (build artifacts, compiled binaries) and you prefer a modular, task-oriented API.
Choose Argo Workflows if you need complex DAG orchestration, strong isolation per step, or are orchestrating heterogeneous workloads (data processing, ML, CI/CD).

Integration with GitOps Both Tekton and Argo fit naturally into GitOps workflows:

Store pipeline definitions (CRDs) in Git and apply them via GitOps tooling.
Trigger pipelines from Git events and push build artifacts to registries or object stores.
Let your GitOps deployment tool (e.g., Argo CD) manage releasing the built artifacts to clusters.

Summary Kubernetes-native CI runs pipeline steps as pods and allows you to use Kubernetes primitives—RBAC, Secrets, resource limits, autoscaling—for your CI workloads. Tekton offers modular CI/CD building blocks with shared-pod Tasks and PVC-backed Workspaces. Argo Workflows is a powerful DAG engine that runs each step in its own pod and uses artifact passing to move data. Both are CRD-based, Git-friendly, and compatible with GitOps-driven delivery. Useful links and references

Tekton: https://tekton.dev/
Argo Workflows: https://argoproj.io/projects/argo-workflows/
Kaniko: https://github.com/GoogleContainerTools/kaniko
GitOps patterns and Argo CD: https://argo-cd.readthedocs.io/

Course Introduction

Platform Architecture and Infrastructure

GitOps and Continuous Delivery

Platform APIs and Self-Service Capabilities

Observability and Operations

Security and Policy Enforcement

Conclusion

CICD on Kubernetes Pipelines Tasks Artifacts and Flow

Watch Video