The problem with traditional CD
In a typical traditional CD workflow, a pipeline runskubectl apply and pushes manifests into the cluster. You see a green status, close your laptop, and move on:
kubectl edit to ramp up replicas during a traffic spike. These manual or ad-hoc changes create “drift”: the live cluster no longer matches what’s stored in Git.
Example — a quick server-side change:
kubectl edit tweak:
Why that matters
Traditional CD pipelines are good at pushing changes, but they don’t guarantee the cluster will remain in the declared state over time or that the cluster state will be visible (and auditable) to the whole team. GitOps flips the direction: instead of pushing desired state into the cluster, the cluster pulls desired state from Git and continuously reconciles itself to match the repo.
How GitOps works (in-cluster pull + reconcile)
Install an agent (for example, Argo CD or Flux) inside the cluster. The agent periodically pulls the desired state from Git and asks one question on a loop: Does the cluster match the repo?- If yes: nothing to do.
- If no: the agent either corrects the cluster to match Git or raises an alert for manual remediation.
The real power of GitOps is the agent’s pull-and-reconcile loop. Treat Git as the single source of truth and let the in-cluster agent enforce that state.
Key benefits of GitOps
- Drift becomes visible and actionable. Manual changes are either reverted or flagged immediately.
- Rollbacks are simple: revert the commit in Git and the agent will reconcile the cluster back to that state—no special redeploy pipeline needed.
- Auditability and change history live naturally in Git (commit history, PRs, code review).
- Security posture improves because write access to the cluster can be limited while Git remains the edit surface.
Common anti-patterns and warnings
A frequent mistake is installing Argo CD or Flux but continuing to push changes directly from CI/CD (e.g., Jenkins) into the cluster. That still allows drift and undermines GitOps guarantees.If CI/CD pipelines continue to write directly to the cluster while an in-cluster GitOps agent is active, you can reintroduce drift and break auditability. Make Git the only source of truth for desired state.
Quick comparison
Practical advice / best practices
- Make Git the only place to change manifests or Kustomize/Helm inputs. Use pull requests, reviews, and CI checks.
- Use the in-cluster agent for reconciliation; configure it to either auto-sync or require manual approval depending on risk tolerance.
- Limit direct access to the cluster (kubectl, SSH) and route fixes through Git (or have a documented emergency process).
- Integrate image scanners, policy checks (e.g., OPA/Gatekeeper), and CI tests into the PR flow before merging into the desired branch.
Further reading
- Argo CD: https://argo-cd.readthedocs.io/
- Flux: https://fluxcd.io/
- Kubernetes GitOps guide: https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/