What a reconciler does (high level)
A reconciler continuously compares the desired state (from Git) with the cluster’s live state and applies changes until they match. Where the reconciler runs (inside the target cluster or outside it) determines the operational model and affects security, network requirements, and credential management.In-cluster reconcilers
In-cluster reconcilers run inside the Kubernetes cluster they manage.- Behavior: The reconciler pulls manifests from Git and applies changes using the cluster’s local Kubernetes API.
- Examples: Argo CD (when deployed inside the target cluster), Flux CD (as an operator).
- Benefits:
- No need to expose cluster credentials outside the cluster.
- Reduced network exposure because reconciliation is performed locally.
- Simpler authentication: reconciler uses in-cluster service account credentials.
- Typical flow:
- Reconciler (running inside the cluster) pulls manifests from Git.
- It compares manifests to the cluster’s live state using the local API server.
- It applies changes by creating/updating Kubernetes resources locally.
In-cluster reconcilers follow a pull model: they fetch desired state from Git and apply it via the cluster’s local API, minimizing external credential handling and network exposure.
External reconcilers
External reconcilers run outside the target Kubernetes cluster. Common examples include CI/CD runners, management VMs, or controllers in a separate management cluster.- Behavior: The reconciler either pulls desired state from Git or receives artifacts from a pipeline, then connects over the network to the target cluster’s API server and applies changes.
- Example deployments: a Jenkins pipeline that applies manifests to production clusters, or a central management cluster that controls multiple tenant clusters.
- Trade-offs:
- Requires storing and managing cluster credentials outside the target cluster (e.g.,
kubeconfig, service account tokens), which increases operational complexity and attack surface. - Requires network access to the target clusters’ API servers.
- Requires storing and managing cluster credentials outside the target cluster (e.g.,
- Typical flow:
- External reconciler obtains manifests (pulls from Git or receives them from a pipeline).
- It uses externally stored credentials to call the target cluster’s API server.
- It applies changes remotely.
External reconcilers require externally stored Kubernetes credentials and network access to the API server. Protect and rotate credentials (e.g.,
kubeconfig, tokens) and restrict API access to minimize risk.How Argo CD fits in
Argo CD is primarily deployed as an in-cluster reconciler: its components (including the Application Controller) normally run inside a Kubernetes cluster. However, Argo CD is built to manage applications across multiple clusters.- When Argo CD manages a remote (external) cluster, the Argo CD instance runs in its own cluster (the “management” cluster). For each registered external cluster, Argo CD stores credentials (for example,
kubeconfigor a service account token) and uses them to connect to that cluster’s API server. - In other words:
- Argo CD is an in-cluster reconciler in the cluster where it is deployed.
- Relative to any external target cluster it manages, Argo CD behaves like an external reconciler because it connects remotely to that cluster’s API to apply changes.
- Key point: Reconciliation is centralized in Argo CD’s controllers, but applying manifests to remote clusters requires Argo CD to hold and use credentials for those clusters—so credential handling and API connectivity are necessary for multi-cluster management.
Comparison summary
| Pattern | Runs where | Credential handling | Network requirements | Operational considerations |
|---|---|---|---|---|
| In-cluster reconciler | Inside the target cluster | Uses in-cluster service accounts / no external kubeconfigs | Local API access only — lower network exposure | Simpler security model; ideal when controllers can be hosted in each target cluster |
| External reconciler | Outside the target cluster | Requires stored kubeconfig or tokens outside the cluster | Must reach API server over the network | Centralized control possible; requires secure credential storage and robust API access controls |
Choosing a pattern: key factors
Consider these when selecting a reconciler deployment model:- Security posture for managing cluster credentials (local service accounts vs. external
kubeconfig). - Network topology and API server accessibility (firewalls, NAT, private clusters).
- Operational preferences: decentralized (deploy controllers in each cluster) vs. centralized management (single management cluster or CI system).
- Scale and multi-tenancy: central management may simplify operations across many clusters, but increases credential management overhead.
Final notes
Both patterns are valid and commonly used in GitOps workflows. In-cluster reconcilers reduce credential sprawl and network exposure, while external reconcilers allow centralized pipelines and cross-cluster control. Argo CD supports both scenarios by running as an in-cluster controller and connecting remotely to registered clusters when needed.Links and references
- GitOps with ArgoCD
- GitOps with FluxCD
- Jenkins course example
- Kubernetes concepts: https://kubernetes.io/docs/concepts/
-
Best practice: store and rotate external credentials (
kubeconfig, tokens) and restrict API server access using network policies and firewall rules.