Skip to main content
This guide explains how ArgoCD exposes metrics and how the Prometheus Operator discovers and configures Prometheus to scrape those endpoints using custom resources such as ServiceMonitor and PodMonitor. Follow the sequence below to verify and configure scraping for ArgoCD metrics and to visualize them in Grafana. Key concepts covered:
  • How ArgoCD exposes metrics via Services
  • How the Prometheus Operator uses ServiceMonitor/PodMonitor CRs to generate scrape configs
  • Verifying the generated Prometheus configuration and troubleshooting common issues
  • Visualizing ArgoCD metrics in Grafana
Overview of the flow
  1. Install the Prometheus Operator and its CRDs (ServiceMonitor / PodMonitor / Prometheus).
  2. ArgoCD exposes metrics through Services that use a named metrics port and identifiable labels.
  3. Create ServiceMonitor resources that select those Services and define scrape endpoints.
  4. The Prometheus CR selects matching ServiceMonitors and the operator generates Prometheus scrape_configs.
  5. The operator triggers a config-reloader sidecar that updates Prometheus and begins scraping the ArgoCD endpoints.
Ensure the Prometheus Operator and its CRDs are installed in the cluster before applying any ServiceMonitor or PodMonitor resources; otherwise these custom resources will not be recognized.

Step-by-step details

1) Inspect the generated Prometheus configuration

The Prometheus Operator writes generated configuration files into the Prometheus pod via a config-reloader sidecar. To inspect the generated config from the config-reloader container:
Note: pod names, container names, and paths can vary by distribution — adjust prometheus-0, config-reloader, and the file path to match your deployment. Example snippet from a generated Prometheus config:
This shows Prometheus scraping both Kubernetes components and ServiceMonitor-discovered jobs for ArgoCD.

2) The Service: how ArgoCD exposes metrics

ArgoCD deploys Services that expose metrics on a named port (commonly metrics). Confirm the Service exists and inspect it:
Representative Service manifest:
Important:
  • The Service must expose the metrics port with a name (e.g., metrics).
  • The Service should have labels that a ServiceMonitor can match (see next section).
  • Ensure the Service’s namespace is within the scope of the Prometheus CR’s namespaceSelector.

3) ServiceMonitor: instruct the operator how to scrape

Create a ServiceMonitor that selects the Service by its labels and references the named metrics port. The Prometheus Operator recognizes ServiceMonitors and will include them if the Prometheus CR selects them. Example ServiceMonitor (adjust namespace/labels to your setup):
Notes:
  • selector.matchLabels selects the Service resource by its labels (not the Service’s pod selector).
  • endpoints.port should reference the named port (metrics) whenever possible.
  • You can configure additional scrape settings inside endpoints (path, scheme, honor_labels, relabeling, etc.).

4) Prometheus CR and automatic config reload

The Prometheus CR (managed by the operator) uses label selectors to discover ServiceMonitors. When a matching ServiceMonitor is found:
  • The operator generates the appropriate scrape_configs.
  • The operator writes those configurations into the Prometheus config volume.
  • A config-reloader sidecar reloads Prometheus with the new configuration automatically.
Make sure the Prometheus CR’s namespaceSelector includes the namespace where the ServiceMonitor is created (common pitfall). Verify the ServiceMonitor exists:
Re-check the generated Prometheus configuration (as in Step 1) to confirm job_name entries for your ServiceMonitors appear.
If a ServiceMonitor is created before the Prometheus Operator and its CRDs are installed, the resource may be ignored or not processed. Always install the operator and CRDs first, then apply monitoring CRs.

5) Visualize ArgoCD metrics in Grafana

Once Prometheus is scraping ArgoCD metrics, point Grafana at your Prometheus data source and create dashboards or import community dashboards for ArgoCD. Useful metric types include:
  • Application sync status and durations
  • Application health and reconciliation counts
  • Controller queue sizes and reconcile errors
Grafana sources:
  • Use the Prometheus URL as the Grafana data source.
  • Search for community dashboards (e.g., “ArgoCD”) to import panels and templates.

Quick reference table

ResourcePurposeExample/Command
ServiceExposes ArgoCD metrics via a named portkubectl get svc argocd-server-metrics -n argocd -o yaml
ServiceMonitorInstructs Prometheus Operator how to scrape a ServiceSee ServiceMonitor manifest above
Prometheus CROperator-managed Prometheus instance that selects ServiceMonitorsCheck Prometheus CR for namespaceSelector and label selectors
Config reloaderSidecar that reloads Prometheus when the config is updatedkubectl exec -it prometheus-0 -c config-reloader -- cat /etc/prometheus/config_out/prometheus.env.yaml

Troubleshooting tips

  • Confirm CRDs are installed: kubectl get crd | grep servicemonitor should show servicemonitors.monitoring.coreos.com.
  • Check operator logs for errors: kubectl logs -l app=prometheus-operator -n <operator-namespace>
  • Verify labels: kubectl get svc argocd-server-metrics -n argocd --show-labels
  • Ensure Prometheus namespaceSelector includes the namespace where ServiceMonitor lives.
Wrapping up
  • ArgoCD exposes metrics via Services with named ports and labels.
  • The Prometheus Operator discovers those Services via ServiceMonitor (or PodMonitor) CRs and auto-generates Prometheus configs.
  • Ensure correct labels, port names, and namespace selection so the operator picks up your monitoring resources.
  • Use Grafana to build dashboards from the scraped ArgoCD metrics.

Watch Video