This guide explains how to use Prometheus and Alertmanager to monitor ArgoCD metrics and automate alerting for synchronization issues.
In this guide, we explore how Prometheus can leverage ArgoCD metrics to raise alerts using Alertmanager—automating the detection of synchronization issues in ArgoCD applications.
Alertmanager handles alerts generated by client applications, such as the Prometheus server. In this article, we explain how to configure alerting rules based on ArgoCD metrics within a Prometheus instance. Previously, we examined how Prometheus scrapes these metrics; now, we will use them to define alert rules for detecting synchronization problems in ArgoCD applications.As part of the Prometheus Operator, configuration details are maintained for both Alertmanager and Prometheus rules. The Prometheus Rules custom resource enables you to define both recording and alerting rules for your Prometheus instance.Below is an example of a PrometheusRule configuration containing a single alerting rule:
To verify the applied configuration, run the following commands. These commands connect to the Prometheus pod and display the contents of the rules configuration file:
Alert name: The identifier for the alert (e.g., “ArgoApplicationOutOfSync”).
Expression: The condition that triggers the alert. In this example, the alert fires when an ArgoCD application’s synchronization status is out of sync (argocd_app_info{sync_status="OutOfSync"} == 1).
Duration: The period (5 minutes in this instance) the condition must persist before the alert triggers.
Labels: Metadata tags for routing and categorizing alerts, such as a severity level set to “warning”.
Annotations: Additional information, including a summary message to describe the alert.
When the Prometheus rule is created, the Prometheus Operator automatically generates the corresponding configuration for Prometheus using the Prometheus Rule custom resource definition (CRD). It then uses the Config Reloader component to update the rules file accordingly.Below is an updated version of the PrometheusRule configuration (functionally identical to the previous example but formatted properly):
After updating the rules file, check the Rules tab in the Prometheus UI. If an ArgoCD application’s synchronization status deviates from the expected state, the alert will trigger and be visible in the Alertmanager UI.