- Prometheus alerting: https://prometheus.io/docs/alerting/latest/
- Alertmanager: https://prometheus.io/docs/alerting/latest/alertmanager/
- Go templates: https://pkg.go.dev/text/template
Labels — machine-readable metadata
Include alabels: section in your rule to attach key/value pairs that identify the alert and guide Alertmanager behaviour (routing, grouping, inhibition, etc.).
Example:
Annotations — human-readable context
Annotations provide descriptive information that is not part of the alert identity and therefore is not used for matching or routing. Use annotations for:- Descriptions
- Runbook links or steps
- Helpful context for responders
- Any text you want included in notifications
.CommonLabels, .CommonAnnotations, and .Alerts are available there).
Use
labels for machine-oriented metadata (routing, matching). Use annotations for human-oriented information (descriptions, runbooks).Template variables — how to access label and value fields
You can reference label sets, specific labels, and the alerting value inside Go templates. Wrap template examples in backticks when displayed inline.
Example template snippets:
Complete alert rule example with labels and annotations
Below is a full rule group demonstratinglabels and an annotation that uses Go templating to include label values and the alert value.
Notes and best practices
- Use
labelswhen you need Alertmanager to make decisions (routing, grouping, inhibition). - Use
annotationsfor textual context or actionable instructions; they are ideal for notification messages (email, SMS, Slack). - Keep labels concise and stable (avoid high-cardinality or changing labels that would fragment routing).
- Use annotations to include runbook URLs, remediation steps, or escalation instructions.
- When referencing labels in templates, prefer
{{ .Labels.<name> }}for valid identifiers and{{ index .Labels "label-name" }}for names with characters not allowed in Go identifiers. - You can access
{{ .Annotations }}inside templates when constructing notification messages.
Quick comparison
Links and references
- Prometheus alerting: https://prometheus.io/docs/alerting/latest/
- Alertmanager: https://prometheus.io/docs/alerting/latest/alertmanager/
- Go templating: https://pkg.go.dev/text/template