DEMO Monitor Flux using Prometheus Grafana

In this guide, you’ll learn how to integrate Prometheus with Flux v2 to scrape controller metrics and visualize them in Grafana. We’ll leverage the monitoring folder from the fluxcd/flux2 GitHub repo and deploy a PodMonitor CRD and Grafana dashboards via Flux Kustomizations.

1. Explore the monitoring directory

Clone or browse the Flux repository and locate the monitoring folder:

Inside monitoring/ you’ll find:

• PodMonitor YAML for scraping all Flux controllers.
• A dashboards/ folder with two Grafana JSON files:
  • cluster.json
  • controlplane.json

2. Apply the Flux PodMonitor

The PodMonitor CRD instructs Prometheus to scrape metrics from Flux controllers in the flux-system namespace:

apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: flux-system
  namespace: flux-system
  labels:
    app.kubernetes.io/part-of: flux
    app.kubernetes.io/component: monitoring
spec:
  namespaceSelector:
    matchNames:
      - flux-system
  selector:
    matchExpressions:
      - key: app
        operator: In
        values:
          - helm-controller
          - source-controller
          - kustomize-controller
          - notification-controller
          - image-automation-controller
          - image-reflector-controller
  podMetricsEndpoints:
    - port: http-prom
      relabelings: []

Apply it directly or via Flux:

kubectl apply -f monitoring/manifests/monitoring-config/podmonitor.yaml

3. Create a Flux Kustomization

Automate the deployment by defining a Flux Kustomization that points to your Git source:

apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: monitoring-config
  namespace: flux-system
spec:
  dependsOn:
    - name: monitoring-kustomization-prometheus-stack
  interval: 1h0m0s
  path: ./manifests/monitoring/monitoring-config
  prune: true
  sourceRef:
    kind: GitRepository
    name: monitoring-source-prometheus-stack

You can export this with the Flux CLI:

flux create kustomization monitoring-config \
  --namespace flux-system \
  --depends-on monitoring-kustomization-prometheus-stack \
  --interval 1h0m0s \
  --path "./manifests/monitoring/monitoring-config" \
  --prune=true \
  --source GitRepository/monitoring-source-prometheus-stack \
  --export > monitoring-config.yaml

Commit and push:

git add manifests/monitoring/monitoring-config
git commit -m "Add Flux PodMonitor and Grafana dashboards"
git push

Reconcile the Git source and kustomization:

flux reconcile source git flux-system --namespace flux-system
flux reconcile kustomization monitoring-config --namespace flux-system

Verify the PodMonitor is ready:

kubectl get podmonitor -n flux-system

4. Validate in Prometheus

Open your Prometheus UI and go to Status → Targets. Within seconds, the Flux controller endpoints should appear as UP:

Common gotk_ Queries

Run any query in Graph view to see real-time metrics:

Switch to Graph mode for time-series visualization:

5. Explore Grafana Dashboards

After Flux applies the dashboard JSON, refresh Grafana. Two dashboards are now available:

• Flux Control Plane
• Flux Cluster Stats

Flux Control Plane

Tracks each controller’s queue lengths, CPU/memory usage, API request rates, and reconciliation durations:

Flux Cluster Stats

Provides cluster-wide health: counts of reconcilers, failing controllers, manifest source statuses, operation durations, and readiness tables:

These dashboards ship out of the box—just apply them to get comprehensive visibility into your Flux GitOps workflows.

Links and References

• Flux v2 Documentation
• Prometheus Documentation
• Grafana Dashboards
• GitOps with Flux