- A running kind cluster used by the lab (example name:
finops-cluster). - Docker available to run the MCP service.
- A Python virtual environment with the auditor scripts installed.
- (Optional) Familiarity with Kubernetes labeling/tagging best practices: Kubernetes Basics
Tip: use three terminals while experimenting — one for MCP, one for the auditor script, and one to interactively inspect cluster resources or logs.
When practicing in the lab, you will typically use three terminals so you can run the MCP server, the auditor scripts, and inspect outputs interactively.
- Start the MCP service Run MCP in a Docker container and mount the kubeconfig produced for the kind cluster. The command below sets environment variables required for the MCP image used in the lab.
- Run the baseline Python auditor This “auditor” script connects to MCP, enumerates available tools, and queries namespaces/resources. It prints the raw LLM output as plain text without enforcing tagging rules or a strict output schema.
AIRLINE Namespace
BOOKING-API Namespace
DEFAULT Namespace
(The baseline auditor continues similarly for other namespaces.)
Why this matters: the baseline output lacks explicit tagging rules and a strict schema, so it can be inconsistent and harder to use in automated remediation workflows.
- Run the structured auditor (policy-aware) The structured auditor passes the tagging rules and a system prompt to the LLM before sending resource data. This enforces a consistent format and makes the output machine-friendly.
Example structured outputs (trimmed):
- Violations: Missing application, slack-channel
- Status: ❌ FAIL (4 of 6 required tags present)
Resource: ConfigMap / kube-root-ca.crt
- Labels: None
- Violations: Missing all 6 required tags
- Status: ❌ FAIL
Resource: ConfigMap / payment-config
- Labels:
- environment=prod
- owner=payments-team
- Violations: Missing application, cost-center, tier, slack-channel
- Status: ❌ FAIL (2 of 6 required tags present)
Resource: ConfigMap / payment-gateway-api-config
- Labels: app=payment-gateway-api
- Violations: Missing owner, environment, cost-center, tier, slack-channel
- Status: ❌ FAIL (1 of 6 required tags present)
Resource: ConfigMap / payment-gateway-ui-config
- Labels: app=payment-gateway
- Violations: Missing owner, environment, cost-center, tier, slack-channel
- Status: ❌ FAIL (1 of 6 required tags present)
Resource: PersistentVolumeClaim / payment-logs
- Labels:
- environment=prod
- owner=payments-team
- Violations: Missing application, cost-center, tier, slack-channel
- Status: ❌ FAIL (2 of 6 required tags present)
Compiled audit summary (example produced by the structured auditor)
KUBERNETES RESOURCE LABEL AUDIT REPORT
FinOps Governance Assessment
Audit Date: 2026-07-18Namespaces Audited: 5 application namespaces
Total Resources Evaluated: 26
Excluded Namespaces: kube-system, kube-public, kube-node-lease, local-path-storage
NAMESPACE: airline
Resource: ConfigMap / kube-root-ca.crt
- Labels: None
- Violations: Missing all 6 required tags (owner, environment, cost-center, application, tier, slack-channel)
- Status: ❌ FAIL
Resource: PersistentVolumeClaim / orphaned-data-pvc
- Labels: app=old-service
- Violations:
- Missing: owner, environment, cost-center, tier, slack-channel
- Present: application (via
applabel)
- Status: ❌ FAIL (5 of 6 required tags missing)
NAMESPACE: booking-api
Resource: Deployment / booking-api
- Labels:
- app.kubernetes.io/name=booking-api
- app.kubernetes.io/tier=backend
- cost-center=booking-engine
- environment=prod
- Violations: Missing owner and slack-channel
- Status: ❌ FAIL (4 of 6 required tags present)
NAMESPACE: flight-search
Resource: Deployment / flight-search-service
- Labels:
- app=flight-search
- app.kubernetes.io/tier=backend
- environment=prod
- Violations: Missing owner, cost-center, slack-channel
- Status: ❌ FAIL (3 of 6 required tags present)
Resource: ConfigMap / flight-search-config
- Labels: app=flight-search
- Violations: Missing owner, environment, cost-center, tier, slack-channel
- Status: ❌ FAIL (1 of 6 required tags present)
NAMESPACE: inventory
Resource: Deployment / inventory-service
- Labels:
- app.kubernetes.io/name=inventory-service
- app.kubernetes.io/tier=backend
- cost-center=inventory
- criticality=P2
- environment=prod
- owner=inventory-team
- Violations: Missing slack-channel
- Status: ⚠️ PARTIAL (5 of 6 required tags present)
Resource: ConfigMap / inventory-config
- Labels:
- app.kubernetes.io/name=inventory-service
- app.kubernetes.io/tier=backend
- compliance-level=standard
- cost-center=inventory
- environment=prod
- owner=inventory-team
- Violations: Missing slack-channel
- Status: ⚠️ PARTIAL (5 of 6 required tags present)
NAMESPACE: payment
Resource: Deployment / payment-gateway
- Labels:
- app=payment-gateway
- environment=prod
- owner=payments-team
- Violations: Missing cost-center, tier, slack-channel, application (if not derivable)
- Status: ❌ FAIL / PARTIAL (depending on application derivation)
- Feeding tagging rules and a clear system prompt to the LLM yields consistent, machine-friendly audit output.
- The structured approach ignores unrelated tags and evaluates resources strictly according to governance rules, improving remediation workflows.
- The structured auditor is better suited for automation and reporting (JSON/CSV export is simpler with a consistent schema).
If your structured auditor output appears truncated or stops mid-report with no error, check the LLM configuration for the max tokens (or response length) parameter. A too-small max tokens setting can cause the LLM to truncate long audit reports. Increase the token limit to ensure the LLM can return the complete report.
- The agent logic completed successfully, but the LLM response was capped by the configured
max_tokens(or equivalent) setting. The truncated output shows the audit logic worked, but the LLM could not return the entire report. Increase the token limit in your LLM client or script to get full reports.
- Increase the LLM token/response-size settings and re-run the structured auditor to retrieve full reports.
- Export the structured audit results to JSON or CSV for integration with ticketing/automation systems.
- Extend tagging rules to include environment-specific or team-specific policies and re-run to validate compliance.
- Kubernetes Documentation: https://kubernetes.io/docs/
- kind: https://kind.sigs.k8s.io/