Demo Fix Truncated Audit Output

Hello and welcome back. In this lesson we’ll open our Python auditor and make the single change that prevents the LLM output from getting truncated. You’ll see exactly where to change the max_tokens parameter, why it mattered in this case, and how to choose an appropriate value for your project. Expose the Kubernetes cluster to the container (run locally in your terminal):

kind get kubeconfig --name finops-cluster --internal > /tmp/kubeconfig-docker.yaml

docker run --rm -p 8000:8000 --network kind \
  -v /tmp/kubeconfig-docker.yaml:/home/appuser/.kube/config:ro \
  -e ENABLE_UNSAFE_STREAMABLE_HTTP_TRANSPORT=1 \
  -e PORT=8000 \
  -e HOST=0.0.0.0 \
  -e ALLOW_ONLY_READONLY_TOOLS=true \
  mcp/kubernetes:latest

Helpful references:

Why the audit was truncated

The earlier truncation was caused by the LLM response being capped by the max_tokens parameter passed to the agent call.
To fix this, open structured_auditor.py, find the prompt definition and the run_agent call, and increase the max_tokens value (examples below).
Always ensure the sum of prompt tokens and desired response tokens stays within your model’s maximum context size.

The relevant prompt and the code that wires it into the agent (kept concise) are shown here:

PROMPT = """
1. List all namespaces.
2. For each application namespace (skip only namespaces listed in excluded_namespaces;
   also skip kube-* namespaces), fetch deployments, configmaps, and persistent volume claims.
3. Report each resource's name, namespace, kind, and labels exactly as returned by the tools.
4. Evaluate each resource against the tagging rules provided separately.
5. Flag resources that violate required tags or label mappings.
6. Summarize the full label audit in plain English, grouped by namespace. Cover every
   audited namespace and resource - do not truncate mid-report.

Evaluation rules (follow strictly; do not invent policy):
- A required tag is present if ANY key listed under that tag in label_mappings appears
  on the resource (for example, `app` satisfies `application`).
- Do not call a mapped key non-standard, legacy, or invalid when it is in label_mappings.
- Use only required_tags, label_mappings, cost_centers, environments, tiers,
  compliance_levels, resource_types, and excluded_namespaces from the rules.
- Do not invent exclusions, bonus labels, recommended tags, or extra commentary
  beyond what the rules define.

Use only data returned by the tools. Do not invent labels or resources.
"""

def load_tagging_rules() -> str:
    if not TAGGING_RULES_PATH.exists():
        raise FileNotFoundError(f"Tagging rules not found: {TAGGING_RULES_PATH}")
    return TAGGING_RULES_PATH.read_text()

async def main() -> None:
    print(
        await run_agent(
            PROMPT,
            tagging_rules=load_tagging_rules(),
            max_tokens=2048,
        )
    )

if __name__ == "__main__":
    asyncio.run(main())

Choose max_tokens based on the length of the expected output and your cost/latency trade-offs. If you need exhaustive, untruncated output, increase the limit; if you only need summaries, consider restructuring the prompt to produce shorter, structured outputs (e.g., JSON).

Updating the token limit

In this demo I doubled the output token budget from 2048 to 4096. Make sure the model you use supports that many tokens (prompt + output combined) or the request may be rejected or still truncated.
Edit the run_agent call in structured_auditor.py to set max_tokens=4096:

async def main() -> None:
    print(
        await run_agent(
            PROMPT,
            tagging_rules=load_tagging_rules(),
            max_tokens=4096,
        )
    )

if __name__ == "__main__":
    asyncio.run(main())

Setting a large max_tokens increases cost and can impact latency. Also confirm your model’s maximum context length (prompt + completion) before raising this value. Consider structured outputs (JSON), streaming, or limiting scope as alternatives.

After updating the file (e.g., open in Vim, edit max_tokens, save) and re-running the script, the audit output was no longer cut off and included the full resource-by-resource details and summary sections. Excerpt of the improved audit output (as printed by the program):

**1. Deployment: analytics-collector**
- **Labels:** `app=analytics`
- **Analysis:**
  - ✅ `application` tag satisfied by `app`
  - ❌ Missing `owner` tag
  - ❌ Missing `environment` tag
  - ❌ Missing `cost-center` tag
  - ❌ Missing `tier` tag
  - ❌ Missing `slack-channel` tag
- **Status:** ❌ Non-compliant (5 of 6 required tags missing)

**2. ConfigMap: analytics-config**
- **Labels:** None
- **Violations:** Missing all 6 required tags
- **Status:** ❌ Non-compliant

**3. ConfigMap: kube-root-ca.crt**
- **Labels:** None
- **Violations:** Missing all 6 required tags
- **Status:** ❌ Non-compliant

**4. ConfigMap: untracked-config**
- **Labels:** None
- **Violations:** Missing all 6 required tags
- **Status:** ❌ Non-compliant

**5. PersistentVolumeClaim: analytics-data**
- **Labels:** None
- **Violations:** Missing all 6 required tags
- **Status:** ❌ Non-compliant

The report now includes the full tag coverage and critical observations:

Fully Compliant Resources: 0
Non-Compliant Resources: 27 (100%)

Tag Coverage Analysis

Required Tag	Present	Missing	Coverage
owner	7	20	26%
environment	11	16	41%
cost-center	8	19	30%
application	14	13	52%
tier	10	17	37%
slack-channel	0	27	0%

Critical Observations

Slack-channel Tag: This tag is completely absent across all 27 resources (0% coverage). This is a critical gap for FinOps governance and operational alerting.
System-Generated Resources: All kube-root-ca.crt ConfigMaps across namespaces lack any labels. These are system-generated resources but still appear in the audit scope.
Inventory Namespace Excellence: The inventory namespace demonstrates the highest compliance with 5 of 6 required tags present on most resources. Only the slack-channel tag is missing.
Payment Namespace Inconsistency: The payment namespace shows inconsistent labeling patterns. Some resources have owner and environment tags while others lack application tags entirely.
Default Namespace: The default namespace contains multiple untracked resources (analytics-collector, untracked-config, analytics-data) with minimal or no labels, suggesting these may be orphaned or legacy resources.

Before increasing max_tokens further, consider these alternatives to reduce output length or avoid higher costs:

Request machine-readable output (JSON) so the agent produces concise, parseable results.
Stream results incrementally if supported by your LLM client.
Limit audit scope (fewer namespaces or resource types) to reduce output size.
Summarize results at higher level and request drilling only for problematic namespaces/resources.

Guidance for picking max_tokens

Use Case	Typical Approach
Short, high-level summaries	`max_tokens` <= 512
Detailed per-resource audits (moderate cluster)	`max_tokens` 1024–4096
Large clusters, full natural-language reports	`max_tokens` > 4096 (only if model supports it)

For model-specific token/context limits and best practices, check your LLM provider’s documentation (for example, see the model token limits in your provider’s docs). A future whiteboarding session will decide the audit output format (human-readable vs. structured) and discuss downstream automation for remediation and reporting.

Introduction

The Problem Statement

Introduction to FinOps

Setting Up SkyLine Air Kubernetes Services

MCP Gateway for Kubernetes

LangChain MCP Agents

LLM Policy-Aware Label Audit

From Findings to Actionable Tickets

Demo Fix Truncated Audit Output

Tag Coverage Analysis

Critical Observations

Watch Video

Practice Lab

​Tag Coverage Analysis

​Critical Observations

Watch Video

Practice Lab

Tag Coverage Analysis

Critical Observations