> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Demo Fix Truncated Audit Output

> Explains fixing truncated LLM audit output by increasing max_tokens in a Python auditor, showing code changes, token guidance, and alternatives to reduce output length.

Hello and welcome back.

In this lesson we'll open our Python auditor and make the single change that prevents the LLM output from getting truncated. You'll see exactly where to change the `max_tokens` parameter, why it mattered in this case, and how to choose an appropriate value for your project.

Expose the Kubernetes cluster to the container (run locally in your terminal):

```bash theme={null}
kind get kubeconfig --name finops-cluster --internal > /tmp/kubeconfig-docker.yaml

docker run --rm -p 8000:8000 --network kind \
  -v /tmp/kubeconfig-docker.yaml:/home/appuser/.kube/config:ro \
  -e ENABLE_UNSAFE_STREAMABLE_HTTP_TRANSPORT=1 \
  -e PORT=8000 \
  -e HOST=0.0.0.0 \
  -e ALLOW_ONLY_READONLY_TOOLS=true \
  mcp/kubernetes:latest
```

Helpful references:

* [Kubernetes Basics](https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/)
* [kind — Kubernetes in Docker](https://kind.sigs.k8s.io/)
* [Docker](https://www.docker.com/)

Why the audit was truncated

* The earlier truncation was caused by the LLM response being capped by the `max_tokens` parameter passed to the agent call.
* To fix this, open `structured_auditor.py`, find the prompt definition and the `run_agent` call, and increase the `max_tokens` value (examples below).
* Always ensure the sum of prompt tokens and desired response tokens stays within your model's maximum context size.

The relevant prompt and the code that wires it into the agent (kept concise) are shown here:

```python theme={null}
PROMPT = """
1. List all namespaces.
2. For each application namespace (skip only namespaces listed in excluded_namespaces;
   also skip kube-* namespaces), fetch deployments, configmaps, and persistent volume claims.
3. Report each resource's name, namespace, kind, and labels exactly as returned by the tools.
4. Evaluate each resource against the tagging rules provided separately.
5. Flag resources that violate required tags or label mappings.
6. Summarize the full label audit in plain English, grouped by namespace. Cover every
   audited namespace and resource - do not truncate mid-report.

Evaluation rules (follow strictly; do not invent policy):
- A required tag is present if ANY key listed under that tag in label_mappings appears
  on the resource (for example, `app` satisfies `application`).
- Do not call a mapped key non-standard, legacy, or invalid when it is in label_mappings.
- Use only required_tags, label_mappings, cost_centers, environments, tiers,
  compliance_levels, resource_types, and excluded_namespaces from the rules.
- Do not invent exclusions, bonus labels, recommended tags, or extra commentary
  beyond what the rules define.

Use only data returned by the tools. Do not invent labels or resources.
"""

def load_tagging_rules() -> str:
    if not TAGGING_RULES_PATH.exists():
        raise FileNotFoundError(f"Tagging rules not found: {TAGGING_RULES_PATH}")
    return TAGGING_RULES_PATH.read_text()

async def main() -> None:
    print(
        await run_agent(
            PROMPT,
            tagging_rules=load_tagging_rules(),
            max_tokens=2048,
        )
    )

if __name__ == "__main__":
    asyncio.run(main())
```

Choose `max_tokens` based on the length of the expected output and your cost/latency trade-offs. If you need exhaustive, untruncated output, increase the limit; if you only need summaries, consider restructuring the prompt to produce shorter, structured outputs (e.g., JSON).

Updating the token limit

* In this demo I doubled the output token budget from `2048` to `4096`. Make sure the model you use supports that many tokens (prompt + output combined) or the request may be rejected or still truncated.
* Edit the `run_agent` call in `structured_auditor.py` to set `max_tokens=4096`:

```python theme={null}
async def main() -> None:
    print(
        await run_agent(
            PROMPT,
            tagging_rules=load_tagging_rules(),
            max_tokens=4096,
        )
    )

if __name__ == "__main__":
    asyncio.run(main())
```

Setting a large `max_tokens` increases cost and can impact latency. Also confirm your model's maximum context length (prompt + completion) before raising this value. Consider structured outputs (JSON), streaming, or limiting scope as alternatives.

After updating the file (e.g., open in Vim, edit `max_tokens`, save) and re-running the script, the audit output was no longer cut off and included the full resource-by-resource details and summary sections.

Excerpt of the improved audit output (as printed by the program):

```text theme={null}
**1. Deployment: analytics-collector**
- **Labels:** `app=analytics`
- **Analysis:**
  - ✅ `application` tag satisfied by `app`
  - ❌ Missing `owner` tag
  - ❌ Missing `environment` tag
  - ❌ Missing `cost-center` tag
  - ❌ Missing `tier` tag
  - ❌ Missing `slack-channel` tag
- **Status:** ❌ Non-compliant (5 of 6 required tags missing)

**2. ConfigMap: analytics-config**
- **Labels:** None
- **Violations:** Missing all 6 required tags
- **Status:** ❌ Non-compliant

**3. ConfigMap: kube-root-ca.crt**
- **Labels:** None
- **Violations:** Missing all 6 required tags
- **Status:** ❌ Non-compliant

**4. ConfigMap: untracked-config**
- **Labels:** None
- **Violations:** Missing all 6 required tags
- **Status:** ❌ Non-compliant

**5. PersistentVolumeClaim: analytics-data**
- **Labels:** None
- **Violations:** Missing all 6 required tags
- **Status:** ❌ Non-compliant
```

The report now includes the full tag coverage and critical observations:

* Fully Compliant Resources: 0
* Non-Compliant Resources: 27 (100%)

## Tag Coverage Analysis

| Required Tag | Present | Missing | Coverage |
| - | -: | -: | -: |
| owner | 7 | 20 | 26% |
| environment | 11 | 16 | 41% |
| cost-center | 8 | 19 | 30% |
| application | 14 | 13 | 52% |
| tier | 10 | 17 | 37% |
| slack-channel | 0 | 27 | 0% |

### Critical Observations

1. Slack-channel Tag: This tag is completely absent across all 27 resources (0% coverage). This is a critical gap for FinOps governance and operational alerting.
2. System-Generated Resources: All `kube-root-ca.crt` ConfigMaps across namespaces lack any labels. These are system-generated resources but still appear in the audit scope.
3. Inventory Namespace Excellence: The inventory namespace demonstrates the highest compliance with 5 of 6 required tags present on most resources. Only the `slack-channel` tag is missing.
4. Payment Namespace Inconsistency: The payment namespace shows inconsistent labeling patterns. Some resources have owner and environment tags while others lack application tags entirely.
5. Default Namespace: The default namespace contains multiple untracked resources (analytics-collector, untracked-config, analytics-data) with minimal or no labels, suggesting these may be orphaned or legacy resources.

Before increasing `max_tokens` further, consider these alternatives to reduce output length or avoid higher costs:

* Request machine-readable output (JSON) so the agent produces concise, parseable results.
* Stream results incrementally if supported by your LLM client.
* Limit audit scope (fewer namespaces or resource types) to reduce output size.
* Summarize results at higher level and request drilling only for problematic namespaces/resources.

Guidance for picking `max_tokens`

| Use Case | Typical Approach |
| - | - |
| Short, high-level summaries | `max_tokens` \<= 512 |
| Detailed per-resource audits (moderate cluster) | `max_tokens` 1024–4096 |
| Large clusters, full natural-language reports | `max_tokens` > 4096 (only if model supports it) |

For model-specific token/context limits and best practices, check your LLM provider's documentation (for example, see the model token limits in your provider's docs).

A future whiteboarding session will decide the audit output format (human-readable vs. structured) and discuss downstream automation for remediation and reporting.

> **[Watch Video](https://learn.kodekloud.com/user/courses/building-kubernetes-finops-ai-agent/module/5045eac8-8b03-4ecd-9213-d50eda3353c5/lesson/b86ab09f-c8ed-425b-a428-b1c7d8033fff)**

> **[Practice Lab](https://learn.kodekloud.com/user/courses/building-kubernetes-finops-ai-agent/module/5045eac8-8b03-4ecd-9213-d50eda3353c5/lesson/c1442fe1-43f8-4d8d-963e-530d56c63cf1)**


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.