Demo Whiteboarding SkyLine Air Kubernetes Services

Hello and welcome back. One important habit to cultivate is whiteboarding. Architects and engineers (both senior and junior) routinely sketch and iterate on designs together. We’ll adopt the same whiteboarding mindset to design and clarify how components in a Kubernetes-based system exchange metadata. Overview

Cluster with multiple namespaces (example: booking-api, payment, search, kube-system).
Conceptually treat the cluster as a container for namespaces and workloads.
For design notes, group namespaces and use labels or color-coding to mark environments (e.g., prod, staging, dev) or functional domains.

What we already have

A Kubernetes cluster with multiple namespaces and workloads.
Common example namespaces: booking-api, payment, search, and kube-system.

Design tip

When whiteboarding, layout the cluster first, then place namespaces, services, and any management/control-plane components around it. Annotate where credentials live (kubeconfig, service accounts) and where audit/logging will be captured.

What we need next We want to collect Kubernetes metadata — such as labels, annotations, and counts of objects — for use in discovery, filtering, observability, or downstream automation. Two common ways to retrieve Kubernetes metadata

Direct, traditional tooling (CLI + libraries)
Prompt-driven retrieval via a Management Control Plane (MCP) that can optionally call an LLM for summarization or natural-language responses

Comparison: direct tooling vs MCP (at-a-glance)

Aspect	Direct (kubectl / client libraries)	MCP + LLM (prompt-driven)
Typical users	Devs, SREs, automation scripts	App owners, non-kubernetes users, chatOps
Access pattern	Direct API queries via kubeconfig / in-cluster creds	One endpoint that queries cluster on behalf of users
Complexity	Low to medium	Medium to higher (auth, filtering, model integration)
Auditability	Native kubectl logs & kube-apiserver audit	Requires additional MCP logging & controls
Use cases	Ad-hoc queries, scripts, tools	Natural-language queries, aggregated summaries, policy-driven views

Direct, traditional tooling

Use kubectl with simple text processing tools for ad-hoc queries.
Use official client libraries (Python, Go, etc.) for programmatic access.

Example CLI commands:

# Show labels for all namespaces
kubectl get namespaces --show-labels

# Show labels for a specific namespace
kubectl get namespace booking-api -o jsonpath='{.metadata.labels}'

# Show labels for pods in a namespace
kubectl get pods -n booking-api --show-labels

Example using the official Python client:

from kubernetes import client, config

# For local development, load kubeconfig from the default location:
config.load_kube_config()
# For in-cluster execution (running inside a pod), use:
v1 = client.CoreV1Api()
ns = v1.read_namespace('booking-api')
print(ns.metadata.labels)

Example using client-go in Go:

import (
    "context"
    "fmt"
    "k8s.io/client-go/kubernetes"
    "k8s.io/client-go/tools/clientcmd"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

func main() {
    config, _ := clientcmd.BuildConfigFromFlags("", "/path/to/kubeconfig")
    clientset, _ := kubernetes.NewForConfig(config)
    ns, _ := clientset.CoreV1().Namespaces().Get(context.TODO(), "booking-api", metav1.GetOptions{})
    fmt.Println(ns.Labels)
}

When to pick direct tooling

You need precise, programmatic access or tight integration into CI/CD.
Low-latency or frequent queries are required.
You want to avoid the added complexity of an intermediate control plane.

Prompt-driven retrieval via an MCP + LLM

Expose a Management Control Plane (MCP) endpoint that accepts prompts or structured queries.
The MCP uses available credentials (kubeconfig or in-cluster identity) to query the Kubernetes API and collect metadata.
Optionally, the MCP passes results to an LLM for summarization, filtering, or to produce a natural-language reply.

High-level MCP + LLM flow

A user or automation submits a prompt to the MCP (example: “List namespaces with label environment=prod and their pod counts”).
The MCP authenticates to the cluster with its kubeconfig or in-cluster credentials and queries the API.
The MCP optionally sanitizes or pre-processes the metadata, then may forward it to an LLM for summarization or to produce a user-friendly response.
The MCP returns the structured results or a human-readable explanation back to the requester.

Key considerations for MCP design

Authentication & RBAC
- The MCP must hold kubeconfig or use a service account with least-privilege RBAC to read the required resources.
Network security
- Secure connectivity between the MCP and the Kubernetes API server (TLS, private networking, firewall rules).
Data exposure & privacy
- Do not send secrets or other sensitive fields to an LLM. Redact or exclude sensitive fields before any model call.
Auditability & logging
- The MCP must log queries, the invoking identity, timestamps, and the data returned so you can trace access.
Rate limiting & caching
- Protect the cluster from excessive queries; consider caching common queries or summaries.
Governance & policy
- Define which query types are permitted and enforce them at the MCP layer.

Be careful granting LLMs or MCPs wide access to cluster data. Ensure RBAC is least-privilege, redact sensitive fields (e.g., secrets), and keep an audit trail of queries and results.

MCP capabilities checklist

Authentication: kubeconfig or service-account with least-privilege RBAC
Query engine: ability to call CoreV1 and other APIs (CustomResourceDefinitions as needed)
Sanitization: redaction rules for secrets and sensitive annotations
Summarization: optional LLM or built-in summarizer for human-friendly output
Auditing: query logs, requestor identity, and response snapshots
Extensibility: plugins for different clusters, multi-cluster support, and connectors to observability tools

Practical example prompts for an MCP

“List namespaces with label environment=prod and pod counts”
“Show deployments in booking-api with replicas < 2”
“Which namespaces have the label team=payments and what services run there?”

Architecture sketch (whiteboarding guide)

Draw the Kubernetes cluster at center.
Surrounding components:
- MCP (with a service account / kubeconfig)
- Optional LLM service (external or integrated)
- Logging / audit sink
- Users or automation (CLI, web UI, chat bot)
Annotate network paths and RBAC scopes.

When to pick MCP + LLM

You want natural-language access for non-kubernetes users (product owners, support).
You need a single control-plane endpoint to centralize queries, redaction, and auditing.
You need to translate raw metadata into higher-level answers or action items.

Next steps (what we will cover next)

Concise definition of an MCP and the core capabilities required for safe Kubernetes metadata access.
MCP design patterns and recommended open-source / vendor options.
A walkthrough: building a simple, safe MCP that queries a cluster and optionally integrates with an LLM for summarization.

Links and references

Thanks — that wraps up this article. Speak with you soon.

Introduction

The Problem Statement

Introduction to FinOps

Setting Up SkyLine Air Kubernetes Services

MCP Gateway for Kubernetes

LangChain MCP Agents

LLM Policy-Aware Label Audit

From Findings to Actionable Tickets

Demo Whiteboarding SkyLine Air Kubernetes Services

Watch Video