Skip to main content
When someone says “This runs on GPT” or “This runs on Claude,” it sounds like they’ve described the product. In reality the model is only half of the system. The other half — the harness — is everything that surrounds, controls, and operationalizes the model. Imagine you hand an AI system the goal: “book a flight under $300.” That goal is received by the harness. The model is the predictive core that proposes the next token, sentence, or action, but it does not itself call APIs, manage state across sessions, or enforce safety policies. The harness is the layer that makes the model useful and trustworthy in real-world applications.
Think of the model as the engine and the harness as the gearbox, steering, and brakes. The harness turns prediction into multi-step work, connecting the model’s outputs to real-world effects.
The harness performs four distinct, complementary jobs:
  1. Orchestration — the loop
    • The harness runs the interaction loop: send a prompt or question to the model, receive the model’s response, decide the next step, execute it when necessary, then iterate. This loop transforms single-turn predictions into multi-step workflows and long-running processes.
    • Example: ask the model to compose a search query → run the query → feed results back to the model → ask it to filter or summarize.
  2. Action execution — connecting model intent to the real world
    • When the model suggests “Search for flights,” the harness actually performs that search: it calls airline or aggregator APIs, queries databases, runs scripts, or invokes services. The harness is responsible for reliably mapping model intents to concrete actions.
    • Clear separation of concerns matters: the model decides, the harness does.
  3. Context and memory management
    • Most models do not persist long-term state between invocations. The harness stores conversation history, user goals, tool availability, permissions, and previous results, and packages the relevant subset into each model request.
    • The harness manages session context vs. long-term memory, performs retrieval of relevant documents, and handles truncation and prompt engineering to fit model input windows.
  4. Guardrails and enforcement
    • The harness mediates model requests and real-world effects and therefore houses safety, compliance, and governance controls: block dangerous actions, require human approval for risky operations, rate-limit downstream calls, and detect runaway loops.
    • Enforcement policies, auditing, and approval workflows should live in the harness, not in the model.
A presenter stands beside a dark, neon-colored diagram drawn on a glass board titled "THE HARNESS," with boxes labeled "GOAL" and "CONTEXT" connected by arrows. The right side shows circular arrows and a dashed box with a question mark, with the phrase "enforces limits" written above.
Put another way: the model is only half of the product. The harness runs the loop, acts on the model’s intents, manages context and memory, and enforces limits. Because the harness defines orchestration, tool access, memory strategies, and guardrails, two products built on the same underlying model can behave like completely different systems. Change the harness, and you change the whole product.

​
Quick reference: harness responsibilities

Never assume safety or compliance is handled by the model alone. Put explicit checks and human-approval gates in your harness for any actions that can affect accounts, finances, privacy, or safety.

​
How to evaluate an AI system (beyond “what model”)

When you evaluate or compare AI products, ask questions about the harness:
  • How does it orchestrate multi-step tasks and retries?
  • Which external tools, APIs, or services can it call?
  • How does it store, retrieve, and truncate context or long-term memory?
  • What guardrails, approval workflows, and audit logs are enforced?
These questions reveal practical differences that matter for reliability, security, and user experience. We often design systems with “one model, one loop, one job.” But when work is large or continuous, you may need multiple orchestrations running concurrently and passing tasks between one another — a direction explored by advanced agentic systems and multi-agent architectures.

​
Further reading and resources

Evaluate both the model and the harness to understand the full capabilities and risks of any AI product.

Watch Video