> ## Documentation Index > Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt > Use this file to discover all available pages before exploring further. # The Selection Triangle Performance Cost and Latency > Guide explaining the Selection Triangle framework for choosing AI models by balancing performance, cost, and latency with testing, metrics, and routing strategies. If you open almost any AI platform today you’ll see a model picker—Pro, Max, Turbo, Mini—each promising something different: higher quality, faster responses, or lower cost. Choosing the “best” model without context is misleading. Best at what? For which workload? This guide introduces a practical mental model—the Selection Triangle—to help you choose models by balancing the three core constraints: Performance, Cost, and Latency. Visualize a triangle with one corner for each axis: Performance, Cost, and Latency. Moving toward one corner prioritizes that property and usually reduces the others. This triangle is a heuristic, not a rule: model improvements, optimizations, and hardware can shift trade-offs over time. Still, for current model options, most real-world workloads force you to prioritize one or two corners at the expense of another. Before selecting a model, create small, representative test suites for each job (20–30 realistic examples). Decide in advance what constitutes an acceptable result and use that evaluation to compare models. ## Performance (top corner) "Performance" here means the quality and correctness of the model’s output for your specific task: correctness, formatting fidelity, policy compliance, and robust decision-making (especially for agents). * Use public benchmarks to narrow choices, but treat them as a starting point—benchmarks rarely capture task-specific constraints (eg. strict formatting, domain jargon). * Build a targeted evaluation: run a representative test set through each candidate model and score the outputs against your acceptance criteria. * Evaluate for edge cases and failure modes that would be costly downstream (wrong facts, broken formatting, unsafe outputs). Examples of tasks that prioritize performance: * Legal or medical summarization where accuracy is critical. * Agent-driven workflows that make irreversible decisions. * Final content generation that will be published or billed. ## Cost (one bottom corner) Token price is a starting point—optimize for cost per successful result. * Compute the true end-to-end cost: API charges, orchestration layers, retries, human review, and downstream cleanup. * A cheaper-per-token model that fails frequently will increase operational costs through retries and manual fixes. * When multiple models meet your quality bar, cost usually becomes the deciding factor—especially for high-volume workloads (eg. summarizing millions of documents). ## Latency (other bottom corner) Latency defines how quickly you get a useful answer. Two useful measurements: * `Time to first token` — when streaming begins. * `Total generation time` — when the full response is ready. Latency matters differently depending on the product feature: * Interactive autocomplete: very low latency (an extra 2–3 seconds per keystroke ruins UX). * Conversational UIs: low time-to-first-token helps perceived responsiveness. * Batch jobs: throughput and cost matter far more than per-request latency. ## Putting it together: choosing a model for a workload Start by setting minimum acceptable thresholds on all three axes: quality, cost, and latency. Then prioritize according to your workload. Follow this evaluation checklist for each candidate model: * Pass rate on your representative test set (quality). * Average cost per successful run (including retries and cleanup). * Latency profile: time to first token, total generation time, and throughput. Table: Workload → Prioritized corner → Recommended model characteristics | Workload type | Prioritized corner(s) | Typical model characteristics | | - | - | - | | Instant autocomplete (editor suggestions) | Latency | Small, fast models with low `time to first token` | | Large-scale bulk summarization | Cost, Throughput | Cost-efficient models with good batching and predictable latency | | Reasoning-heavy agents | Performance | Larger models with stronger reasoning/factuality, even at higher cost | | Mixed product (UI + background jobs) | Multiple | Route interactive requests to small, fast models; route critical decisioning to high-quality models | Measure and compare models with these metrics: | Metric | What to measure | How to interpret | | - | - | - | | Quality (pass rate) | % of test cases that meet acceptance criteria | Primary gate: if below threshold, discard model | | Cost per successful run | API + orchestration + retry + cleanup costs | Useful for ROI and capacity planning | | Time to first token | Median and tail latency (p95/p99) | Affects perceived responsiveness | | Total generation time | Median and tail latency | Determines throughput and batch scheduling | | Retry rate | % of runs needing retries or human fixes | Indicates hidden cost and instability | ## Architectural notes * Use different models for different jobs within the same application. Don’t force every task through a single model. * Example: small, fast models for UI autocompletion; larger, higher-quality models for final document synthesis or agent decisions. * Consider an LLM gateway or routing layer to dispatch requests to the model that best fits the job’s performance/cost/latency needs. * Periodically re-run your tests. As models improve or pricing changes, a cheaper or smaller model may begin to meet your requirements. The model menu changes rapidly. Whenever new models, capabilities, or pricing appear, re-run your representative tests—your optimal choice may change. ## A final reminder There is no single “best” model—only the right model for a specific task given your performance, cost, and latency constraints. Match models to jobs, measure using realistic examples, and choose models that meet your acceptance criteria instead of defaulting to the largest or newest option. ## Links and references * [Kubernetes Documentation](https://kubernetes.io/docs/) * [Docker Hub](https://hub.docker.com/) * [Terraform Registry](https://registry.terraform.io/) This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.