Skip to main content
You’ve just made your first API call. The model generated text, followed instructions, and responded to different temperatures. It’s impressive — and it’s tempting to reach for an LLM for every problem. Don’t. LLMs are powerful, but they aren’t the right tool for every task. Using one where ordinary code will do is like hiring a translator to read a stop sign: possible, but slow, costly, and unnecessary. The key to building reliable, cost-effective AI systems is understanding when to call the model and when to just write code: the distinction determines latency, cost, reliability, and maintainability. The key to building good AI applications, and eventually AI-driven systems,
A dark, grid-patterned graphic with retro pixelated text saying "YOUR FIRST API CALL" and a large yellow "DON'T." Below it reads "Not the right tool for every job" and "Like hiring a translator to read a stop sign."
is knowing when to call the model and when to just write code. Every task you encounter generally falls into one of two categories.
  • Deterministic tasks — the same input always produces the same output. There is a single correct answer and you can write exact rules to compute it:
    • Calculate a tip
    • Sort a list
    • Convert a date format
    • Check if an email contains an @ symbol
    These are best handled with a few lines of code: they run instantly, deterministically, and cheaply.
  • Non-deterministic tasks — the right answer depends on judgment, context, or interpretation:
    • Summarize a legal document
    • Decide if a customer email sounds angry or confused
    • Write a professional reply to a complaint
    These require nuance, tone, and contextual understanding — exactly what LLMs are designed to handle.
An infographic titled "Two Kinds of Tasks" that contrasts deterministic tasks on the left with non-deterministic tasks on the right. The deterministic examples list actions like "calculate a tip," "sort a list," "convert a date," and "check email format," while the non-deterministic examples include "summarize a document" and "detect angry vs confused."
Quick rule of thumb: if the answer is always the same for the same input, use code. If the answer requires judgment, context, or language understanding, use an LLM.
Rule of thumb: Deterministic → code. Judgment/language → LLM.
Decision guide (examples): Examples, side by side:
  • Calculate shipping cost based on weight — code (deterministic).
  • Summarize a product review — LLM (reading comprehension).
  • Check if a password meets a minimum length — code:
  • Decide if a review is positive or negative — LLM (requires interpretation).
  • Convert Celsius to Fahrenheit — code (exact formula).
  • Extract a name and job title from an email — LLM (names and formats vary).
Using an LLM when you don’t need one has tangible costs:
  • It’s slower. An API call takes hundreds of milliseconds; a line of code runs in microseconds — roughly a thousandfold difference.
  • It costs money. Every call consumes tokens and billing.
  • It can be wrong. LLMs are probabilistic token predictors and may make arithmetic or factual errors.
  • It can be unpredictable. Even with temperature set to 0, outputs may vary across calls or model versions.
A slide titled "REAL COSTS" with four colored boxes summarizing drawbacks: "Slower" (ms vs µs — 1000× difference), "Costs money" (every call uses tokens), "Can be wrong" (predicting tokens, not doing math), and "Unpredictable" (output can vary even at temp 0).
Avoid unnecessary LLM calls: they add latency, cost, and potential inconsistency. Encapsulate deterministic logic in reliable functions and reserve LLMs for tasks that require judgment, context, or natural language.
Concrete example: building a receipt generator.
  • The math — subtotal, tax, total — is deterministic and should be implemented in code.
  • The friendly thank-you note at the bottom that varies in wording is a good fit for an LLM.
Deterministic code for receipt calculations:
When building agent-like AI systems, structure them as a set of tools (functions) that perform deterministic work: search a database, send an email, run calculations, validate inputs. Those tools should be reliable code. The LLM’s role is to interpret intent, decide which tool to call, and orchestrate higher-level behavior. Design pattern summary:
  • Tools = code (deterministic, testable, fast).
  • LLM = decides, composes, interprets (judgment, context, natural language).
A retro-styled dark slide titled "LOOKING AHEAD" with the subtitle "AI agents will have tools." It displays colorful button-like labels such as "SEARCH DATABASE," "SEND EMAIL," "RUN CALCULATION," "TOOLS = CODE," and "LLM = DECIDES."
Let the LLM think — and let code do the rest. Further reading and references:

Watch Video