AI Workflows

This lesson explains how developers combine large language model (LLM) calls and tools into larger systems — AI workflows. These workflows are orchestrated by predefined code paths: developers design the steps and ordering in advance, while LLMs perform the uncertain, generative parts at each step. That separation (deterministic control in code + probabilistic generation in models) makes systems easier to reason about, test, and audit. Below you’ll find common architecture patterns for building AI workflows, guidance on when to choose each pattern, and practical considerations such as validation, cost, latency, and stopping criteria. Summary table — choose a pattern at a glance

Pattern	What it does	Best when	Key trade-offs
Prompt chaining	Sequential LLM calls where each step consumes the previous output	You need auditable, stepwise transformations	Easier to reason about, increases end-to-end latency
Routing (classifier + handler)	Classify input and route to specialized handlers	Diverse request types with different cost/latency needs	Depends on classifier accuracy; misrouting costs matter
Parallelization	Run independent subtasks concurrently or use voting across runs	Independent subtasks or need robustness via redundancy	Lower latency, higher concurrent compute & cost
Orchestrator-Worker	Central orchestrator dynamically decomposes tasks into workers	Inputs vary widely and require runtime decomposition	Complex orchestration, potential runaway cost from many workers
Evaluator-Optimizer	Generate, evaluate, and iterate until acceptance criteria are met	High-quality or safety-sensitive outputs (code, legal, translation)	Higher compute/latency; requires clear stopping rules

Pattern 1 — Prompt chaining (sequential steps)

Break a task into sequential steps; each LLM call consumes the previous step’s output.
Insert programmatic checks between steps (length, schema, content safety) and decide whether to continue, retry, or abort.
Typical use case: Draft an email (step 1) → validate required sections → polish the draft (step 2) → final review.

A dark, neon-styled flowchart titled "Pattern 1 Prompt Chaining" showing a sequential LLM pipeline. It maps boxes labeled "User Prompt" → "Write Email" → "Validate" → "Polish Email" → "Final Draft" with arrows for output, validation, and polishing.

When to use:

Need deterministic, auditable transformations and clear retry semantics.
Simple to implement and debug.

Considerations:

Latency accumulates as steps are run serially.
Validate intermediate outputs programmatically to avoid cascading failures.

Example pseudocode (sequential with validation and retry):

draft = llm.call(prompt="Draft an email about X")
if not validate_schema(draft):
    draft = retry_with_clarified_prompt(draft)
polished = llm.call(prompt=f"Polish this email:\n\n{draft}")
final = content_safety_check(polished)

Pattern 2 — Routing (classifier + handler)

Use a classifier model to route requests to an appropriate handler pipeline: trivial Q&A vs. scheduling vs. complex planning.
Enables cost optimization: route simple queries to cheap, low-latency models and hard tasks to larger, more capable models.

A neon-styled flowchart titled "Pattern 2 Routing" showing a "User Prompt" feeding a "Classifier" that branches to "Simple QA", "Scheduler", and "Travel Plan" nodes which then converge into a "Response" box. The branching arrows are labeled "simple", "schedule", and "complex" to indicate routing paths.

When to use:

Heterogeneous inputs where specialized handlers provide better quality/cost tradeoffs.

Considerations:

Classifier precision is critical; add fallback or human-in-the-loop paths for uncertain classifications.
Log routing decisions for auditing and to measure misroute cost.

Sample routing logic:

intent = classifier.predict(user_input)
switch(intent) {
  case "faq": return handle_faq(user_input)        // cheap model
  case "schedule": return handle_scheduler(user_input)
  default: return handle_complex_plan(user_input)  // expensive model
}

Pattern 3 — Parallelization (sectioning and voting)

Sectioning: split input into independent subtasks and run them concurrently (e.g., “Plan my Thursday”: check calendar, email, weather).
Voting: run the same prompt across different models or random seeds and aggregate via majority/consensus to reduce hallucination.

A dark-themed flow diagram titled "Pattern 3 Parallelization" showing a User Prompt branching to three parallel services (Calendar, Emails, Weather) that feed into an Aggregator and produce a final Briefing.

When to use:

Independent subtasks that don’t block each other.
Need lower latency for aggregated results or higher reliability through redundancy.

Considerations:

Parallel calls increase concurrency and cost.
Aggregator logic must handle partial failures and inconsistent results.

Pattern 4 — Orchestrator-Worker (dynamic decomposition)

A central orchestrator inspects the input, decides which subtasks are needed at runtime, and delegates those to worker LLMs.
Subtasks are dynamically determined, not predeclared.

A neon-styled flowchart titled "Pattern 4 Orchestrator-Worker" showing a User Prompt feeding an Orchestrator that splits tasks into Worker A, Worker B, and Worker C which then merge into a final "Combined" output. The diagram labels the split pieces (Piece 1/2/3) and includes the phrase "Dynamic Breakdown" at the bottom.

When to use:

Inputs vary widely and benefit from a centralized coordinator that decides decomposition.
Good for complex, open-ended tasks where fixed pipelines would be brittle.

Considerations:

Orchestrator must be robust: enforce limits (max workers, cost caps), idempotency, and retry/backoff behavior.
Instrumentation is essential to prevent runaway costs from large dynamic decompositions.

Pattern 5 — Evaluator-Optimizer (generate + critique loop)

Two distinct roles interact: a generator produces candidate outputs; an evaluator reviews and gives feedback. The generator refines outputs iteratively until acceptance criteria or retry limits are reached.
Ideal for tasks demanding high correctness or nuanced judgment (legal text, code, translations).

A neon-styled flowchart titled "PATTERN 5 EVALUATOR-OPTIMIZER" showing a User Prompt feeding a Generator that loops with an Evaluator via "Review" and "Feedback." The Evaluator outputs a "Passed" result to produce the Final Quality.

When to use:

Outputs must meet strict quality thresholds and benefit from iterative critique.

Considerations:

Define clear acceptance criteria to avoid infinite loops.
Combine automated checks (unit tests, linters, schema validators) with model-based evaluation for stronger guarantees.

Example evaluate-refine loop:

for i in range(max_iters):
    candidate = generator.generate(prompt)
    score, feedback = evaluator.review(candidate)
    if score >= threshold:
        return candidate
    prompt = incorporate_feedback(prompt, feedback)
# fallback if threshold never met
return best_seen_candidate

Operational considerations (applies to all patterns)

Stop conditions: enforce max iterations, runtime limits, and quality thresholds.
Intermediate validation: use schema validation, token/length checks, content-safety filters, and unit tests where applicable.
Model selection: route low-latency/low-cost tasks to smaller models and reserve large models for tasks that need them.
Observability: log inputs, outputs, routing decisions, retries, and evaluation scores for auditing, debugging, and cost analysis.
Cost control: set per-request and per-workflow budgets, hard limits on spawned subtasks, and monitor for anomalies.

Best practices checklist

Keep control logic deterministic and in code; use LLMs for uncertain or generative work only.
Implement programmatic guards before and after LLM calls (schema checks, sanitization).
Provide fallbacks for classifier or orchestrator uncertainty (human review, simpler model).
Track and visualize workflow metrics: latency, cost per pattern, success rate, and misroute rates.

Design workflows so that control logic lives in your code (deterministic and auditable) while LLMs handle the uncertain parts (content generation, classification, synthesis). This separation makes behavior predictable and easier to test and monitor.

In all patterns the developer decides which steps run, their order, what data passes between them, and when to stop. LLMs are powerful components within the structure you design — use orchestration, validation, and observability to build reliable, cost-effective AI workflows. Further reading and references

Designing reliable LLM systems — patterns and tradeoffs (replace with your internal docs)
Content safety and moderation best practices
Instrumentation and observability for AI pipelines

LLM Fundamentals

Tools and Workflows

Building AI Agents

Introduction

Production & OpenClaw

Watch Video