> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.
# From Prompt Engineering to Context Engineering
> Explains context engineering versus prompt engineering and practical RAG techniques for supplying relevant logs, summaries, and retrieved passages to improve AI responses under token limits.
Imagine this: you ask an AI assistant, "Why did my application deployment fail?" The assistant responds with a vague answer. You rewrite the prompt with more detail and try again—still vague. The problem isn’t only the prompt wording. The model never received the logs, error traces, or configuration files you meant to include.
What you tweaked by rewriting the prompt is prompt engineering. But in real-world applications there are two prompt layers to consider:
* System prompt — instructions the application supplies to the model (for example, “Answer only from the supplied incident data”).
* User prompt — the question the user types (for example, “Why did our deployment fail?”).
Both matter, but they’re only part of what the model actually sees. Everything that the model receives at call time sits inside the context window — the model’s working memory. The context window is a fixed-size buffer measured in tokens. Every piece of text you place into that buffer consumes tokens and reduces remaining space.
Context engineering is the discipline of deciding what fills that space so the model has the right information to answer. Prompt engineering optimizes the system and user prompts; context engineering controls the rest: conversation history, tool outputs, retrieved documents, summaries, and any other data you include before the prompt.
## Why context engineering matters
* A polished prompt cannot compensate for missing context — if logs or private documents never reach the model, the answer will be limited.
* Token limits force trade-offs: include more relevant data or longer instructions, but not both.
* Retrieval-augmented generation (RAG) is how you bring external knowledge into the model’s working memory without exhausting tokens.
## Typical sources that occupy the context window
| Source type | Purpose / Why include it |
| - | - |
| Conversation history | Keeps the model aware of the task flow and previous clarifications |
| Tool/API outputs | Results from log collectors, monitoring tools, test runners, or diagnostics |
| Retrieved documents | Passages pulled from a large corpus (via embeddings + vector search) |
| System and user prompts | Instructions and the current question (must be concise to save tokens) |
## Retrieval-augmented generation (RAG) in brief
Because you can’t forward thousands of documents directly, RAG pipelines:
1. Index a large document collection using embeddings.
2. Use vector search to find the most relevant passages (top-k).
3. Insert only those passages into the context window alongside the system and user prompts.
This “R” — retrieval — is the practical way to bring external knowledge into the model’s context without exhausting tokens.
## Key distinctions
* Prompt engineering: improves instructions (system prompt) and the user’s explicit question.
* Context engineering: decides what the model actually sees before it generates a response (everything placed in the context window).
* Practical impact: better context (right logs, relevant document passages, concise history) yields better answers.
## Practical techniques for context engineering
* Prioritize content
* Include the most relevant logs, error messages, or concise summaries first.
* Place the highest-value items near the beginning of the context block so they’re less likely to be trimmed.
* Chunk and summarize
* Break long documents into smaller passages and summarize each chunk.
* Use automated summarizers to compress repeated context.
* Use retrieval (RAG)
* Index content with embeddings and fetch the top-k relevant passages rather than sending entire documents.
* Tune k to balance breadth vs token cost.
* Control token usage
* Trim boilerplate, shorten system prompts, and remove unnecessary conversation history.
* Prefer structured outputs from tools (which can be summarized) over verbatim dumps.
* Use tools where appropriate
* Invoke log parsers, diagnostics, or structured APIs and insert only their essential outputs into the context.
* Cache useful summaries
* Store compressed context (e.g., summarized incident state or metadata) for quick reuse in follow-up queries to avoid re-retrieving everything.
## Example RAG / API request pattern
A typical flow to assemble context before calling a model:
* Retrieve top-k passages from the vector store.
* Summarize or filter those passages.
* Construct the system prompt, concatenated with prioritized passages and the user question.
* Send the assembled request to the model.
Example request body (illustrative; backend-specific fields will vary):
```json theme={null}
{
"model": "example-model-8k",
"messages": [
{ "role": "system", "content": "Answer only from the provided incident data." },
{ "role": "assistant", "content": "Relevant diagnostics and summaries included below." },
{ "role": "user", "content": "Why did our deployment fail? See logs and config excerpts." }
],
"context_passages": [
"top passage 1 from retrieval",
"top passage 2 from retrieval"
],
"max_tokens": 1500
}
```
## Practical tuning checklist
* Limit conversation history to recent, relevant turns.
* Place critical logs and error summaries near the top of the context.
* Compress repeated diagnostic content into short summaries.
* Use vector DBs and embeddings for fast, relevant retrieval.
* Monitor token usage and adjust top-k or passage lengths to stay within limits.
| Technique | Benefit | When to use |
| - | - | - |
| Summarization | Reduces token cost, preserves intent | Large incident reports or long logs |
| Chunking + passage selection | Ensures relevant segments are delivered | Multi-file or very long documents |
| Cached summaries | Faster response for repeated questions | Ongoing incidents or recurring queries |
| Structured tool outputs | Easier to parse and summarize | Log parsers, monitoring APIs |
## Tools & references
* Embeddings + vector stores: FAISS, Milvus, Chroma, Pinecone
* RAG patterns and engineering: see articles on retrieval-augmented generation and production pipelines
* Token considerations: use model documentation for exact context window sizes (e.g., \~8k, 32k tokens)
Links and further reading:
* [Retrieval-Augmented Generation (overview)](https://en.wikipedia.org/wiki/Retrieval-augmented_generation)
* [Vector search and similarity](https://en.wikipedia.org/wiki/Vector_space_model)
* [Pinecone vector DB](https://www.pinecone.io/)
* [FAISS (Facebook AI Similarity Search)](https://github.com/facebookresearch/faiss)
Context engineering = deciding what the model *actually sees*. You can tune prompts, but the answer is limited by the context window. Retrieval (RAG), summaries, and selective inclusion of logs/documents are the practical levers to improve answers.
Remember: garbage in, garbage out — and the inverse is true as well. The better the context you provide (the right logs, the right document passages, concise history), the better the model’s answer will be. Context engineering is the practical, application-level work that makes prompt engineering effective.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.