Adding Memory to LLM Apps

Welcome back. In this lesson we cover a core capability for conversational AI: adding memory to LLM applications. Memory (also called history or context) enables multi-turn conversations by supplying prior exchanges to the model so it can respond coherently over time.

The image is a flowchart showing the interaction between a user and an application, which processes a prompt and response through context, history, and a language model.

Why history is necessary

LLMs are stateless by default. Each request is treated independently unless you include prior context.
Without history, the model has no memory of earlier turns and can produce inconsistent or hallucinated responses.
To preserve continuity, you must capture relevant prior exchanges, persist them as needed, and send the appropriate context with each prompt.

LLMs do not remember past requests on their own. Think of each model call like a single HTTP request: to provide continuity, you need to re-supply the prior conversation or a summary of it with each request.

Memory: short-term vs long-term Memory for LLMs generally falls into two categories:

Memory Type	Description	When to use
Short-term (session)	Volatile, kept in application memory (e.g., RAM) for the duration of a user session.	Real-time chats, single-session context where persistence across restarts is not required.
Long-term (persistent)	Stored in an external durable store so history survives restarts and is available across sessions.	User preferences, personal profiles, long-running projects, or systems that require retrieval over time.

Choosing a long-term store Long-term memory is implemented by externalizing conversational data to a database or vector store. Your choice depends on retrieval patterns, scale, and search needs. Common options include:

Redis — simple to integrate, supports multiple data structures and expiration policies.
SQLite — lightweight SQL database for local persistence.
Vector stores — for semantic similarity search when using embeddings (e.g., Faiss, Milvus, Pinecone, Weaviate).
Other databases — Postgres, MongoDB, cloud-managed stores.

The image lists Redis and SQLite as examples of long-term memory storage solutions.

Example: basic Redis-backed conversation memory Below is a minimal pattern for persisting and rehydrating conversational history to Redis. Store each message as a list element per conversation_id, then retrieve recent messages to build the prompt.

Pseudocode to append a message:

# Append a message to a conversation list
redis.rpush(f"conv:{conversation_id}:messages", json.dumps({"role": "user", "text": message}))
# Optionally set TTL or trim list length to limit memory
redis.ltrim(f"conv:{conversation_id}:messages", -100, -1)  # keep last 100 messages

Pseudocode to rehydrate context before generating a response:

# Retrieve last N messages and build the model prompt
messages = [json.loads(m) for m in redis.lrange(f"conv:{conversation_id}:messages", -10, -1)]
prompt = build_prompt_from_messages(messages, new_user_input)
response = llm.generate(prompt)
# Append model response back to Redis
redis.rpush(f"conv:{conversation_id}:messages", json.dumps({"role": "assistant", "text": response}))

This pattern can be adapted for vector stores: instead of saving raw text only, index embeddings to support semantic retrieval for long-term facts and memories. Security, privacy, and retention

Be mindful of privacy and compliance when persisting conversation data. Mask or redact sensitive information, enforce retention policies, and secure access to your storage backend.

Putting it together

Short-term memory is ideal for active sessions and quick context passing.
Long-term memory is necessary for cross-session continuity, personalization, and recall of past facts.
Choose the right store based on access patterns: Redis for fast ephemeral lists, vector stores for semantic search, and SQL/NoSQL systems for structured user data.
Always apply retention, redaction, and security best practices to stored conversation data.

Introduction

Overview of LangChain

Key Components of LangChain

Interacting with LLMs

Tips, Tricks, and Resources

Introduction to LCEL

Adding Memory to LLM Apps

Performing Retrieval

Implementing Chains

Using Tools

Building Agents

Conclusion

Building Blocks of LLM Apps

Adding Memory to LLM Apps

Watch Video