Architecting Long Term Memory LTM

Why long-term memory matters Long-term memory (LTM) is what turns a simple chatbot into a true personal assistant. While short-term context only persists within a single conversation or execution, LTM preserves facts, preferences, events, and outcomes across sessions. This persistence enables continuity, personalization, and better decision-making over time—key elements for assistants that feel consistent and useful.

The image compares a basic chatbot with short-term memory to a true personal assistant with long-term memory, highlighting the benefits of personalized interactions and continuity.

LTM in LangGraph In LangGraph, long-term memory typically lives outside the graph: vector stores, databases, or file systems are common choices. Specific graph nodes read from or write to these external stores during execution. Retrieved information is merged into the current state or prompt so the model can reason with past context without permanently bloating the graph’s internal state. Think of it this way: if Robbie has a notebook for today’s deliveries, LTM is the file cabinet where he keeps all past routes, preferences, and customer notes.

The image shows a person named Ravi holding a clipboard, with documents flying from his position towards a file cabinet labeled "LTM (File Cabinet)" under the title "Long-Term Memory in LangGraph."

Common LTM storage formats Choose the storage format according to the type of memory you need:

Storage Type	Best for	Example / Notes
Vector store (embeddings)	Semantic memories: conversations, ideas, summaries	Good for similarity search and retrieval-augmented generation (RAG). Examples: FAISS, Pinecone, Milvus
Relational/NoSQL DB	Structured facts: preferences, order history, account data	Use for strongly-typed queries and transactions
File system / Object storage	Logs, summaries, raw documents	Suitable for archival, simple summaries, or large binaries

The image illustrates the long-term memory structure in LangGraph, comprising a vector store, relational database, and raw text file to manage embeddings, structured data, and summaries/logs.

Read and write patterns Reading: retrieval nodes query external stores (for example, a vector database) before an important decision point. The retrieved facts or documents are merged into the graph state or prompt so the LLM can reason with that context. Note that injected context consumes the model’s context window. Writing: after an interaction, summarize or extract relevant details (task outcomes, preferences, goals, recurring intents) and persist them. Store metadata such as user ID, timestamp, and source to improve later retrieval accuracy.

The image illustrates a workflow for writing to Long-Term Memory (LTM) after execution, showing steps from post-interaction summarization to writing and indexing with metadata into a vector store and structured database. It includes elements like task outcomes, user facts, goals, and recurring intents.

A common semantic-memory flow (vector store)

Convert texts into embeddings and index them in a vector store.
At runtime, query the vector store using the current user input to find relevant documents.
Inject retrieved documents (or summaries) into the graph state or prompt.
Use the enriched inputs for downstream reasoning or response generation.
Optionally summarize and persist new information back to the vector store with metadata.

The image is a diagram illustrating the "Vector Store Memory Pattern," showing how text is converted into vector embeddings for storing and retrieving information to solve short-term memory limitations. It highlights components like semantic similarity search and accessible memory items for improving agent intelligence and consistency.

Concise, practical example The example below demonstrates loading a saved FAISS index, turning it into a retriever, and wrapping retrieval logic as a LangGraph-compatible node using RunnableLambda. Ensure you use the same embedding model when loading the index that you used when creating it.

from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import FAISS
from langchain_core.runnables import RunnableLambda

# Load vector store as a retriever (use the same embedding model used to create the index)
embedding_model = OpenAIEmbeddings()
vectorstore = FAISS.load_local(
    "ltm_index",
    embeddings=embedding_model
)
retriever = vectorstore.as_retriever()

# Define a LangGraph-compatible node that performs semantic retrieval
def retrieve_memory(state: dict) -> dict:
    query = state.get("user_input", "")
    # Use the retriever's standard API to get relevant documents
    docs = retriever.get_relevant_documents(query)
    # Return retrieved documents into the graph state under the `retrieved_memory` key
    return {"retrieved_memory": docs}

retrieve_node = RunnableLambda(retrieve_memory)

Key notes about the example:

Use the same embedding model to create and to load the FAISS index.
as_retriever() provides a stable retriever API (e.g., get_relevant_documents).
Wrapping the function with RunnableLambda makes it usable as a node within LangGraph workflows.

How to use retrieved memories Decide how to inject retrieved documents into the LLM prompt. Common strategies:

Direct injection: place raw retrieved text under a labeled “User Facts” or “Context” section.
Summarize first: reduce token usage by summarizing documents before injecting them.
Merge into conversation state: append selected facts to the state buffer consumed by later nodes.

Example: prompt injection by joining retrieved documents

# Build a prompt by joining the page content of retrieved documents
retrieved_facts = "\n".join([doc.page_content for doc in state["retrieved_memory"]])
user_input = state["user_input"]

prompt = f"""
User Facts:
{retrieved_facts}

User: {user_input}
Assistant:"""

This retrieval-augmented generation (RAG) pattern lets the LLM reason with external memory as if it were part of the context window without permanently expanding the model’s internal state. See also: Retrieval-augmented generation. Memory hygiene and best practices

Store consistent metadata: user ID, timestamps, source, and tags improve targeted retrieval.
Preprocess text: chunk long documents, remove irrelevant content, and normalize formatting before embedding. Clean inputs yield better embeddings.
Tune retrieval: experiment with similarity thresholds and k (how many items to return). Often top 3–5 documents work best.
Maintain indexes: deduplicate, merge similar entries into summaries, archive or delete stale entries periodically.

The image outlines three best practices: using consistent metadata, preprocessing text before storing, and experimenting with similarity thresholds and k-values.

Store and index metadata alongside embeddings (e.g., user ID, timestamp, source). This drastically improves targeted retrieval and enables efficient filtering of memories.

Avoid over-injecting Injecting too much retrieved text into the prompt can inflate token usage and confuse the model. Prefer filtering, summarizing, or selecting the most relevant items for the current intent. Use structured prompts that clearly label retrieved facts and the current user query.

The image outlines strategies for memory hygiene and management, addressing a noisy LLM problem with actions like periodic cleaning and merging redundant memory, enabled by strong metadata tagging for better accuracy and speed.

Real-world value Long-term memory makes agents persistent and context-aware:

Customer support: retrieve past tickets and resolutions to maintain continuity.
Education: remember which topics a learner covered and adapt future lessons.
Sales: surface lead-specific details during conversations.
Copilots: fetch relevant docs or past decisions to assist with context.

Whether you choose vector stores for semantic memory or databases for structured facts, the architecture of your memory layer controls how personalized and consistent your agent will be over time. Further reading and references

FAISS: https://github.com/facebookresearch/faiss
Retrieval-augmented generation (RAG): https://en.wikipedia.org/wiki/Retrieval-augmented_generation
LangGraph concepts and workflow patterns (see your LangGraph docs for implementation specifics)

Course Introduction

Foundational Graph Concepts and Toolkit Setup

The Core Workflow: Nodes, Edges, and Routing

State Management and Iterative Loops

Context Management and Short-Term Memory

Long-Term Memory and Stateful Persistence

Human-in-the-Loop (HILT) Architectures

Advanced Control and Debugging UX

Conclusion

Architecting Long Term Memory LTM

Watch Video