Building an Agent with Search Tool

This lesson walks through a compact agent demo that connects a search tool (Tavily) with an LLM (ChatOpenAI). The goal is to demonstrate how message history, a prompt with an agent scratchpad, tools, and a runnable agent fit together to handle multi-step and context-aware queries. Some module names or imports may change over time—if you run into import errors, consult the latest LangChain Python SDK docs: https://python.langchain.com/en/latest/.

Overview

Build a chat prompt that includes a system instruction, a conversation history placeholder, the user input placeholder, and an agent_scratchpad placeholder.
Create an LLM (ChatOpenAI) and attach the Tavily search tool for web results.
Wrap an AgentExecutor with RunnableWithMessageHistory so each session can preserve history and the agent scratchpad.
Invoke the runnable agent with per-session IDs to support follow-up questions.

Key components

Component	Purpose	Example / Notes
Prompt Template	Structures the system, user, and scratchpad messages	`ChatPromptTemplate.from_messages(...)`
Agent Scratchpad	Temporary workspace for chain-of-thought / tool calls	`MessagesPlaceholder("agent_scratchpad")`
Search Tool	External retrieval (Tavily) to fetch web results	`TavilySearchResults(api_key=TAVILY_API_KEY)`
LLM	Generates language responses	`ChatOpenAI()`
Message History	Session-aware chat persistence	`ChatMessageHistory()`
Runnable Wrapper	Adds session-aware history to the agent executor	`RunnableWithMessageHistory(...)`

Imports and basic setup

We need imports for prompt templates, message history, runnables, the LLM, the Tavily search tool, and agent utilities. The agent_scratchpad placeholder is used as the agent’s temporary workspace.

from langchain_core.prompts.chat import ChatPromptTemplate, MessagesPlaceholder
from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory

from langchain_openai import ChatOpenAI

from langchain_community.tools.tavily_search import TavilySearchResults
from langchain.agents import create_tool_calling_agent, tool
from langchain.agents import AgentExecutor, AgentType, initialize_agent, load_tools

import os

# TAVILY_API_KEY="Your Tavily API Key"  # set this in your environment if required
TAVILY_API_KEY = os.getenv("TAVILY_API_KEY")
# Some tool implementations accept an explicit api_key parameter; others read from env.
search = TavilySearchResults(api_key=TAVILY_API_KEY)

# Build the prompt
prompt = ChatPromptTemplate.from_messages(
    [
        ("system", "You are a helpful assistant. Think step by step before responding."),
        MessagesPlaceholder("chat_history"),
        ("human", "{input}"),
        MessagesPlaceholder("agent_scratchpad"),
    ]
)

Explanation:

ChatPromptTemplate.from_messages(...) composes the prompt with system instructions, a chat_history placeholder, the user input placeholder ({input}), and the agent_scratchpad.
search is the Tavily search tool used by the agent to retrieve web results.
MessagesPlaceholder and ChatMessageHistory enable in-memory chat history persistence for a session.

Using MessagesPlaceholder("agent_scratchpad") gives the agent a workspace to append intermediate reasoning and tool calls. This helps the LLM and the tool orchestrator maintain context across a single turn and between turns when history is preserved.

Be careful with API keys in code or logs. Prefer environment variables, secrets managers, or encrypted stores. If a tool reads the key from the environment, do not hardcode it in production.

Create LLM, tools, and the message history

Create the LLM instance, assemble the tools list, and initialize an in-memory ChatMessageHistory for this demo.

llm = ChatOpenAI()
tools = [search]
message_history = ChatMessageHistory()

Agent creation and wrapping with message history

Use create_tool_calling_agent to construct a tool-calling agent with the LLM, tools, and prompt.
Use AgentExecutor to manage execution and orchestrate tool calls.
Wrap the executor with RunnableWithMessageHistory to provide session-aware message history retrieval.

The example below uses a simple lambda that returns the same ChatMessageHistory for any session_id. In production, map each session_id to its own persisted ChatMessageHistory (Redis, database, etc.).

agent = create_tool_calling_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=False)

agent1 = RunnableWithMessageHistory(
    agent_executor,
    # In real-world usage, map session_id -> ChatMessageHistory instance
    lambda session_id: message_history,
    input_messages_key="input",
    history_messages_key="chat_history",
)

Invoking the agent (examples)

When invoking the runnable agent, pass the user input using the input key and include a config that carries a session_id. RunnableWithMessageHistory uses this session_id to look up and persist the chat history across calls. Say hello to the agent:

agent1.invoke({"input": "hi!"}, config={"configurable": {"session_id": "session1"}})

Possible returned structure (example):

{
  "input": "hi!",
  "chat_history": [],
  "output": "Hello! How can I assist you today?"
}

Ask the agent a question that requires search:

response = agent1.invoke(
    {"input": "When is the ICC Men's T20 2024 World Cup scheduled?"},
    config={"configurable": {"session_id": "session1"}}
)
print(response["output"])

Expected (example) output:

The ICC Men's T20 World Cup 2024 is scheduled to take place from June 1 to June 29, 2024. It will feature 16 teams competing in this global cricket tournament.

Follow-up question that uses conversation context and search:

response = agent1.invoke(
    {"input": "Which countries are hosting?"},
    config={"configurable": {"session_id": "session1"}}
)
print(response["output"])

Expected (example) output:

The ICC Men's T20 World Cup 2024 will be hosted by the West Indies and the United States. The tournament venues include Antigua & Barbuda, Barbados, Guyana, Saint Lucia, St. Vincent and the Grenadines, Trinidad & Tobago, and three venues in the USA (e.g., Dallas, Florida, and New York).

Example: compute days until the tournament

A follow-up that relies on context and a date calculation:

response = agent1.invoke(
    {"input": "How many days before the first match starts?"},
    config={"configurable": {"session_id": "session1"}}
)
print(response["output"])

You may observe the LLM returning an incorrect or imprecise numeric answer (for example, saying “17 days” when the current date or calculation is not accurate). This highlights a limitation.

LLMs are not always reliable for precise arithmetic or time-based calculations unless you explicitly delegate the calculation to a deterministic tool (like a Python REPL or a date utility). For exact answers (e.g., “days until a date”), add a deterministic tool to the agent that performs the arithmetic and returns the correct result.

How the agent adds value over raw search

A plain search tool typically returns raw documents, snippets, or URLs.
The agent can call the search tool, aggregate results, and ask the LLM to synthesize a concise, user-friendly answer.
With session-aware message history, follow-up questions that refer to earlier turns are handled naturally because the prompt includes previous chat and the agent scratchpad.

Extending this agent

To make the agent more robust and capable of deterministic computation:

Add a Python REPL or date-calculation tool so the agent can delegate numeric or date arithmetic to a deterministic environment.
Persist ChatMessageHistory per session_id using Redis, a database, or another storage backend for production usage.
Add more retrieval tools or structured data sources (APIs, knowledge bases) to broaden the agent’s factual coverage.

Conclusion and next steps

In this lesson we built a session-aware agent that:

Uses a search tool (Tavily) to fetch web results.
Uses an LLM (ChatOpenAI) to synthesize and present answers.
Persists message history via RunnableWithMessageHistory.
Uses an agent_scratchpad in the prompt to manage intermediate reasoning and tool calls.

Next steps:

Add a deterministic tool (Python REPL or date calculator) to handle precise calculations.
Implement a per-session persistent store for chat history (e.g., Redis).
Explore multi-tool orchestration and richer prompts to improve answer reliability.

Links and references

LangChain Python Docs
Tavily — community tool integrations (check the relevant tool implementation in your SDK)
OpenAI API and Chat Models

Introduction

Overview of LangChain

Key Components of LangChain

Interacting with LLMs

Tips, Tricks, and Resources

Introduction to LCEL

Adding Memory to LLM Apps

Performing Retrieval

Implementing Chains

Using Tools

Building Agents

Conclusion

Building Blocks of LLM Apps

Building an Agent with Search Tool

Overview

Key components

Imports and basic setup

Create LLM, tools, and the message history

Agent creation and wrapping with message history

Invoking the agent (examples)

Example: compute days until the tournament

How the agent adds value over raw search

Extending this agent

Conclusion and next steps

Links and references

Watch Video

​Overview

​Key components

​Imports and basic setup

​Create LLM, tools, and the message history

​Agent creation and wrapping with message history

​Invoking the agent (examples)

​Example: compute days until the tournament

​How the agent adds value over raw search

​Extending this agent

​Conclusion and next steps

​Links and references

Watch Video

Overview

Key components

Imports and basic setup

Create LLM, tools, and the message history

Agent creation and wrapping with message history

Invoking the agent (examples)

Example: compute days until the tournament

How the agent adds value over raw search

Extending this agent

Conclusion and next steps

Links and references