> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Performing Semantic Search
> Guide to creating embeddings with OpenAI, storing them in Chroma, and performing semantic similarity search using LangChain for retrieval augmented applications
In this lesson you'll convert text into embeddings, store them in a vector database (Chroma), and run a simple semantic (similarity) search using LangChain and OpenAI embeddings. This pattern is useful for retrieval-augmented generation (RAG), search interfaces, and any application that needs semantic relevance rather than exact text matches.
## Prerequisites
Make sure you have:
* Installed the required Python packages (for example, [langchain](https://python.langchain.com/) and [Chroma](https://www.trychroma.com/)).
* Configured your OpenAI API key (for example by setting the `OPENAI_API_KEY` environment variable). Creating embeddings calls the [OpenAI API](https://platform.openai.com/docs/guides/embeddings).
Useful references:
* [LangChain Documentation](https://python.langchain.com/)
* [Chroma (vector DB)](https://www.trychroma.com/)
* [OpenAI Embeddings Guide](https://platform.openai.com/docs/guides/embeddings)
## Step 1 — Imports and Example Documents
Import the required modules, create an embeddings object, and define a small set of example documents (headlines). These headlines will be embedded and indexed in the vector store.
```python theme={null}
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Chroma
embeddings = OpenAIEmbeddings(model="text-embedding-3-large")
docs = [
"Thrilling Finale Awaits: The Countdown to the Cricket World Cup Championship",
"Global Giants Clash: Football World Cup Semi-Finals Set the Stage for Epic Showdowns",
"Record Crowds and Unforgettable Moments: Highlights from the Cricket World Cup",
"From Underdogs to Contenders: Football World Cup Surprises and Breakout Stars"
]
```
## Step 2 — Create the Chroma Vector Store
Chroma is an open-source vector database that:
* Indexes and stores vectors (embeddings) with optional metadata.
* Performs fast similarity search / retrieval over those vectors.
You can create a Chroma vector store directly from strings, or from pre-chunked documents/pages. Here we pass raw text strings:
```python theme={null}
vectorstore = Chroma.from_texts(texts=docs, embedding=embeddings)
```
If you have metadata (source, URL, id), pass documents with metadata instead of plain strings to make downstream identification easier.
## Step 3 — Run Semantic Similarity Searches
When you query the vector store, the query is embedded with the same model and compared to the stored vectors. Use the `k` parameter to control how many nearest neighbors you retrieve.
```python theme={null}
# Example: retrieve top 2 documents most related to "Rohit Sharma"
results_rohit = vectorstore.similarity_search("Rohit Sharma", k=2)
print(results_rohit)
# Example output (list of Document objects):
# [Document(page_content='Record Crowds and Unforgettable Moments: Highlights from the Cricket World Cup'),
# Example: retrieve top 2 documents most related to "Lionel Messi"
results_messi = vectorstore.similarity_search("Lionel Messi", k=2)
print(results_messi)
# Example output:
# [Document(page_content='From Underdogs to Contenders: Football World Cup Surprises and Breakout Stars'),
# Document(page_content='Global Giants Clash: Football World Cup Semi-Finals Set the Stage for Epic Showdowns')]
```
## Why This Works (Concise)
* Both documents and the query are converted into embeddings by the same model (`text-embedding-3-large`).
* The vector database compares these vectors (e.g., using cosine similarity) and returns the nearest vectors/documents.
* This is semantic search: the system can associate concepts (e.g., player names) with relevant documents even when the exact token does not appear in the text.
Tip: The `k` (top-k) parameter controls how many nearest neighbors you retrieve. Choose `k` based on how many documents you want to use downstream (for example, as context for a language model). You can also store metadata with each text to help identify sources.
## Common Parameters and Options
| Parameter | Purpose | Example |
| ---------- | ------------------------------------------------------------ | ------------------------------- |
| `model` | Embedding model used to create vector representations | `text-embedding-3-large` |
| `texts` | List of strings to embed and store | `["text1", "text2"]` |
| `k` | Number of nearest neighbors to retrieve in similarity search | `k=2` |
| `metadata` | Optional: identify source, doc id, URL for each text | `{"source": "news", "id": 123}` |
## End-to-End Pattern
A concise end-to-end workflow:
1. Initialize embeddings with your chosen model.
2. Convert texts (or chunks) into embeddings and store in Chroma.
3. For each user query, embed the query and run `similarity_search(query, k=...)`.
4. Use the retrieved documents as context for downstream tasks (summarization, QA, RAG).
This simple pattern powers many production retrieval pipelines: obtain relevant chunks quickly and then pass them to a language model for generation, question answering, or summarization.
## Links and References
* [Kubernetes Documentation](https://kubernetes.io/docs/) (example resource link)
* [LangChain Documentation](https://python.langchain.com/)
* [Chroma (vector DB)](https://www.trychroma.com/)
* [OpenAI Embeddings Guide](https://platform.openai.com/docs/guides/embeddings)