Skip to main content
Welcome back. In this lesson we continue from Part 1 and finish the remaining vector query methods used in modern similarity search systems. We explain when to choose each approach, the trade-offs you’ll face, and concise real-world examples you can use when designing or debating architectures with engineers and data scientists. Key topics covered:
  • Range / Threshold Search (quality-first filtering)
  • Multi-Vector Search (multimodal queries)
  • Approximate Nearest Neighbor (ANN) Search (scale and latency)
  • Two-Stage Search (retrieve-and-rerank for both speed and accuracy)
Range or threshold search filters results by a minimum similarity score rather than returning a fixed number of neighbors. Use this when you care about the absolute quality of each match instead of the top-K set. When to use it
  • You require high-confidence matches (not simply the top-k results).
  • Correctness and compliance matter (e.g., legal, regulatory, or safety checks).
  • You want deterministic filtering across different queries and model versions.
Real-world examples
  • Plagiarism detection: return all documents with similarity > 0.85 to a submitted paper.
  • Fraud detection: flag transactions with similarity > 0.9 against known fraudulent patterns.
Practical tip: validate thresholds on representative labeled data to balance precision and recall. Too low a threshold increases noise; too high misses borderline but valid matches.
Thresholds are a filtering step. Choose them based on validation data: too low yields noise, too high may miss valid matches.
Example query patterns
  • Generic threshold pseudocode:
-- Pseudocode: filter by similarity score
SELECT id, similarity_score
FROM vector_index
WHERE similarity_score >= 0.8
ORDER BY similarity_score DESC;
  • API-style:
{
  "query_vector": [0.01, 0.23, ...],
  "min_similarity": 0.8
}

​
Multi-Vector Search (Multimodal Queries)

Multi-vector search is for queries that combine different input types (text, image, audio). Each modality gets encoded to its own vector and the system merges or scores them jointly to find items that satisfy multiple signals. When to use it
  • User queries include mixed modalities (e.g., image + text).
  • You need results that match several signals simultaneously (visual features + textual intent).
  • Cross-modal relevance is critical for user experience.
Real-world examples
  • E-commerce: upload an image of a dress and type “red dress for summer” — the system merges image and text vectors to find matching inventory.
  • Social media search: combine uploaded image, caption text, and hashtag vectors to surface relevant posts.
The image is an infographic titled "When to Use Which Query Method? - Part 2" and explains the use of Multi-Vector Search for multi-modal inputs with example use cases in e-commerce and social media.
Implementation notes
  • Fusion strategies include weighted dot-product sums, concatenation with a learned MLP, or late fusion via re-ranking.
  • Tune modality weights based on validation metrics and user intent: e.g., if visual similarity is more important, raise image weight.
Approximate Nearest Neighbor (ANN) search is optimized for extremely large vector stores (millions to billions of vectors) where sub-second latency is required. ANN uses indexing structures and heuristics to return very likely nearest neighbors far faster than an exhaustive exact KNN scan, at a controlled loss in recall. When to use it
  • Very large datasets (millions–billions of vectors).
  • Real-time or low-latency applications where slight accuracy trade-offs are acceptable.
  • Services where throughput and fast tail latency are primary constraints.
Real-world examples
  • Web-scale search (e.g., search engines): serving billions of documents with strict latency SLOs.
  • Recommendation systems at scale (Spotify, Pinterest) where fast retrieval across massive catalogs is required.
The image is a table explaining when to use the Approximate Nearest Neighbor (ANN) query method, listing scenarios like large-scale databases and real-time apps, with example use cases from Google Search, Spotify, and Pinterest.
Performance trade-offs and tuning
  • ANN provides massive speedups but may reduce recall. Tune index parameters (e.g., number of probes, ef_search, M) to balance latency vs. accuracy.
  • Monitor recall/precision on labeled queries to guide the acceptable operating point.
ANN is a trade-off: optimize index parameters and measure recall on real query workloads. A misconfigured ANN index can silently degrade user experience.
Example ANN parameters (tool-specific) 
# Example (HNSW-style) tuning knobs:
- M (max neighbors per node): 16–64
- efConstruction (index build time): 200–1000
- efSearch (query quality vs latency): 50–200

​
Two-Stage Search (Retrieve-and-Rerank)

Two-stage search (retrieve-and-rerank) mixes a fast retrieval stage with a more expensive re-ranking stage. Typically you use an ANN or lightweight vector index to fetch a candidate set, then apply a stronger cross-encoder or business-logic-aware model to re-rank the candidates. When to use it
  • You need both low latency and high final ranking quality.
  • Ranking must incorporate complex signals: recency, personalization, seller priorities, or business rules.
  • The quality of the top results (e.g., top 10) is critical to user satisfaction.
Real-world examples
  • Search engines: retrieve ~1000 candidates with ANN, rerank the top subset using a cross-encoder, and display the best ~10 results.
  • E-commerce: retrieve product candidates, then rerank by personalization, conversion likelihood, and business constraints.
Benefits
  • Keeps initial retrieval cheap and scalable.
  • Allows sophisticated models to focus on a much smaller candidate set, improving final relevance without prohibitive cost.
The image is a table explaining when to use a two-stage search with re-ranking, detailing the query method, use cases, and examples such as speed, accuracy, and complex ranking factors in environments like e-commerce and search engines.
Architecture example
  • Stage 1 (fast): ANN -> 1,000 candidates
  • Stage 2 (expensive): cross-encoder or ensemble model -> rerank top 100 -> final top 10

​
Comparison Table

Query MethodBest ForProsConsExample Use Cases
Range / Threshold SearchHigh-confidence filteringDeterministic quality control, simpleMay return variable counts, sensitive to thresholdPlagiarism, compliance checks
Multi-Vector SearchMultimodal queriesCaptures multiple modalities, richer relevanceMore complex encoding/fusionImage+text e-commerce, multimodal social search
ANN SearchVery large scale, low latencyExtremely fast, scalablePotential recall loss; requires tuningWeb search, large recommenders
Two-Stage (Retrieve & Rerank)Speed + high-quality rankingBalances latency and top-result qualityMore complex pipeline, higher costSearch engines, conversion-optimized e-commerce

​
Summary — Choosing the Right Strategy

  • Range / Threshold: Use when you must enforce a strict quality bar for each returned item.
  • Multi-Vector: Use when user intent spans multiple modalities and you need joint relevance.
  • ANN: Use for massive datasets and strict latency requirements where a small accuracy trade-off is acceptable.
  • Two-Stage (Retrieve-and-Rerank): Use when you need both fast retrieval and the highest-quality top results incorporating complex signals.
With these patterns and trade-offs in mind, you’ll be better prepared to design vector search systems that match your performance, accuracy, and business requirements. Discuss these options with engineers and data scientists to select the right combination for your product. That is it for this lesson. See you in the next one.

Watch Video