Skip to main content
Now that we have vectors stored and compressed, we must confront an important reality: data is not static. Vector databases are constantly changing. You continuously insert new vectors, update existing ones, and delete obsolete entries. That’s normal for any production datastore, but it creates a problem for the indexes we use for nearest-neighbor search. Nearest-neighbor indexes (for example, HNSW, IVF, PQ) are often designed and tuned for largely static or append-only datasets. Some structures (like HNSW) support dynamic updates, but frequent mutations can still fragment or stale the index. When the underlying data changes a lot, index components can drift from the true distribution of vectors: partitions and centroids become inaccurate, internal storage grows with many small fragments, and both latency and recall of searches can degrade. Think of it like maintaining a perfectly cataloged library while patrons continually add books to random shelves and remove others without updating the catalog. Over time, the catalog becomes unreliable and locating books is slower and less accurate. Also note: even with a managed vector-database service, the provider cannot fully optimize an index for your specific workload and business semantics. You still need engineering effort to maintain index quality for your use case. Why index maintenance matters
  • Performance: Fragmented indexes increase search latency and reduce throughput.
  • Accuracy: Out-of-date centroids or partitions hurt recall and precision.
  • Cost: Poor index quality forces larger search budgets (more probes, more compute).
  • Predictability: Unmaintained indexes produce variable query latencies and inconsistent results.
Common strategies for index maintenance To handle index maintenance at scale (for example, services like Netflix that add content and update user vectors hourly), teams generally use a mix of three strategies:
  1. Write first, index later
  • Buffer new writes (for example, in a write log, small in-memory shards, or a write-optimized store) and perform indexing asynchronously in the background.
  • Pros: very low write latency; you can batch and tune indexing jobs for throughput.
  • Cons: recent writes may be missing or lower-quality in search results until the background indexing completes.
  1. Merge small pieces (compaction)
  • Periodically merge many small or fragmented index shards into larger, optimized shards (similar to compaction in LSM trees).
  • Pros: improves consistency and search performance without a full rebuild.
  • Cons: requires scheduling and resource management; merges consume CPU, memory, and I/O while running.
  1. Rebuild sometimes
  • Run a full reindex from the canonical data store at scheduled intervals or when metrics indicate fragmentation exceeds a threshold.
  • Pros: restores optimal index structure and accuracy by clearing accumulated fragmentation.
  • Cons: expensive and potentially disruptive for large datasets; usually performed during low-traffic windows or on dedicated infrastructure.
The image explains a problem with data changes affecting search indexes and offers three strategies to keep the index healthy: "Write first, index later," "Merge small pieces," and "Rebuild sometimes."
Summary table — quick decision guide
StrategyWhen to use itKey benefitsTrade-offs
Write first, index laterHigh write volume, tight write latency requirementsFast writes, batch-friendly indexingRecent data may be temporarily unsearchable or lower-quality
Merge small pieces (compaction)Steady mutation with many small updatesKeeps index tidy, improves steady-state performanceRequires periodic resources and careful scheduling
Rebuild sometimesPeriodic full refresh windows or when fragmentation is severeRestores optimal index structure and accuracyExpensive; can be disruptive at scale
Best practices and operational tips
  • Monitor index health: track fragment counts, average shard size, query latency, and recall metrics.
  • Define SLA-driven policies: decide acceptable staleness and schedule compaction/rebuilds around traffic windows.
  • Combine strategies: buffer for low-latency writes, compact frequently to reduce fragmentation, and schedule periodic full rebuilds for a reset.
  • Automate thresholds: trigger compaction or full reindexing based on measurable indicators (e.g., shard count, average search recall).
  • Test on representative workloads: simulate production traffic to tune merge frequencies and batch sizes.
Choose a mix of buffering, compaction, and periodic rebuilds based on your workload: write volume, read SLAs, and how fresh search results must be.
That concludes the index-maintenance problem statement. We will now discuss the first solution — write first and index later — in detail.

Watch Video