- Removes tombstones and stale records that make searches scan more data than needed.
- Reorders data so related vectors live close together, improving locality and cache behavior.
- Restores predictable latency patterns after long-term churn (many updates/deletes).
| Step | Purpose | Notes / Best practices |
|---|---|---|
| Detect degradation | Identify when fragmentation impacts performance | Monitor latency percentiles, query throughput, segment/merge counts, and merge failure rates. |
| Schedule or trigger | Decide when to run reindexing | Use automated schedules (e.g., nightly) or manual triggers tied to thresholds. Prefer low-traffic windows. |
| Reindex into a fresh index | Build an optimized index from authoritative data | Stream or bulk-copy documents into a new index with optimized merge policy and shard layout. |
| Validate and swap | Verify correctness before serving traffic | Run end-to-end checks, consistency tests, and then perform an atomic alias/routing swap. |
| Monitor post-swap | Ensure rebuild achieved goals | Compare latency, error rates, and search results between old and new indices. Roll back if anomalies appear. |
- Atomic swap: Use index alias swaps or routing rules to switch traffic atomically to the rebuilt index, avoiding partial results being served.
- Validation: Compare sample queries and result counts between old and rebuilt indices. Run checksums or hash-based validation where feasible.
- Scheduling: Avoid peak hours. For large datasets, coordinate rebuilds across regions and replicas to limit load spikes.
- Cost: Reindexing can be CPU-, memory-, and network-intensive. Factor compute and time into maintenance windows and budgets.
Avoid running full rebuilds during peak traffic. Schedule reindexing for low-traffic windows and use validation and an atomic swap to prevent serving incorrect or partial results.
