> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.
# News Feed The Two Caches and the Database
> Explains precomputed feed architecture focusing on two caches, sizing examples, active-user optimization, database choices, and sharding strategies for scalable news feed systems
In this lesson we cover the final step of our feed design framework: identifying bottlenecks and planning for scale. For a precomputed feed architecture the two caches are the most important components — they keep latency low and make feed reads fast. Below we identify those caches, estimate their size, and discuss database choices and sharding strategies.
## The two caches: what they store and why they matter
### Cache 1 — Per-user feed lists (IDs)
* Purpose: store a precomputed, bounded list of post IDs per active user (the feed list).
* Typical list size: store up to 500 post IDs per user.
* ID size: approximately 8 bytes per post ID.
Sizing example:
* 500 post IDs × 8 bytes = 4,000 bytes ≈ 4 KB per user.
* For 10,000,000 active users: 4 KB × 10,000,000 = 40 GB (estimate; overhead and metadata will increase this).
Cache 1 is primarily keyed by user and optimized for fast reads of the feed list (IDs only).
### Cache 2 — Post content cache (payloads)
* Purpose: cache the actual post payloads (text, media pointers, metadata) so the app server can fetch content quickly after reading the per-user ID list.
* Example activity: if each active user posts on average 3 times per month, and there are 10M active users:
* 3 × 10,000,000 = 30,000,000 posts per month.
* If each post payload averages \~1 KB:
* 30,000,000 × 1 KB ≈ 30 GB for one month of posts.
In practice Cache 2 is sized to hold the “hot” subset of posts and uses eviction policies like LRU (Least Recently Used) to keep new and popular posts in memory.
## Quick sizing summary
| Cache | What it stores | Per-user / per-post estimate | Example size for 10M active users |
| - | - | -: | -: |
| Cache 1 | Per-user feed lists (post IDs) | 500 IDs × 8 bytes ≈ `4 KB` per user | ≈ `40 GB` |
| Cache 2 | Post payloads (content) | \~`1 KB` per post × 30M posts/month | ≈ `30 GB` (hot set only; evicted via LRU) |
## Active users only
* We typically only precompute feed lists for active users — for example, users who opened the app in the last 30 days (configurable).
* Avoid pushing updates into feed lists for users who haven’t used the app in months; this work is likely wasted and consumes cache capacity.
* If an inactive user returns, build their feed on read (generate the list when they open the app) rather than continuously updating their list while they are inactive.
Sizing numbers above are order-of-magnitude estimates to reason about capacity. In production measure actual post sizes, access patterns, and add overhead factors for cache metadata, replication, and other inefficiencies.
## Database considerations: choose technology by data shape
Different tables and access patterns map better to different storage systems.
| Data / Table | Characteristics | Recommended storage style |
| - | - | - |
| Posts table | Append-only, written once, time-ordered reads (author/time) | Wide-column / log-structured stores (e.g., [Cassandra](https://cassandra.apache.org/)) |
| Users / Follows | Small rows, many relationships, lookups in both directions | Relational DBs or other OLTP stores |
Growth and sharding
* At \~30M posts/month → \~360M posts/year. This growth requires planning for distributed storage.
* One machine cannot hold all posts at scale. Partition (shard) posts across multiple machines and design for rebalancing, replication, and index distribution.
* Consider read/write patterns, hotspots, and the cost of cross-shard joins when selecting a sharding key.
## Summary / operational checklist
* Two caches are central:
* Per-user feed lists (IDs): small, per-user, low-latency read.
* Post content cache: holds payloads for hot reads; eviction (LRU) keeps memory bounded.
* Precompute lists only for active users to reduce wasted work and storage.
* Choose database technologies based on data shape (append-only vs relational) and plan for sharding/storage growth.
* Monitor actual post sizes, request patterns, cache hit rates, and re-evaluate cache sizing and eviction policies regularly.
Further reading and references:
* Least Recently Used (LRU): [https://en.wikipedia.org/wiki/Least\_recently\_used](https://en.wikipedia.org/wiki/Least_recently_used)
* Cassandra: [https://cassandra.apache.org/](https://cassandra.apache.org/)
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.