Mobile apps and browsers make API calls to the server. The server exposes endpoints; the client invokes them to create posts and to fetch feeds.
- POST /create-post — Clients send the post payload (author + content) to this endpoint so the server can persist the post.
- GET /get-feed — Clients request a user’s feed. For our baseline implementation the server returns the top 50 most-recent posts from the accounts the user follows.
- Create a post (client → server)
- Get a feed (client → server)
- Example server-side responsibilities for GET /get-feed:
- Read the follow list for the requesting user.
- Fetch recent posts from those authors.
- Merge, rank, and return the top N posts (e.g., 50).
- users — user metadata and flags used by ranking/fan-out logic
- follows — follower → followee relationships (large, often billions of rows)
- posts — every post; indexed by author and time for efficient recent-post queries
- Stores user metadata such as
user_id, display name, small profile text, and a booleancelebrityflag. - The
celebrityflag is useful for ranking and for choosing more aggressive fan-out-on-write behavior for high-follower accounts.
Follows table
- Each row represents one follower → followee relationship. This table can grow extremely large (for example, 10M users × 200 follows = ~2B rows).
- Two essential queries:
- Given user X, which accounts does X follow? (Used when building X’s feed.)
- Given account Y, who follows Y? (Useful for certain feed-generation strategies and notifications.)
Posts table
- Stores every post. Typical columns:
post_id,author_id,content,created_at. - Index by
(author_id, created_at DESC)so queries like “get John’s most recent posts” are efficient. This also supports fetching recent posts from multiple authors when merging to produce a feed.
Access patterns — what the server must do
- Create post
- Client calls POST /create-post with
{ author_id, content }. - Server inserts a new row into
posts.
- Client calls POST /create-post with
- Get feed
- Client calls GET /get-feed.
- Server:
- Reads the list of accounts the user follows from
follows. - Reads recent posts from those accounts from
posts. - Merges (time-merge or ranked merge) and returns the top 50 results.
- Reads the list of accounts the user follows from
Scaling considerations (brief)
- Fan-out on write vs fan-out on read:
- Fan-out on write: push new posts into each follower’s materialized feed when created. Pros: fast reads. Cons: expensive writes for high-follower accounts (celebrities).
- Fan-out on read: store only the posts; at read time fetch recent posts from followees and merge. Pros: cheaper writes. Cons: heavier read latency and more complex merging/caching.
- Hybrid approaches: use fan-out-on-write for normal users and specialized strategies for celebrities (e.g., served from cached global timelines).
- Caching and rate-limiting: use edge caches (CDN or in-memory caches) and apply limits for heavy users.
- Indexing: ensure efficient indexes on
(author_id, created_at)and onfollowerorfolloweedepending on read patterns. - Ranking: use
users.celebrityand other profile signals to influence ranking and to decide fan-out behavior.
- Decide your primary scale target (writes-heavy vs reads-heavy) and test both fan-out strategies at small scale.
- Add endpoints for likes/comments/follow to evolve the graph and measure query patterns.
- Instrument metrics (latency, QPS, tail latencies) and run load tests simulating real-follow graphs.
- Fan-out strategies and lessons from large-scale feeds:
- For further reading on data modeling and scaling: