> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.
# News Feed Media Viral Posts and the Wrap Up
> Describes scalable media storage and feed architecture using object storage and CDN, Redis-based feed caching, hybrid push pull fan out, and strategies for viral posts and celebrity accounts.
We’ve covered the feed design extensively. The fourth clarifying interview question asks: posts can include photos and videos as well as text—where should those media files live, and how do we serve them efficiently?
Important constraint: we do not store media binaries (images/videos) directly in the posts table. Persisting raw media in the relational database makes it large, slow, and harder to scale. Instead, each post row contains the post text and one or more URLs that reference the media.
Do not store large media blobs in your primary database. Databases are optimized for structured data and transactional consistency—not for serving high-bandwidth binary assets. Use object storage + CDN instead.
Typical media storage and delivery flow
* Client uploads the image/video to an object store (for example, [Amazon S3](https://aws.amazon.com/s3/)).
* The object store is fronted by a CDN which caches and accelerates delivery to end users.
* The app, feed caches, and the client all use the CDN URL to load the media.
* The database stores only the media URL(s), never the binary.
* The second-level post cache stores these same URLs so the caches and DB never carry the heavy binary payloads.
This design keeps the database and cache layers compact and lets the CDN handle bandwidth-heavy delivery.
Media delivery stack (summary)
| Layer | Responsibility | Example / Notes |
| - | - | - |
| Object storage | Durable storage of media binaries | `Amazon S3`, `Google Cloud Storage` |
| CDN | Global caching, edge delivery, TLS, caching headers | `CloudFront`, `Fastly` |
| Application | Generates signed URLs, records media metadata | Store only URLs in DB |
| Cache (second-level) | Store assembled post objects with media URLs | Keep assets out of DB and caches |
| Client | Requests CDN URL for media | Browser/mobile app fetches media directly from CDN |
Handling viral posts and extreme read load
A viral post creates a concentrated, heavy read hotspot. If that post’s cache entry is only on a single cache node, that node becomes a bottleneck. Strategies to handle viral traffic:
* Replicate hot cache entries across multiple cache instances, or use a distributed cache that automatically replicates hot keys.
* Load-balance read requests across those cache copies.
* Offload bulk media delivery to the CDN so application and cache nodes are not overwhelmed.
Fan-out behavior for posts (write vs read)
* For normal users:
* Fan-out on write (push): When a user posts, we push the post ID into each follower’s Redis feed list via background workers.
* Each Redis list retains up to a configured limit (e.g., 500 IDs) to bound memory.
* For celebrities / high-follower-count users:
* Avoid pushing to millions of followers at write-time (write amplification).
* Use a hybrid push/pull model: keep a precomputed feed for the ordinary follower set, but pull recent celebrity posts at read-time and merge them in.
* For inactive users:
* Do not precompute feed lists. Build their feed on demand when they return.
When a regular user creates a post (detailed flow)
| Step | Component | Action |
| - | - | - |
| 1 | Database | Save post row and a small outbox note in a single transaction |
| 2 | Outbox scanner | Background worker reads outbox and enqueues a job |
| 3 | Job queue/worker | Worker takes job and pushes the post ID into followers’ Redis lists |
| 4 | Redis | Each follower’s feed list keeps up to N post IDs (e.g., 500) |
Celebrity / high-follower model (read-optimized)
* At read time, the GetFeed API:
* Reads the user’s precomputed Redis list.
* Fetches post objects from the second-level cache.
* Pulls recent posts from celebrities the user follows (on-demand).
* Merges precomputed posts + pulled celebrity posts, sorts them, and returns top N (e.g., 50) to the client.
Inactive users
* Do not allocate cache/list resources for rarely active users.
* When they return, build their feed on demand (pull + merge).
We merge both the precomputed posts (from Redis lists) and the pulled celebrity posts, then return the top N (commonly 50) posts to the client. This hybrid push/pull design balances write-time work, read-time latency, and overall scalability while ensuring media traffic is handled efficiently by object storage + CDN.
Design tips:
* Store only media URLs in your DB and caches; never the binaries.
* Serve media via object storage + CDN for scalability and low latency.
* For viral posts, replicate hot cache entries or use an auto-replicating distributed cache to avoid single-node hotspots.
* Apply a hybrid fan-out/fan-in strategy for celebrity accounts to limit write amplification while keeping feed reads fast.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.