System Design · Design Question
Design a News Feed
Social timeline: fan-out on write vs read, ranking, caching, and celebrity problem.
1. Requirements
Functional
- User posts content; followers see it in home feed.
- Feed is ranked (recency + affinity + quality), not only chronological.
- Pagination / infinite scroll; like and view author profile posts.
Non-functional
- Low latency scroll (p99 tens–hundreds of ms for feed page).
- Eventual consistency OK for a few seconds after publish.
- Scale: millions of users; celebrities with millions of followers.
Estimation snippet
DAU 100M; 2 posts/user/day → ~2.3k post writes/s avg
Read: 20 feed opens/user/day → ~23k feed reads/s avg (peaks higher)
Fan-out: avg 200 followers → 2.3k * 200 ≈ 460k inbox writes/s → async workers
Celebrity 10M followers: never sync fan-out; hybrid read path
Ask: group feed vs home feed? ads? media? mutual friends ranking?
Start: home feed of posts from people you follow
2. API
POST /v1/posts { text, mediaIds[] }
GET /v1/feed/home?cursor=&limit=20
→ { items: [{ postId, author, text, rankScore, ... }], nextCursor }
POST /v1/posts/{id}/like
GET /v1/users/{id}/posts
3. Data model
users, follows(follower_id, followee_id)
posts(post_id, author_id, text, created_at, ...)
feed_inbox(user_id, post_id, rank_key, created_at) -- precomputed timeline
PK / cluster: (user_id, rank_key DESC)
counters / engagement in Redis or separate store
4. Fan-out strategies
| Fan-out on write | Fan-out on read | |
|---|---|---|
| Idea | On post, push post_id into each follower’s inbox | On feed read, query recent posts from followees and merge |
| Read cost | Cheap (read inbox) | Expensive merge/sort |
| Write cost | Expensive for huge follower counts | Cheap |
| Best for | Normal users | Celebrities / very high fan-out |
Hybrid (common interview answer):
- Regular authors: fan-out on write to followers' inboxes
- Celebrity authors: skip massive fan-out; mix their posts at read time
5. High-level design
Client → API → Post Service → Post DB
\
enqueue Fanout Job
|
v
Fanout Workers → write feed_inbox (Cassandra/Redis)
|
Feed Service ← read inbox + celebrity merge + rank
|
Cache (Redis) hot feed pages
6. Ranking
- Features: recency, relationship strength, engagement probability, content type.
- Offline ML → score model; online apply on candidate set (hundreds, not millions).
- Always keep a chronological fallback if ranker fails.
7. Caching
Cache: user:{id}:feed:page1 → list of post ids (short TTL)
Post objects cached separately by post_id
Invalidate/update on new post for active users (optional)
8. Scaling & celebrity problem
- Do not write 50M inbox rows synchronously on publish.
- Async workers; shard inbox by user_id.
- Celebrity: store posts only; at read, fetch top K celebrity posts and merge with inbox.
- Media via CDN; feed API returns URLs only.
9. Consistency
User may not see their own post instantly on another device if fan-out lags — mitigate by injecting self-posts into feed on read.
10. Edge cases
- Unfollow / mute → remove or filter future inbox rows; soft-delete old.
- Deleted post → tombstone; feed reads skip missing post bodies.
- Celebrity posts during peak → read-time merge must be bounded (top K).
- Empty follow graph → show recommended / global fallback, not blank forever.
Quick revision
- Home feed = candidates from follow graph + ranking.
- Fan-out on write for normal users; on read for celebrities.
- Precomputed inbox makes scroll fast.
- Async fan-out workers; never block publish on 1M writes.
- Cache feed pages and post bodies separately.
- Ranker on a small candidate set; chronological fallback.
- Inject author’s own posts to fix read-your-writes.