Design a News Feed

The problem

A news feed shows each user the recent posts of the accounts they follow, newest (or most relevant) first, and keeps loading older ones as they scroll. Every social app has one, and it is a favorite interview question because the obvious design falls over at scale in an interesting way: reading is easy until a few accounts have millions of followers.

The basics come first; the Going deeper parts cover what experienced engineers get asked about.

Step 1: Requirements

Try it first: what would you put on each list?

Functional:

• Users create posts (text, maybe an image).
• Users follow and unfollow other users.
• Users see a feed of posts from the accounts they follow, newest first, and can scroll back through older posts.

Non-functional:

• Opening the feed is fast: a few hundred milliseconds at most.
• The feed is highly available. If a post shows up a few seconds late, that is fine (eventual consistency).
• It scales to tens of millions of daily users, including accounts with millions of followers.

Out of scope, said aloud: ranking with machine learning, comments, likes, ads. Mention ranking as a later extension; design for a chronological feed first.

Step 2: Estimates

• Users: 50 million daily active users.
• Feed reads: 10 opens a day each = 500 million a day ≈ 6,000 per second, with peaks several times higher.
• Posts: if 10% post once a day, that is 5 million posts a day ≈ 60 per second.
• Fan-out: with 200 followers on average, each post lands in 200 feeds: 1 billion feed inserts a day ≈ 12,000 per second.
• Celebrities: one account with 50 million followers turns a single post into 50 million inserts.

The headline: reads outnumber posts by about 100 to 1, and the follower distribution is extremely uneven. Both facts shape the design.

Step 3: Core entities and API

users     (id, name, ...)
follows   (follower_id, followee_id, created_at)
posts     (id, author_id, created_at, text, media_url)
feeds     (user_id → ordered list of post IDs)   ← a cache, rebuildable

POST /posts                       { "text": "...", "media_url": "..." }   → 201 { "id": "..." }
POST /follows                     { "followee_id": "..." }
GET  /feed?limit=20&cursor=...    → { "posts": [...], "next_cursor": "..." }

The feed endpoint returns a cursor, not a page number (Step 6 explains why).

Step 4: High-level design

Start with the simplest version: build the feed when it is requested.

It works for a small app and it is always fresh. But each feed open fans out to hundreds of authors and merges their posts: about 6,000 times a second, on the path users wait for. Since reads are 100 times more common than posts, it makes sense to move the work to the write side.

Step 5: Deep dive — building the feed

Bad: fan-out on read for everyone. As above. Simple, but every read does the expensive part, and a user who follows 2,000 accounts waits the longest.

Good: fan-out on write. When someone posts, put the post on a queue. Fan-out workers look up the author's followers and prepend the post ID to each follower's feed, kept in a cache (a Redis sorted set per user, scored by time, works well). Opening a feed becomes one cache read. Keep each feed capped (say, the newest 800 IDs); older posts can come from the database on the rare deep scroll. Twitter's 2013 "Timelines at Scale" talk described this kind of fan-out into in-memory timelines.

The cost moves to writes, and it is uneven: an author with 200 followers costs 200 inserts, but a celebrity with 50 million followers costs 50 million, and every post queued behind it waits.

Great: a hybrid. Fan out on write for normal accounts. For accounts above a follower threshold (say 100,000), skip fan-out; when a user opens their feed, fetch the recent posts of the few celebrities they follow and merge them in. Each read adds a handful of lookups (cheap, and cacheable, since everyone reads the same celebrity posts), and the write storm disappears.

Going deeper: inactive users and backfill

Fanning out to users who have not opened the app in months wastes most of the work. A common refinement is to fan out only to recently active users; when an inactive user returns, build their feed once on read (fan-out on read, just this time) and then switch them back to fan-out on write.

Following a new account only affects future posts, so a background job backfills that account's recent posts into the follower's feed. Unfollowing either removes entries the same way or filters them at read time.

Step 6: Deep dive — pagination and hydration

Pagination. ?offset=40 breaks on a live feed: every new post shifts everything down, so the next page repeats posts the user has already seen. Use a cursor: the last item's (created_at, post_id), and ask for "the next 20 older than this." It stays stable as new posts arrive, and it maps directly onto a sorted-set range query.

Hydration. The feed cache stores post IDs only. Reading a page means taking 20 IDs, then fetching the posts in one multi-get from a post cache (falling back to the posts database). Each feed entry stays tiny, an edited post is fixed in one place, and author names and like counts come from their own caches. Deleted posts or blocked authors are filtered out during hydration.

Step 7: Scaling and the final design

• Posts database: shard by author_id (a user's posts live together, which suits both fan-out on read and profile pages). Read replicas for the many reads.
• Follows: a large table queried both ways (who follows X, whom does X follow); shard by one side and keep a second index for the other, or use a graph store built for it.
• Feed cache: shard by user_id. It is a cache: if a feed is lost, rebuild it with fan-out on read.
• Fan-out workers: scale with the queue backlog; give celebrities' posts nothing to do (hybrid) so the queue stays healthy.

Going deeper: ranking and consistency

A ranked feed keeps the same skeleton: candidate generation (the fan-out feed plus celebrity posts) produces a few hundred candidates, and a ranking service scores and orders them per request. The cache then holds candidates, not the final order.

On consistency: a user who posts and immediately opens their own feed expects to see the post. Fan-out is asynchronous, so add the user's own recent posts at read time (read-your-writes), rather than waiting on the queue.

What interviewers expect at each level

• Early career: gets a working design with the data model, a feed endpoint, and fan-out on read or write, and can explain the read-heavy ratio.
• Mid-level (2–5 years): compares fan-out on read and on write with numbers, uses a cache of post IDs, cursor pagination and asynchronous workers, and spots the celebrity problem.
• Senior: proposes the hybrid with a threshold, handles inactive users, backfill and read-your-writes, explains sharding keys for posts, follows and feeds, and says what to monitor (fan-out lag, cache hit rate, feed latency percentiles).

Remember this

• Reads outnumber posts about 100 to 1, so do the work at write time: fan out post IDs into cached feeds.
• Very large accounts break fan-out on write. Merge their posts in at read time instead.
• Paginate with a cursor, never an offset, on a feed that changes while you scroll.
• Cache IDs in feeds and hydrate them from a post cache; feeds are rebuildable caches, not the source of truth.