SproutStackSproutStack home
SeniorCaching

An old name comes back for a day

Writes update the database and delete the key, yet some users see stale profiles for hours.

The setup today

User profiles use cache-aside. A write updates the primary database and then deletes the profile's cache key. Cache misses are filled from a read replica to keep load off the primary. Cached profiles live for a day.

Current configuration

write: UPDATE users ... (primary)  ->  DEL profile:{id}
read:  GET profile:{id}  ->  miss  ->  SELECT ... (replica)  ->  SET profile:{id} ... EX 86400
replica lag: usually under 50 ms, up to 200 ms at peak

What you know

  • A few times a day, a user changes their display name and other people keep seeing the old one for up to a day.
  • Writes and deletes are confirmed as successful in the logs.

The new requirement

Product wants profile edits visible to everyone within a few seconds, every time.

Make the call

2 questions · first-try answers earn the most XP

1. What is the cause?
2. Which change closes the gap?
What a strong answer also covers
  • Facebook's NSDI 2013 memcache paper uses leases for exactly this stale-set race, and for stampedes.
  • Invalidating from the database's change log (CDC) catches writes from every code path, not just the ones that remember to delete.
  • Keep TTLs short enough that any missed case heals quickly, and measure staleness rather than assuming it.
  • Reading from the primary on misses adds load: estimate the miss rate before switching.

The background

More drills