SproutStackSproutStack home
BeginnerEvent-driven

Checkout fails when analytics is down

Checkout calls three services in a row before replying. A fourth team wants in.

The setup today

After saving an order, checkout calls three services over HTTP, one after the other, and only then replies to the customer: email (receipt), loyalty (points) and analytics.

Current configuration

checkout -> POST email/send      timeout 5 s
         -> POST loyalty/points   timeout 5 s
         -> POST analytics/track  timeout 5 s
then reply 200 to the customer

What you know

  • Last week analytics was down for 20 minutes, and checkout was slow or failing the whole time.
  • Checkout p99 latency is 2.5 seconds, mostly spent in these calls.

The new requirement

Checkout must succeed even if any of the three is down, and the fraud team now wants every order too.

Make the call

2 questions · first-try answers earn the most XP

1. What design meets both needs?
2. Loyalty is down for an hour. What happens to points for orders in that hour?
What a strong answer also covers
  • Consumers must be idempotent, since delivery is at least once.
  • Publishing after saving is a dual write; the outbox pattern makes sure no event is lost.
  • Version the event schema so new fields do not break older consumers.
  • Monitor each subscriber's backlog and oldest message age, not just whether it is up.

The background

More drills