Like 0

Interview prep notes

Revision 9e3e99d370ce86d8bad39266b68136fd4c74644b

adclick.md Raw

System Design Interview: Designing an Ad Click Aggregator

Advertisers are billed on click counts, so the numbers must be correct, and they want them within a minute.

Requirements

  • Ingest click events (ad_id, user, timestamp, ...) at ~50k per second, peaks of 5x.
  • Query clicks per ad per minute for the last N minutes, and top-K ads per minute.
  • Counts must be accurate for billing; late and duplicate events happen.

Pipeline

  1. Click endpoint logs the event and redirects the user to the advertiser. Each click gets a unique ID (signed into the ad URL) so duplicates can be detected.
  2. Events go into Kafka, partitioned by ad_id.
  3. A stream processor (Flink-style) aggregates into 1-minute tumbling windows keyed by ad, emitting (ad_id, minute, count) rows.
  4. Results land in an OLAP store (ClickHouse, Druid, or Cassandra keyed by ad and minute) that serves dashboards.

Correctness details

  • Event time, not processing time: aggregate by when the click happened. Use watermarks to decide when a window is complete, and allow a lateness budget (say 2 minutes) for stragglers; very late events go to a correction path.
  • Dedup: keep recently seen click IDs per window in state (or a Bloom filter backed by an exact check) and drop repeats.
  • Exactly-once output: the processor checkpoints its state and Kafka offsets together and writes results idempotently (upsert by ad_id, minute).

Reconciliation

Also write raw events to object storage. A nightly batch job recomputes counts from the raw log and compares them with the streaming results; billing uses the reconciled numbers. This is the classic Lambda-style safety net, and interviewers like hearing it for money-related systems.

Hot ads

A viral ad can overload one partition. Split its key (ad_id#0..7), aggregate the pieces, then sum them in a second stage.


Disclosure: I'm with Linkjob AI, which offers AI help for live interviews.