CodePark/ questions / Monitoring, errors & rollout
START PRACTICINGPRACTICE
Frontend question · Frontend system design › Monitoring, errors & rollout

How would you run an A/B test on the frontend without hurting performance or skewing results?

Assign users on the server, render the right variant in the HTML, and decide the metrics and duration before you start.

Assignment: put users into buckets on the server or at the edge (a CDN server near the user), using a stable id. Render the chosen variant in the HTML. Assigning on the client after load causes problems:

  • a flicker of the control version (the current version), plus layout shift
  • blocking scripts that hide the page until the experiment loads, which hurts LCP (Largest Contentful Paint: when the biggest image or text block appears)

Code: variants sit behind the same flag system as feature flags. Large variants are code-split, so users only download their own variant.

Measurement:

  • Pick one primary metric and some guardrails before starting. Guardrails are metrics that must not get worse, like errors, performance and revenue.
  • Log exposure when the user actually sees the variant, not at assignment. Otherwise the effect gets diluted.
  • Run for a pre-planned duration and sample size. Do not stop early because the result looks good. Checking early like this is called peeking.

Hygiene:

  • Avoid overlapping experiments on the same surface, unless they are designed for it.
  • Exclude bots and internal users.
  • Clean up the losing variant afterwards.
  • Assign variants on the server or at the edge to avoid flicker.
  • Code-split the variants.
  • Set the primary metric and guardrails up front.
  • Log exposure when the user sees the variant, not at assignment.
  • Run for a planned duration, and do not peek at results early.