Programming

Ran into a situation last week that got me thinking about how most teams handle visual changes in production.

We pushed a CSS update that was supposed to fix button alignment on one page. Looked fine in staging. Looked fine in the PR preview. Got merged, deployed, and broke the card layout on three other pages because of a shared utility class.

Caught it about four hours later when a customer mentioned it in a support ticket. Not a great look.

What we tried

After that incident we talked about adding visual regression testing to CI. Looked at a few options:

• Percy — solid, but the pricing gets steep once you have more than a handful of pages and multiple PRs a day

• Playwright screenshots in CI — free, but you need to maintain baseline images and the diffs are noisy. Every font rendering difference across runners triggers a false positive

• Manual spot checks after deploy — what we were doing. Obviously not sufficient

We ended up going with Playwright screenshot comparisons but with a pretty generous diff threshold (0.3%) to cut down on false positives. It catches big layout shifts but ignores sub-pixel rendering differences. Good enough for now, but not ideal.

The production monitoring gap

CI-based visual testing only covers what happens before deploy. It doesn't catch things that break after — CDN caching serving old assets, third-party scripts injecting unexpected elements, A/B test variants rendering wrong.

Some teams run synthetic monitoring that takes periodic screenshots of production pages and compares them. That covers the post-deploy gap but it's a separate system to set up and maintain.

Curious what others do

How are you handling visual verification in production? Specifically:

• Do you run any visual checks post-deploy, or only in CI?

• If you use screenshot comparison, how do you deal with dynamic content (dates, user-specific data, ads)?

• Anyone running continuous visual monitoring on production pages on a schedule?

Feels like there should be a simpler answer than "maintain 200 baseline images and pray the CI runner has the same font rendering as last time."