Programming

Can you start to document the css in the source? So someone touches something, right above it is a comment: 'this class used for all card layouts on these type pages' or some such. Then there's a chance to spot the danger, in code review at least.

I agree with others here: visual testing should be last result as it is brittle, sweeping and numbing.

Depends how much it matters. See if you can get the business to put it in dollar value terms or at least as opportunity costs against other eng work.

Would be sort of a spike project, but would be interested to see how well off-the-shelf VLMs could handle screenshot comparisons, and then a bit extra to see how to deploy them efficiently and effectively.

Last time I worked on a large, long running web project, a person would manually check nearly everything. He kept a checklist of what to check, which covered nearly every page, modal, and dialog on staging before we deployed. We did try the snapshot thing for a little bit, but it caused more problems than it solved.

When I tried Percy it was really bad. I think it wasn't even doing real screenshots, but some horrible serialization of the dom nonsense. Some of what came out of their tests just wasn't accurate at all.

Every time I've tried to implement visual tests it's a bad time.

A good unit test encodes intent. This kind of test, and snapshot tests, tends to be too broad. Something changed but was it on purpose? I dunno maybe just accept it. People don't look at it closely. It's numbing.

As others have said, you probably want to stop having shit randomly shared in multiple places. If you change something used in more than one place, probably manually check.

I can't really provide a better answer — I would've recommended Playwright — but I will say this:

I really don't like sharing classes across UI elements. I know that bridge is likely crossed for you, but it violates separation of concerns. When I worked at a big automaker, instead of setting sizes on elements, we used a standard class like size-3, where everything computed to size value *4 pixels. It was the stupidest thing I've ever seen. The only thing you could do if you wanted to change element size or positioning is change the base value across the entire web UI. Or change the class. Just like you could change the size on the element itself.

Standards like this are why I left "full stack" for back-end only. I can do everything on the client end, but I can't stand how front ends are built.

I can say a little bit on tooling: We’ve experimented with Chromatic. We’re on a mobile app though, so the idea with Chromatic is to render via react-native-web, which has its own issues, so the ROI is such that we haven’t taken it beyond POC. Seems nice though, and probably more in line with your domain.

But for org strategy, I take it you’re on a much bigger team than I ever have been. (10 people max for me) So I can’t speak to personal experience there, but what I will say is that being able to control your entire env top to bottom is crucial for pretty much anything where you want high fidelity testing.

Like, being able to set up an env that allows you to play with stuff like externally-sourced ads and CDN caches in a space that’s private and temporary. No small feat, but that’s pretty much what you have to do if you want to catch those Rube Goldberg style failures before prod.

It depends how much it’s worth it to you, to have a test env that’s 99.999% realistic. As with anything, the first two 9s are the cheapest.

I've used BackstopJS, but it sounds a lot like Playwright. You can target and exclude certain elements, which is how I test individual components or exclude dynamic content. It's old and janky but it's better than nothing I guess.

Playwright is a lot of maintenance.

As a dev, I usually have a good idea of what my change may "touch". I write detailed notes for testers. I fixed a dropdown box glitch this week, and I had to mention the one other dropdown that called this particular function in a place you wouldn't expect.

We're told to make changes that "touch" as little as possible. Obviously because this stuff is a nightmare to test for regressions.