this post was submitted on 04 Dec 2025
60 points (95.5% liked)
Programming
23835 readers
97 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities !webdev@programming.dev
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
1500 tests is a lot. That doesn't mean anything if the tests aren't testing the right thing.
My experience was that it generates tests for the sake of generating them. Some are good. Many are useless. Without a good understanding of what it's generating, you have no way of knowing which are good and which are useless.
It ended up being faster for me to just learn the testing libraries and write my own tests. That way I was sure every test served a purpose and tested the right thing.
Yeah. Totally agree on this. I spend maybe 3-4h a day reviewing code, and these are my thoughts....
The LLM generated tests I see are generally of very low quality. Perfectly fitting the bill of looking like a test, but not actually being a good test.
They often don't test the precise expected value. As an overly simplistic example: They rarely check 2+2==4. But just assert 2+2>0, or often just that 2+2 doesn't cause an error.
The tests often contain mountains of redundancy. Again, an oversimplified example: They have a test for 2+2, and another for 2+3.
There is never any attempt to make the tests nice to read for humans. It is always just heaps of boilerplate code. No helpers introduced, or affordances to simplify test setup.
Coupling the proclivity for boilerplate together with subtly redundant tests makes for some very poor programming. Worse than I'd expect from a junior, tbh.
And 1500 tests... That is not necessarily a lot! If that is the output of 1 month of pumping out code, I would say bare minimum
30×50=1500, 50 tests per day is a lot. That is a lot to read and understand all the edge cases, let alone writing them.
That depends on what you count as a "test". In some langs/frameworks it is a lot, indeed.
30 is assuming you write code for all 30 days. In practice, it's closer to 20, so 75 tests per day. It's doable on some days for sure (if we include parameterized tests), but I don't strictly write code everyday either.
Still, I agree with them that you generally want to write a lot of tests, but volume is less important than quality and thoroughness. The author using the volume alone as a meaningful metric is nonsense.