Someone made that cake mix, though. A team of people came together, worked out how to make that mix at scale, tasted, iterated, and finally shipped the product.

Nobody cares. I don't think you care. I'm serious - I don't think, before today, when you ate a cake you bought from a large store or made yourself from a box mix, that you got any measurable amount of enjoyment from the mere fact that the product was developed by people. I think you got 99% of your enjoyment from the cake being delicious.

This is getting cake mix by scooping up a random amount of the statistically most likely ingredients for a cake.

The statistically most likely ingredients for a cake are going to make a good cake. I actually have a cookbook that has a recipe for "the average cookie" which literally takes an arithmetic mean of the ingredients in a dozen or so cookie recipes, and the result is quite good. According to the book, I mean, because I never had the will to bake cookies containing 2.35 eggs or however much it ended up being. (The book is "cooking for geeks" and is excellent, by the way)

The problem with averaging stuff is that you might accidentally average out more than one completely different type of cake and end up with with a little bit of cocoa powder, a little lemon, a little coffee, a little pistachio, etc etc. But this problem doesn't really occur with AI in the way you're implying, because it's statistically very unlikely to have all of those ingredients in the same recipe.

Another problem with "most likely" is that you might only get vanilla sponge cakes because they're the most common - most likely. But this also doesn't happen, because it's very unlikely that someone would reply with a vanilla sponge cake recipe to a request for a sticky chocolate fudge cake.

We can see the problems with AI content in the OP's post, but that's not really relevant to the comment that set off this sub-thread and analogy which was asking why people would use AI for something like this. If you'll let me speculate, I think what you really want to say is that AI comes up with some utter shite, and you're trying to talk in broad terms about how AI works to explain how that happens - but if that is what's going on, you're off the mark. It'd be better to just bring us back to the point that AI comes up with some utter shite! Of course, bringing that back to the question, people can be quite tolerant of shit responses in some situations, like if there are no consequences for mistakes, or if they can filter the output for quality themselves.