Technology

Every single one of us, as kids, learned the concept of "garbage in, garbage out"; most likely in terms of diet and food intake.

And yet every AI cultist makes the shocked pikachu face when they figure out that trying to improve your LLM by feeding it on data generated by literally the inferior LLM you're trying to improve, is an exercise in diminishing returns and generational degradation in quality.

Why has the world gotten both "more intelligent" and yet fundamentally more stupid at the same time? Serious question.

Oh no. Anyways...

oh no are we gonna have to appreciate the art of human beings? ew. what if they want compensation‽

If mainstream blogs are writing about it, what would make someone think that AI companies haven't thoroughly dissected the problem and are already working on filtering out AI fingerprints from the training data set? If they can make a sophisticated LLM, chances are they can find methods to XOR out generated content.

Oh no

Sooner or later it is supposed to happen, but I don't think we are quite there....Yet.

Fake news, just like that one time Nightshade "killed" stable diffusion (literally had no effect) Flux came out not long ago and it's better than ever

is it not relatively trivial to pre-vet content before they train it? at least with aigen text it should be.

Kind of like how true thoughts and opinions on complex topics are boiled down to digestible concepts for others to understand who then perpetuate those concepts without understanding them and the meaning degrades and we dont think anymore, just repeat stuff in social media comments.

Side note... this article sucks and seems like it was ai generated. Repetitive and no author credit? Just says it was originally posted elsewhere.

Generative AI isnt in danger of being killed as this clickbait titled suggests... just hindered.

More like degenerative AIs

It is their own fault for poisoning the internet with their slop.

"Model collapse" is just a fancy way of saying "our stupid ideas are bad and nobody wants them."

Old news? Seems to be a subject of several papers for some time now. Synthetic data has been used successfully already for very specific domains.

Let's go, already!

How you can help: If you run a website and can filter traffic by user agent, get a list of the known AI scrapers agent strings and selectively redirect their requests to pre-generated AI slop. Regular visitors will see the content and the LLM scraper bots will scrape their own slop and, hopefully, train on it.

…………………. Good?

The solution for this is usually counter training. Granted my experience is on the opposite end training ai vision systems to id real objects.

So you train up your detector ai on hand tagged images. When it gets good you use it to train a generator ai until the generator is good at fooling the detector.

Then you train the detector on new tagged real data and the new ai generated data. Once it's good at detection again you train the generator ai on the new detector.

Repeate several times and you usually get a solid detector and a good generator as a side effect.

The thing is you need new real human tagged data for each new generation. None of the companies want to generate new human tagged data sets as it's expensive.

Model collapse is just a euphemism for “we ran out of stuff to steal”

More like... Degenerative AI *ba dum tsss

Wait now hold on a minute. Why would I want to do this? Is this activism by people against LLMs in general or..? I'm confused as to why I would want to do this.