Technology

87162 readers

3855 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 3 years ago

MODERATORS

L3s@lemmy.world

enu@lemmy.world

technopagan@lemmy.world

L4s@lemmy.world

L3s@hackingne.ws

447

Project Analyzing Human Language Usage Shuts Down Because ‘Generative AI Has Polluted the Data’ (www.404media.co)

submitted 2 years ago by Stopthatgirl7@lemmy.world to c/technology@lemmy.world

13 comments fedilink hide all child comments

The creator of an open source project that scraped the internet to determine the ever-changing popularity of different words in human language usage says that they are sunsetting the project because generative AI spam has poisoned the internet to a level where the project no longer has any utility.

Wordfreq is a program that tracked the ever-changing ways people used more than 40 different languages by analyzing millions of sources across Wikipedia, movie and TV subtitles, news articles, books, websites, Twitter, and Reddit. The system could be used to analyze changing language habits as slang and popular culture changed and language evolved, and was a resource for academics who study such things. In a note on the project’s GitHub, creator Robyn Speer wrote that the project “will not be updated anymore.”

all 15 comments

sorted by: hot top controversial new old

[–] grue@lemmy.world 204 points 2 years ago (2 children)

The project creator doesn't mince words:

wordfreq was built by collecting a whole lot of text in a lot of languages. That used to be a pretty reasonable thing to do, and not the kind of thing someone would be likely to object to. Now, the text-slurping tools are mostly used for training generative AI, and people are quite rightly on the defensive. If someone is collecting all the text from your books, articles, Web site, or public posts, it's very likely because they are creating a plagiarism machine that will claim your words as its own.

So I don't want to work on anything that could be confused with generative AI, or that could benefit generative AI.

OpenAI and Google can collect their own damn data. I hope they have to pay a very high price for it, and I hope they're constantly cursing the mess that they made themselves.

[–] Solumbran@lemmy.world 102 points 2 years ago (1 children)

Seems pretty mild and reasonable, to be honest.

[–] kn33@lemmy.world 78 points 2 years ago

Yeah, it seems really restrained for someone who has to end a project they've put so much effort into.

[+] Randomgal@lemmy.ca -33 points 2 years ago (3 children)

NGL sounds like a butthurt dude. Emotional arguments without logic.

[–] Croquette@sh.itjust.works 24 points 2 years ago

I'd be fucking butthurt as well if my pet project was being destroyed by mega corpos for a shitty generative thief AI.

[–] jaggedrobotpubes@lemmy.world 11 points 2 years ago

This does not say wonders about reading comprehension.

[–] SirQuackTheDuck@lemmy.world 1 points 2 years ago* (last edited 2 years ago) (1 children)

Imagine being an author whose sole income is writing books.

Here comes an AI that ~~stole~~ indexed your work and is asked by a customer of OpenAI to summarise your books. It does so perfectly and the issuer is able to use your results freely, since they think it's AI generated and doesn't require attribution.

You receive nothing in return.

Good luck making a living.

Edit: stole to indexed, added edit note

[–] Gorillazrule@lemmy.dbzer0.com 2 points 2 years ago

This is such a nothing argument. If all you're talking about is a summary of a book, people have been able to get that long before AI. I can go to a wikipedia entry right now of any book and look at a plot summary. The author does not get paid for me looking at the summary on Wikipedia. There are numerous other sites where you can find summaries of books. And if you're asking an AI for a summary of a specific book by a specific author, what attribution would you like to see? The user already knows the source because they're specifically asking for a summary of that source.

A bigger concern would be the AI reproducing your works and using them in responses.

[–] kibiz0r@midwest.social 53 points 2 years ago

Don’t worry. Someone will soon come by to remind us that it’s pointless to regulate AI, and also harmful to do it, and it’s actually a good thing for everyone, and also we’ll be shoveling shit until we die if we don’t get on board, and please oh please just let me get off to one more deepfake of my classmate before you take away my toy it’s not faiiiiir.

[–] Greg@lemmy.ca 25 points 2 years ago (1 children)

Hello fellow humans, I like digesting avian flesh. Beep boop