this post was submitted on 04 Nov 2023
265 points (92.9% liked)
Technology
59402 readers
2532 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
This isn’t anything they actively did though. The literal point of AI is that it learns on its own and comes up with its own response absent human interaction. Meta very likely specifically added code to try and prevent this, but it just fell short of overcoming the bias found in the overwhelming majority of content that led to the model associating Hamas with Palestine.
It's up to them to moderate the content generated by their app.
And yes it's almost impossible to have a completely safe AI so that will be an issue for all generative AIs like that. It's still their implementation and content generated by their code.
Also I highly doubt they had a specific code to prevent that kind of depiction of Palestinian kids.
Even if they did, someone will come up with an injection prompt that overrides the code in question and the AI will again display biased or racist stuff.
An AI generating racist stuff is absolutely not more acceptable because it got inspired by real racist people...
I imagine they likely have hardcoded rules about associating content indexed as “terrorist” against a query for a nationality. Most mainstream AI models do have specific rules built in to prevent stuff like this, they just aren’t all encompassing and can still happen if there is sufficient influence from the training data.
While FB does have content moderators, needing human verification of every single piece of AI generated defeats the purpose of AI. If people want AI there is a certain amount of non politically correct results that will slip through the cracks. The bottom line is content moderation as we know it has extreme biases applied to fit the safest “viewpoint model” and any system based on objective data analysis, especially with biased samples such as openly available internet, is going to get results that do not fit the standard “curated” viewpoint.
It doesn't matter. I don't really care about moderation being impossible to do. Google decided most moderation should be done automatically on YT and there are constantly false positives. They are not being held accountable both for false positives and false negatives. No human is involved.
And reading that type of comment I'm assuming we are heading the same way. Businesses not being accountable for something that is absolutely being generated by their code. If you choose to deploy a black box that generates random stuff you can't understand how it was generated it shouldn't make you not responsible for the damage done.
I don't think we should naively just accept apologies from AI owners and move on. They knew the risk of dangerous content being generated and decided it was acceptable.
Also considering the damage that Facebook has done in the past and their careless attitude toward privacy, I cannot understand why you would find it likely that they took the time to add some kind of safeguard for nationality and terrorism to be wrongfully associated.
Even then, the very concept of nationality is certainly not clear for an AI. For some Palestine is not a country. How would you think they would have coded a safeguard to prevent that kind of mistake anyway ?
There is a contradiction also in saying that you can't moderate every single AI output manually but that they manually added a moderation of sort to the AI specifically for Palestinians and terrorism. There is no way they got so specific. As you said it's not a practical approach.
The very important point for me to convey is that just because some black box generating text can randomly say racist stuff doesn't and shouldn't be more socially acceptable. That's it.
Then obviously I think these AI shouldn't have been released before their owners have a very good understanding on how they work and on how to prevent 99.9999999999% of the dangerous outputs. Right now my opinion is that Whatsapp deployed this knowing a lot of racist stuff would be generated and they just decided they will figure it out along the way with the help of the users.
It was either that or being late to the competition for the AI market.
If an innocent user can generate that easily some racist output I would argue they did not responsibly released this AI.