this post was submitted on 17 May 2026
463 points (97.5% liked)
Technology
84748 readers
4661 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
My point is that for Agentic AI mistakes with catastrophic consequences are just as likelly as minor mistakes, which is not the case for people because humans can spot the "obviously stupid" or "obviously dangerous", plus they make more of an effort to avoid mistakes that can have very bad consequences, so they tend to make catastrophic mistakes will less probability than minor mistakes.
People giving psychological advice are incredibly unlikely to tell suicidal people to "kill yourself", those giving food recipes are incredibly unlikely to say that pizza should have glue on top or those deploying software in Production are incredibly unlikely to delete the whole fucking Production environment including backups.
So even if the total rate of mistakes of an an Agentic AI was less than a human, its rate of catastropic mistakes would still be much higher than a human.
This is however not obvious unless one actually analises the risk profile of using Agentic AI in a specific place in a specific process, a skill very few people have plus it requires information about and/or understanding of Agentic AI which itself very few people have and the AI vendors activelly do not want people to have.
So you end up with an e-mail fluffing and defluffing machine being used to summarize and store medical info about patients and then down the line somebody gets given something that kills them because the data on file had a critical mistake.
This is why I said that its "the worst possible consequence of a mistake done here" that limit Agentic AI suitability: because generally you're going to have way more catastrophic mistakes with an AI that you will even with even an human with no domain experience.
No AI was used in the creation of these clusterfucks:
The Lake Peigneur Maelstrom - In November 1980, Texaco was conducting exploratory oil drilling directly on top of a shallow, 10-foot-deep freshwater lake. Operating directly underneath that same lake was a massive, active multi-level salt mine
The Banqiao "Iron Dam" Collapse (China, 1975) Built in the early 1950s for flood control, the Banqiao earthen dam was heavily reinforced with Soviet engineering assistance and proudly nicknamed the "Iron Dam" by the government, which declared it completely unbreakable
The Capsizing of the Vasa Warship (Sweden, 1628) In 1628, King Gustavus Adolphus built the Vasa, an opulent warship meant to serve as the crown jewel of the Swedish Navy. It was designed to intimidate enemies with unprecedented firepower.
The gas tank in the back of the Ford Pinto.
The Tesla Cybertruck (well, maybe some AI got in there, but the core bad ideas were well established before ChatGPT was "a thing".)
Lead in gasoline
The Triangle Shirtwaist Factory Fire (New York, 1911) The Triangle Shirtwaist Factory occupied the top floors of a Manhattan building, employing hundreds of young immigrant women. Management routinely ignored basic industrial safety measures to maximize profits and prevent employee theft. The "Obviously Dangerous" Reality: Locking workers inside a high-rise room filled with flammable textiles and scraps creates a lethal death trap in an emergency.
Bhopal 1984
Chernobyl 1986
The Hillsborough Stadium Disaster (Sheffield, UK, 1989) During an FA Cup semifinal match between Liverpool and Nottingham Forest, thousands of fans arrived outside the Leppings Lane end of the stadium just before kickoff, creating a massive, chaotic bottleneck at the turnstiles. The "Obviously Dangerous" Reality: Opening a massive exit gate to let thousands of frantic people rush blindly down a narrow tunnel into an already overcrowded, fenced-in terrace creates a lethal human crush.
The Who - December 3, 1979 at the Riverfront Coliseum in Cincinnati, Ohio.
School shootings...
That's just not even true. People with no experience are going to fuck shit up completely. We have a human president and look where that's getting us.
As they always have.
Even people with zero experience in counseling don't tell a person who is thinking of committing suicide to "kill themselves" and even those with zero culinary experience don't tell others they're supposed to put glue on top pizza when you're making it.
To do that a human needs not just have zero experience but actually have no common sense whatsoever.
Further, even with such people, it's only if they've been given the tools to do things with a huge impact that it becomes a problem: that's pretty much "child with a loaded gun" situations.
The number of humans that inept given such power is minuscule (pretty much just children given loaded guns), whilst every single Agentic AI out there is that stupid and they're currently being given "loaded guns" all the time.
The problem is exactly that Agentic AIs are being given adult responsibilities and have the capacity for complex operations whilst having the common sense and reasoning abilities equivalent to those of a small child.