Technology

The people claiming it’s humanity ending are all directly involved in the AI companies. One take is that they are the ones that know the most about it.

My take is they are financially incentivized in making sure their investments grow in value.

The biggest risk I see is Idiocracy. Like, the movie.

Business workers become afraid to do or propose anything without AI signoff. Retail & service workers are told to do what the AI says, the way the AI says to do it, or get fired.

Technology innovator Marshall Brain (RIP) wrote a book in the early 2000s about an AI called "Manna" designed to replace fast food managers:

https://web.archive.org/web/20201204080050/https://marshallbrain.com/manna1

The AI just talks in your ear all day: "Empty one bag of fries into the fryer. You have 6.5 minutes, switch to the sandwich station. Make one Big Burger: grab one patty from the 1/4 pound bin and place it on the griddle... ... The fries are ready, turn around and lift them into the drain hopper..."

All day, every day, exactly 8 hours plus mandatory breaks, employees on autopilot listening on ear buds.

I knew a guy in high school that told girls they shouldn't date him because his dick was so big it would hurt them. You can wax philosophical on his motives if you'd like, but spoiler alert - his dick wasn't big.

Ed Zitron’s podcast “Better Offline” recently talked about this. There is a small tech community called the Rationalists who believe AI super intelligence is inevitable and it will either accelerate humanity to new heights or kill off the species. These rationalists are actually a very prominent faction within tech and the AI industry. Apparently the language used within top AI firms is largely about the immense danger posed by AI super intelligence if any other group or company were to achieve it first. This creates an echo chamber that makes the people in the industry kind of lose their mind and misinterpret the capability of LLMs.

Everyone outside that faction recognizes that LLMs are just unreliable cloud software with limited use cases and that there’s no evidence LLMs could actually lead to any kind of intelligence.

Edit: Please, someone stop me from commenting on an AI post next time I’m tempted. I’m so tired of people replying, arguing with opinions I don’t have, based entirely around a hypothetical technology that doesn’t exist yet. LLMs make slop. Thats it. They hacked a company after weeks of 100s of instances being run, using old exploits published online. San Francisco will be underwater from climate change before LLMs can do anything more.

Personally, this just feels like marketing and these big asshole companies trying to resurrect the 2025 level of hype in order to salvage their IPO's.

If they honestly believed this I can't imagine that their workers wouldn't do something more drastic.

I'm not pro-AI by any stretch for a huge number of reasons, but in my day job as a technical product manager I have to use a number of AI-powered tools and workflows. Whereas 18 months ago they were a novelty that the bosses were enamored of, today they get real work done and speed up a number of key areas, mainly like fancy automation workflows (which have been around forever). The rate of improvement and capability for certain use cases has been very, very rapid. We do seem to be reaching a plateau though. The tools and models from 6 months ago are just about as good as the ones from today. And China's models from today are better than the US's models from 6 months ago, but at a fraction of the cost.

So when I hear people talking about the "end of the world", I think a realistic case for that is that the AI tools that come out in the next few years will be good enough to do a fair amount of white-collar work, and we won't have a plan in place to deal with all of the now out-of-work people that leads to. Whole classes of jobs in areas like data entry, telemarketing, and customer support will be almost entirely automated out of existence. New classes of jobs will be created, of course, but it's hard to imagine them existing in sufficient numbers to make up for the losses. And this is just for LLM-workloads.

In other areas, other types of AI are already enabling robotic automation of factories and in the near-ish future might make self-driving truck fleets practical. This would be devastating because factory workers and truck drivers make up like 30% of the US workforce -- and again, there are no signs of new classes of jobs that could absorb the out-of-work workers, and no plans for a social safety net or wealth distribution.

So I'm not worried about a rogue AI unleashing some bioweapon or launching a nuclear attack. I'm worried about a bunch of CEOs deploying barely-acceptable AI to displace millions of workers as soon as it is safe(ish) to do so.

I'm more worried about the people using AI than the tech itself.

They are lying about their fear. That's all they can do. Stop believing any claim of theirs. Stop being manipulated.

They are snake oil salesman. They sell vaporware. Don't fucking buy it.

I've heard a few explanations for the "AI will kill us all" narrative.

1. AI techies are in a cult. They genuinely believe this stuff because they're crazy.

2. It makes the AI sound powerful and impressive, which is ultimately good for the AI companies.

3. They're starting to face competition from open-weight models, so they want to scare politicians into crafting regulations. They think that they will be able to influence the regulations to favor them and squeeze out their competition.

And, of course, there's

4. It keeps people talking about AI. Because whatever you do, don't quit talking about AI, because if you quit talking about it, the line might go down! And the line going down is very, very bad.

They want regulation to create a cartel. Failing that they want hysteria to increase pre IPO valuations.

As Patrick Boyle said. It would be like Coca Cola, Pepsi and Irn Bru getting together to beg for regulations because their soft drinks are just too delicious.

I feel like the AI hype has died and people are trying to prop it up with doomsday scaring.

These people are too high on their own supplies.

Humanity should be way more frightened of greedy psychopaths running the world than any technology.

AI industry makes money from doomsday prophecy about it being so intelligent that it will create a point of no return: the [singularity]. It is a magaian claim, a claim that is so increadibly stupid that one does not know where to even begin; and if you do begin, you've already lost, because you have given attention to the claim. Lets not give these stupid fucks attention.

AI is not dangerous because of the [singularity]. It is dangerous because of its power consumption, automation of war, turning everybody to unthinking, surveillance automation, burning books, the economic instability it will cause due to the bubble bursting, water consumption, destroying the internet, it gaining responsibilities it is not qualified to take and so on.

But we can see how the story of the [singularity] and similar stories helps AI companies, because now, instead of criticizing real unwellnesses, we criticize a fictional unwellness that generates discussion regardless of our positions. It is a conversational black hole.

lets get back to the real world

Many are skeptical that there is a push by ai companies to make them appear more than they are due to their weak economic footing. From my experience with the llms I am not frightened they will become self aware. I am frightened that we are wasting energy and resources while polluting the planet further for little return and it will result in messing up our economies even more.

In addition to what everyone else said, I'll tell you why I don't think that what we have currently can become some runaway superhuman intelligence:

All “AI” people are talking about are basically “just” LLM wrappers: tools built to run a LLM in a loop while giving it access to the ability to write and run code. This gets pretty powerful, since you can do a lot programmatically (including monetary transactions), but has a fundamental limit:

LLMs are trained ahead of time. And that's a very cost intensive, long process. Beyond that, the LLM has only a context window of finite length. So there are no long-term planning capabilities. While running after being trained, the LLM can only condense knowledge into a summary and then re-read it, a lossy process that might introduce mistakes in “understanding” and will make any prolonged task degrade over time.

And even if they figure out how to make the things actually self-modify and learn things, you can't give every session that ability, since that'll vastly increase the storage and performance requirements. Do sessions with that ability will be closely monitored and limited.

So I don't see any space where an evil AI could emerge, even when factoring in an essential possible future improvement to the technology.

Everytime I see an article about the dangers of AI, I'm thinking of some version of this comic every single time.

And the reason it is dangerous is generally because someone is convinced that Ai models are not as shown above.

Good job in helping with their guerilla marketing campaign.

When a corporation realises that working for government will be more profitable than working for consumers, they pivot to scaremongering, but always with the undercurrent of "and that's why you should PAY US so that you own the nukes, and not your enemies"

its already here, mass surveillance under various "camera" companies/tech companies, Mass layoffs due to AI, inability to get jobs due to AI.

AI isn't a danger to humanity, per se. Humans abandoning their duties of being responsible stewards of systems is a genuine source of danger though.

We already have algorithmic abuse where people are subjugated by welfare and debt systems with little oversight or flexibility. Some organisations even breaking the law and going against their stated missions to achieve goals and maintain order.

Social murder is the concept that society can kill a person through stress, pressure, abandonment and indifference to the individuals struggles and needs. We're seeing a lot of it today with the declining health of younger people being especially alarming.

AI can scale the two things I've mentioned beyond limits set by current bureaucratic systems, I.e. the slowness of people running a system. Systems which already kill and harm people without AI. The small erroneous tendencies of logic and algorithms in systems will be scaled up, further and faster. AI can enable algorithmic abuse and social murder on scales comparable to mass murder.

Look up “rationalists” and “zizians”

It will quickly become clear to you that the dichotomy is caused by one side being completely off their fucking rockers.

'Sensationalist hooey, cooked by superstitious yokels seeing phantoms of their own imagining.'

AI use is like drug-use. One of its effects is hopium. These people are effectively high on their own supply.

Here's an idea: don't use unsafe software where safety is needed.

How will "AI kill us all"? I can see it creating harm and getting people out of jobs. But kill us??

„Computers are still computers are still computers and will only do what a human being programs them to do, and people are stupid. Stop assigning them agency. Say that "Microsoft created an automatic hacking program which was inadequately contained." Assign the blame CORRECTLY, because you cannot blame a computer, it's doing exactly what it's programmed to do.

OpenAI is a dead slab of metal and electrons arranged to mimic a human face and if you refuse to see that then you're as brain-dead as any chunk of silicon.“

https://chaosfem.tw/@theogrin/117299179832324445

No matter how often I read this kind of stuff, I simply don't buy the 'AI is dangerous for humanity!! We gotta stop their development or at least press for sane guardrails!!!' narrative.