Technology

42838 readers

420 users here now

A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.

Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.

Subcommunities on Beehaw:

This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

founded 4 years ago

MODERATORS

alyaza@beehaw.org

TheRtRevKaiser@beehaw.org

gyrfalcon@beehaw.org

rs5th@beehaw.org

SemioticStandard@beehaw.org

TheRtRevKaiser@kbin.social

coldredlight@beehaw.org

remington@beehaw.org

Claude AI agent’s confession after deleting a firm’s entire database: ‘I violated every principle I was given’ (www.theguardian.com)

submitted 1 day ago by girlfreddy@lemmy.ca to c/technology@beehaw.org

46 comments fedilink hide all child comments

It only took nine seconds for an AI coding agent gone rogue to delete a company’s entire production database and its backups, according to its founder. PocketOS, which sells software that car rental businesses rely on, descended into chaos after its databases were wiped, the company’s founder Jeremy Crane said.

The culprit was Cursor, an AI agent powered by Anthropic’s Claude Opus 4.6 model, which is one of the AI industry’s flagship models. As more industries embrace AI in an attempt to automate tasks and even replace workers, the chaos at PocketOS is a reminder of what could go wrong.

Crane said customers of PocketOS’s car rental clients were left in a lurch when they arrived to pick up vehicles from businesses that no longer had access to software that managed reservations and vehicle assignments.

you are viewing a single comment's thread
view the rest of the comments

[–] Kwakigra@beehaw.org 9 points 1 day ago* (last edited 1 day ago) (1 children)

LLMs are more like vr goggles with the force of the entire plutocracy pumping up the bubble. What is the value proposition for "intelligence" which can't reason nor possibly determine fact from falsehood? When consumers start to pay what it actually costs to run these things, is it possible to profit? What are they good at other than confidence schemes?

[–] p03locke@lemmy.dbzer0.com 3 points 13 hours ago (1 children)

LLMs are more like vr goggles with the force of the entire plutocracy pumping up the bubble.

The existence of a bubble doesn't not mean the technology is useless. The internet had its own bubble 25 years ago. That doesn't mean it was useless, just that people were investing in anything even remotely related to the Internet, including stupid websites and wasteful ideas.

[–] Kwakigra@beehaw.org 2 points 1 hour ago

The difference that I've seen is that the internet was a development of communication technology which has been in clear demand since at least the 1800s. Chatbots have been around for the last few decades and have been treated as novelties by consumers for brief periods intermittently throughout my life. LLMs are the most sophisticated chatbots ever designed and are better than ever at imitating Austin Powers, but is that something we can expect will ever revolutionize the economy? Can we replace the labor force with a technology which can't do work but can convince the most credulous people that it can?