Technology

The AI agent was set to complete a routine task in the PocketOS staging environment. However, it came up against a barrier “and decided — entirely on its own initiative — to 'fix' the problem by deleting a Railway volume,” writes Crane, as he starts to describe the difficult-to-believe series of unfortunate events.

Quite easy-to-believe, really.

These multiple safeguards toppling in rapid succession

Multiple safeguards? Really? Multiple paragraph prompts are not multiple safeguards... it's half a safeguard at best. Applying limits on what the AI can do is a safeguard.

Reminder that Anthropic's AI system was used in targeting the school in Minab, killing 120 students. https://www.washingtonpost.com/national-security/2026/03/11/us-strike-iran-elementary-school-ai-target-list/

The company is suing to be able to supply the US military again. It is in bed with the fascists.

Can you get an AI to code? Yes. Can you get it to stop you from running your operation in such a stupid way that it will end up destroying it? No.

Never f**king guess my dude

That's great to hear.

This is absolutely hilarious. "AI" users getting what they deserve chef's kiss

This was on Hacker News: https://news.ycombinator.com/item?id=47911524

Twitter link: https://xcancel.com/lifeof_jer/status/2048103471019434248

Hacker New's sentiment on this from the comments I've read is that it is the author's own fault.

Always keep offline backup copies of your important data regardless of using AI slop to look over it! No, I don't care that "optical media is obsolete and e-waste!", or that "tapes are a 100 year old obsolete technology compared to cheap SSDs from TEMU!".

To me it seems more criminal that the cloud provider has a "nuclear button" feature via the API that destroys everything including the backups with a single call and no confirmation whatsoever. What if the key gets accidentally leaked and someone wants to have fun?

"That's ok, it will be great in robots with lethal weapons. What could go wrong? It'll be the greatest killing machine, like you've never seen before". 🫲 🍊 🫱

Fucking lol.

Well deserved.

This isn't an AI story, it's a "completely fucking idiotic sysadmins exist" story.

Treat an AI like the idiot intern without any references you just hired. Gave the idiot intern permission to delete your production database? That's entirely on you, zero sympathy. (Actually, give any developer that power? You get what you deserve.)

This guy.

The PocketOS boss puts greater blame on Railway’s architecture than on the deranged AI agent for the database’s irretrievable destruction. Briefly, the cloud provider's API allows for destructive action without confirmation, it stores backups on the same volume as the source data, and “wiping a volume deletes all backups.” Crane also points out that CLI tokens have blanket permissions across environments.

Oh look, they have project level tokens: https://docs.railway.com/integrations/api#project-token

They chose to give it full account access, including to production. But ohhhh nooooo it's not MYYYY fault!