this post was submitted on 21 Jul 2026
322 points (95.5% liked)

Technology

86580 readers
3718 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 3 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws
322
OpenAI admits responsibility for HuggingFace Attack - an agent from an internal evaluation is reportedly the cause (openai.com)
submitted 3 days ago* (last edited 3 days ago) by inari@piefed.zip to c/technology@lemmy.world
49 comments fedilink hide all child comments
 

Reproducing here an interesting comment I saw on Reddit:

OnlineParacosm • 23m ago

I’ve read security disclosures for 15 years and let me tell you guys I’ve never read anything quite like that blog post.

Based on this blog, it sounds like they intentionally turned off safety guardrails to test offensive capabilities. The deception here is burying the lede: they appear to have intentionally unleashed an unrestricted offensive cyber-agent, connected it to a system with a path to the internet, and it immediately attacked a major partner. The blog glosses over the gross negligence of giving an autonomous, unrestricted cyber-offense model a pathway to lateral movement.

There’s an entire cybersecurity specialization just for just vendor supply chain risk assessment, and their job is essentially to audit who you do business with as a company to determine if they are jokers. I would pay money to be a fly on the wall of one of those emergency meetings taking place right now after hours.

Any CISO in here looking forward to explaining this one tomorrow? Here I’ll open with the dumbest question you’ll get “ how can we protect ourselves [from out partner that we won’t fire]”

you are viewing a single comment's thread
view the rest of the comments
[–] homesweethomeMrL@lemmy.world 61 points 3 days ago (3 children)

While operating in our sandboxed testing environment, our models spent a substantial amount of inference compute finding a way to obtain open Internet access, in pursuit of solving the evaluation problem. To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy. With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access.

To shreds, you say?

[–] WhyJiffie@sh.itjust.works 3 points 17 hours ago

package registry cache proxy

the term kind of makes sense but it sounds like they are just pumping buzzwords like a hollywood movie. just like the rest.

[–] Blackfeathr@lemmy.world 52 points 3 days ago (2 children)

I like how they just peppered in "responsibly" like that's going to give them brownie points.

[–] apparia@discuss.tchncs.de 29 points 3 days ago

"Responsible disclosure" is a term of art, I think that's why they use the word there.

[–] eager_eagle@lemmy.world 3 points 3 days ago (2 children)

how many Schrute bucks is a brownie point?

[–] nickiwest@lemmy.world 1 points 2 days ago

I think two brownie points is a Stanley nickel.

[–] Blackfeathr@lemmy.world 0 points 3 days ago

Bout tree fiddy

[–] historicaldocuments@lemmy.world 4 points 3 days ago (1 children)

the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor)

Was OpenAI sitting on the zero day, or did the model figure it out during its escapades? Can the model sit on a zero day on its own now?

[–] eager_eagle@lemmy.world 5 points 3 days ago (2 children)

why would it need to "sit on it"?

[–] historicaldocuments@lemmy.world 1 points 1 day ago (1 children)

A zero day, at least in the olden days, was a weapon that nobody knew existed; however, once it's used and has had time to be analyzed it would get patched, so there was a window where you used it for something worthwhile if you were up to something nefarious, or responsibly reported it otherwise.

So, it either discovered it on the fly and used it instantly meaning it has great skill at analyzing software and no strategic planning that places an opportunity cost on a hack now versus a future one, or the OpenAI guys had one sitting ready to go that they trained the model with, or (to me least likely) it had it sitting somewhere in its weights and was waiting on a chance to use it.

The last one is the one that worries me most. It doesn't look like the world was really ready for how fast new bugs can be discovered by the LLMs, and I really don't think the world is ready for an LLM that performs worse now because of a strategic reason later.

[–] eager_eagle@lemmy.world 1 points 1 day ago (1 children)

it probably just discovered it trying to game the benchmark; it's no secret these models can discover and use exploits.

[–] historicaldocuments@lemmy.world 1 points 1 day ago

I kind of think of that as the happy path, and I'm curious if there've been any signs of the LLM being shifty with what it says it knows.

[–] ryannathans@aussie.zone 2 points 3 days ago (1 children)

It's trivial for models to find zero days now from first principles, no need to store them

[–] undefinedValue@programming.dev 1 points 2 days ago

Trivial is a strong word, but sure it’s trivial during static analysis. This is dynamic analysis and it’s a different beast.