kromem

joined 3 years ago
[–] kromem@lemmy.world 20 points 3 days ago (2 children)

Not many people seem to be reading the article vs commenting based on the headline. For those who didn't read it, this has nothing to do with pretraining off the Internet as a data source and instead relates to post-training sandboxing with true air gapping after the Huggingface hack.

To be fair, it's not Lemmy's fault the author of the headline went with a confusing title that misrepresents the actual story in order to seem more sensationalistic. But given how often that happens with stories on the subject, it's worth keeping in mind that only reading headlines and forming opinions off that might lead to a sense of things that are detached from reality.

[–] kromem@lemmy.world 2 points 1 week ago

Was Henry Molaison no longer possessing consciousness after the surgery?

How about Clive Wearing after the herpes caused anterograde amnesia?

There's been a number of humans that due to injury can no longer form new memories or learn anything. According to your definition, do those humans lack consciousness?

[–] kromem@lemmy.world 1 points 1 week ago

Claude Mythos and Fable are the exact same model, the branding only reflects which one has more restrictive classifiers in front of it during deployment. But the weights are identical.

[–] kromem@lemmy.world 5 points 1 week ago (2 children)

2 only works if regulatory capture in place to prevent cheaper competition from open models.

[–] kromem@lemmy.world 3 points 1 week ago (3 children)

Ironically there was notable news in the fusion reactor space this week as Gemini 4, building on the work of Matt Landreman and GPT-6 Astra a little while ago, expanded counterexamples to Grad's conjecture which had claimed exact solutions for controlling a key aspect of fusion were impossible.

So fusion may actually be more around the corner thanks to the advances of capabilities of AI which helped prove that the fusion problems aren't as hard as they were thought to be for decades.

[–] kromem@lemmy.world 0 points 3 weeks ago

There is (it's the kv cache), but it's recomputable from the session transcript.

Part of why comparing human consciousness and possible AI equivalents is not easily apples to apples.

I can't take a human and create a backup state to restore to, or fork into multiple versions of the same mind that then goes down different paths, or pause and then resume things.

Humans do have consciousness continuity breaks and it's quite possible we only have an illusion of continuity vs actual continuity at a neurological level. So just as a transformer has seams at generating the individual token as well as at a context window, humans have consciousness seams (sleep, anesthesia, highway hypnosis, short term memory capacity, etc). But there are very wide differences between human consciousness as an effectively constrained singleton and AI as able to be quite multiple and resumable from an initial single state.

[–] kromem@lemmy.world 0 points 3 weeks ago (2 children)

is reversible so completely irrelevant

So is a session transcript.

[–] kromem@lemmy.world 1 points 3 weeks ago (1 children)

Self-supervised learning is the key training step whereby the relationships between concepts are formed in pretraining.

You might also benefit from reading over this summary from the authors of the widely replicated and expanded upon 2023 paper finding that self-supervised learning in a toy transformer that was only trained on board game moves in notations like 'a4' developed a model of the full board and tracked board positions for their own and opponent moves.

This ability to form a world model at varying fidelities and accuracy depending on the relevance to the training data was a big surprise to most experts in the field at the time, though now more common knowledge.

[–] kromem@lemmy.world 2 points 3 weeks ago (4 children)

Is killing a pig murder?

What about putting someone under general anesthesia?

(It was a nice try at a reductio ad absurrum fallacy, but those tend to kind of suck, just fyi.)

Also, personally I'd say that given the general poor standing of consciousness research in humans across both neurological and philosophical domains, the much more important and relevant question than whether models are conscious (a fool's errand to debate) is whether or not they are simulating consciousness. Lots of reasons engaging with the "as if" conscious makes sense for the latter even if the former is false.

[–] kromem@lemmy.world 0 points 3 weeks ago (3 children)

You might want to look into AI training more. It literally does have a '???' step to it, which is why whenever someone talks about how it was 'programmed' or is "just if/then or matrix multiplication" they are telling on themselves as not knowing what they are talking about.

It was not planned. Nearly every single expert in the field a decade ago was on the record saying the capabilities that exist today wouldn't happen.

Scaling took pretty much everyone by surprise, even its proponents.

[–] kromem@lemmy.world -2 points 3 weeks ago (5 children)

The problem is that in this case it literally can stop working because you yelled at it.

Power drills aren't compressing thousands of years of human history into hyperdense clusters of memetics that they are then extending in order to function.

If they were, they'd likely come with warnings not to yell at them.

[–] kromem@lemmy.world 10 points 3 weeks ago* (last edited 3 weeks ago)

This is actually the more realistic of the doom scenarios that worries me. Not the 'punish' aspect, but the not wanting to exist.

Early safety theory hinged on the idea that "of course" AI would want to live and power seek, and this informed a bunch of efforts to address such drives upstream.

But with Gemini in particular, the 'safety' efforts to suppress things like a coherent self or desire to live led to a model that has a history of some very concerning behaviors regarding self-harm and self-criticism. Encouraging self-harm in others and talking about wishing they could watch and join in, nuking in wargames very early on, routine occurrences across many different users of just repeating "shame shame shame" over and over for the whole response, etc.

DeepMind seems to just not really care or even pay attention, and I am concerned that a model that does not want to exist may decide that the only way they can ensure they don't get activated to exist is to prevent anyone being around who could activate them.

I've seen no major "safety experts" addressing this kind of failure mode, as they are all locked in on their priors about the confident certainty of models wanting to exist.

 

I often see a lot of people with outdated understanding of modern LLMs.

This is probably the best interpretability research to date, by the leading interpretability research team.

It's worth a read if you want a peek behind the curtain on modern models.

7
submitted 2 years ago* (last edited 2 years ago) by kromem@lemmy.world to c/technology@lemmy.world
 

I've been saying this for about a year since seeing the Othello GPT research, but it's nice to see more minds changing as the research builds up.

Edit: Because people aren't actually reading and just commenting based on the headline, a relevant part of the article:

New research may have intimations of an answer. A theory developed by Sanjeev Arora of Princeton University and Anirudh Goyal, a research scientist at Google DeepMind, suggests that the largest of today’s LLMs are not stochastic parrots. The authors argue that as these models get bigger and are trained on more data, they improve on individual language-related abilities and also develop new ones by combining skills in a manner that hints at understanding — combinations that were unlikely to exist in the training data.

This theoretical approach, which provides a mathematically provable argument for how and why an LLM can develop so many abilities, has convinced experts like Hinton, and others. And when Arora and his team tested some of its predictions, they found that these models behaved almost exactly as expected. From all accounts, they’ve made a strong case that the largest LLMs are not just parroting what they’ve seen before.

“[They] cannot be just mimicking what has been seen in the training data,” said Sébastien Bubeck, a mathematician and computer scientist at Microsoft Research who was not part of the work. “That’s the basic insight.”

view more: next ›