Machine Learning

1 readers

1 users here now

Community Rules:

Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.

founded 11 months ago

MODERATORS

communick@academy.garden

Bill Gates told a German newspaper that GPT5 wouldn't be much better than GPT4: "there are reasons to believe that we have reached a plateau" [N] (www.handelsblatt.com)

submitted 10 months ago by we_are_mammals@alien.top to c/machinelearning@academy.garden

130 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] venustrapsflies@alien.top 1 points 10 months ago (11 children)

It’s not a fallacy at all. It is just statistics, combined with some very useful inductive biases. The fallacy is trying to smuggle some extra magic into the description of what it is.

Actual AGI would be able to explain something that no human has understood before. We aren’t really close to that at all. Falling back on “___ may not be AGI yet, but…” is a lot like saying “rocket ships may not be FTL yet, but…”

[–] InterstitialLove@alien.top 1 points 10 months ago (8 children)

The fallacy is the part where you imply that humans have magic.

"An LLM is just doing statistics, therefore an LLM can't match human intellect unless you add pixie dust somewhere." Clearly the implication is that human intellect involves pixie dust somehow?

Or maybe, idk, humans are just the result of random evolutionary processes jamming together neurons into a configuration that happens to behave in a way that lets us build steam engines, and there's no fundamental reason that jamming together perceptrons can't accomplish the same thing?

[–] red75prime@alien.top 1 points 10 months ago (2 children)

LLMs might still lack something that the human brain has. Internal monologue, for example, that allows us to allocate more than fixed amount of compute per output token.

[–] InterstitialLove@alien.top 1 points 10 months ago (1 children)

You can just give an LLM an internal monologue. It's called a scratchpad.

I'm not sure how this applies to the broader discussion, like honestly I can't tell if we're off-topic. But once you have LLMs you can implement basically everything humans can do. The only limitations I'm aware of that aren't trivial from an engineering perspective are

current LLMs mostly aren't as smart as humans, like literally they have fewer neurons and can't model systems as complexly
humans have more complex memory, with a mix of short-term and long-term and a fluid process of moving between them
humans can learn on-the-go, this is equivalent to "online training" and is probably related to long-term memory
humans are multimodal, it's unclear to what extent this is a "limitation" vs just a pedantic nit-pick, I'll let you decide how to account for it

[–] red75prime@alien.top 1 points 10 months ago

It's called a scratchpad.

And the network still uses skills that it learned in a fixed-computation-per-token regime.

Sure, future versions will lift many existing limitations, but I was talking about current LLMs.

load more comments (5 replies)

load more comments (7 replies)