Machine Learning

1 readers

1 users here now

Community Rules:

Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.

founded 1 year ago

MODERATORS

communick@academy.garden

Bill Gates told a German newspaper that GPT5 wouldn't be much better than GPT4: "there are reasons to believe that we have reached a plateau" [N] (www.handelsblatt.com)

submitted 11 months ago by we_are_mammals@alien.top to c/machinelearning@academy.garden

130 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] red75prime@alien.top 1 points 11 months ago (1 children)

LLMs might still lack something that the human brain has. Internal monologue, for example, that allows us to allocate more than fixed amount of compute per output token.

[–] InterstitialLove@alien.top 1 points 11 months ago (1 children)

You can just give an LLM an internal monologue. It's called a scratchpad.

I'm not sure how this applies to the broader discussion, like honestly I can't tell if we're off-topic. But once you have LLMs you can implement basically everything humans can do. The only limitations I'm aware of that aren't trivial from an engineering perspective are

current LLMs mostly aren't as smart as humans, like literally they have fewer neurons and can't model systems as complexly
humans have more complex memory, with a mix of short-term and long-term and a fluid process of moving between them
humans can learn on-the-go, this is equivalent to "online training" and is probably related to long-term memory
humans are multimodal, it's unclear to what extent this is a "limitation" vs just a pedantic nit-pick, I'll let you decide how to account for it

[–] red75prime@alien.top 1 points 11 months ago

It's called a scratchpad.

And the network still uses skills that it learned in a fixed-computation-per-token regime.

Sure, future versions will lift many existing limitations, but I was talking about current LLMs.