this post was submitted on 09 Nov 2023
1 points (100.0% liked)
Machine Learning
1 readers
1 users here now
Community Rules:
- Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
- Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
- Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
- Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Well, you sort of answered the matter yourself - the fact that prompting works in some cases means you don't strictly need weight updates for new skills to be learned. It doesn't mean prompting is an end-all solution, but for DeepMind, this seems enough to consider LLMs "emerging AGI".
Most people entering in the field now (in the literal sense, aka academia, not some random r/singularity ramblers) disregard current LLM capabilities, but their current level of reasoning was deemed almost a fantasy 5 years ago.
and current LLMs are pretty great for automating simple, easily defined tasks that would drive a human insane (labelling datasets etc). i'm really optomistic about their use in online moderation in the short term, lots of horror stories of facebook employees having mental breakdowns
When I mentioned prompt engineering, I more so meant that people where explaining what to do in a if/else manner to get the LLM to play tiktaktoe (not chain of thoughts or any of those techniques).
In my opinion, learning is both 1) acquiring new skills, and 2) improving upon those skills with repetition. I think it’s very debatable if an LLM could learn something truly novel (or even something like an existing game with some new rules, I.e., chess but with the game pieces in different positions) with in context learning. Secondly, no matter how much you play tiktaktoe with an LLM, it will never improve at the game.
This is just my two cents on why I don’t believe LLMs to fit the criteria of “emerging AGI” that the researchers laid out. Imo I think that to fit that criteria they would need to implement some type of online learning but I definitely could be wrong.