Machine Learning

1 readers

1 users here now

Community Rules:

Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.

founded 1 year ago

MODERATORS

communick@academy.garden

Bill Gates told a German newspaper that GPT5 wouldn't be much better than GPT4: "there are reasons to believe that we have reached a plateau" [N] (www.handelsblatt.com)

submitted 11 months ago by we_are_mammals@alien.top to c/machinelearning@academy.garden

130 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] sergeyzenchenko@alien.top 0 points 11 months ago (2 children)

Coding is not Important to make better LLM, it’s all about math

[–] ILikeCutePuppies@alien.top 1 points 11 months ago (1 children)

There was a paper which I couldn't find at the moment, which says in the early stages of the gpt, when they added code into its knowledge base, it got better at reasoning. I think that math might help in some other ways, but code can be used to solve math problems and do more than math in anycase.

[–] farmingvillein@alien.top 1 points 11 months ago (1 children)

I think OP is responding to (without commenting on correctness...)

At some point, if coding is good enough, it might be able to write its own better LLM system.

[–] sergeyzenchenko@alien.top 1 points 11 months ago

Yes, it’s all about math problems, code is just tool to express them

[–] davikrehalt@alien.top 1 points 11 months ago (1 children)

It's strange that coding is so much better than math given that they are theoretically equivalent, and the the difference is only in the distribution of things we find interesting. I guess a lot more code is repetitive/mundane compared to harder math?

[–] ILikeCutePuppies@alien.top 1 points 11 months ago

Code can do things like launch severs, build out system architectures, read in a file, write pixels to the screen, call system calls, call a calculator app or talk to a quantum computer, move a robot, etc... significantly more than math.