this post was submitted on 25 Nov 2023
1 points (100.0% liked)
Machine Learning
1 readers
1 users here now
Community Rules:
- Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
- Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
- Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
- Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I think we'll get better models by having LLMs start to filter out less quality data from the training set and also have more machine generated data, particularly in the areas like code where a AI can run billions of experiments and use successes to better train the LLM. All of this is gonna cost a lot more compute.
ie for coding LLM proposes experiment, it is run, it keeps trying until its successful and good results are fed back into the LLM training and it is penalized for bad results. Learning how to code has actually seemed to help the LLM reason better in other ways, so improving that I would expect it to help it significantly. At some point, if coding is good enough, it might be able to write its own better LLM system.
But I wonder if the degree of freedom that you have in coding is just too much for RL to work. For Chess and Go or teaching robots how to move you still have a rather finite number of degrees of freedom whereas there should be much more Combinations of code.
Maybe a kinda risc language could be used initally and expanded over time although chatgpt is already doing some amazing things with more complex languages.
Coding is not Important to make better LLM, it’s all about math
There was a paper which I couldn't find at the moment, which says in the early stages of the gpt, when they added code into its knowledge base, it got better at reasoning. I think that math might help in some other ways, but code can be used to solve math problems and do more than math in anycase.
I think OP is responding to (without commenting on correctness...)
Yes, it’s all about math problems, code is just tool to express them
It's strange that coding is so much better than math given that they are theoretically equivalent, and the the difference is only in the distribution of things we find interesting. I guess a lot more code is repetitive/mundane compared to harder math?
Code can do things like launch severs, build out system architectures, read in a file, write pixels to the screen, call system calls, call a calculator app or talk to a quantum computer, move a robot, etc... significantly more than math.