LocalLLaMA

11 readers

4 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 2 years ago

MODERATORS

communick@poweruser.forum

New Microsoft codediffusion paper suggests GPT-3.5 Turbo is only 20B, good news for open source models? (alien.top)

submitted 2 years ago by obvithrowaway34434@alien.top to c/localllama@poweruser.forum

27 comments fedilink hide all child comments

Wondering what everyone thinks in case this is true. It seems they're already beating all open source models including Llama-2 70B. Is this all due to data quality? Will Mistral be able to beat it next year?

Edit: Link to the paper -> https://arxiv.org/abs/2310.17680

https://preview.redd.it/kdk6fwr7vbxb1.png?width=605&format=png&auto=webp&s=21ac9936581d1376815d53e07e5b0adb739c3b06

you are viewing a single comment's thread
view the rest of the comments

[–] Most-Trainer-8876@alien.top 1 points 2 years ago

wtf? Really? I mean I kinda thought that too because of the way GPT3.5 compares to Falcon 180B. Even tho Falcon has more parameters still GPT3.5 works way better than it. I credited all this to the Data used to train the model. I believe that Not just more parameters but more quality data will help AI Models increase proportionally in terms of quality & performance.
Can't believe that ChatGPT is just 20B, I always thought that it's 175B Model. What about the actual 175B+ Model? Are they going to be AGI? lol.
If this is true then it means all Open Source Models are trained cheaply and is nothing compared to what OpenAI did.