LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

Is it possible for SLM to outperform GPT4 in any tasks? (alien.top)

submitted 11 months ago by OneConfusion3313@alien.top to c/localllama@poweruser.forum

5 comments fedilink hide all child comments

We've seen pretty amazing performance of mistral 7b when comparing with Llama 34B & Llama2 13B. I'm curious, theoretically, will it be possible to build an SLM, with 7-8B parameters, able to outperform GPT4 in all tasks? If so, what are potential difficulties / problems to solve? And when do you expect such SLM to come?

ps: sorry for the typo. This is my real question.

Is it possible for SLM to outperform GPT4 in all tasks?

you are viewing a single comment's thread
view the rest of the comments

[–] steph_pop@alien.top 1 points 11 months ago (1 children)

It's unfair to compare standalone LLMs with GPT4 which is whole engineering system we know nothing about.

People are working for improving quality of LLM and reduce their sizes for sure and you can always train a 7B to be very good at some tasks and beat a bigger model but only on this small task.
However the lower the number of parameters, the less the model can handle complex tasks, and the less it can be good at several different tasks at the same time.

Take a look to the tests made by https://www.reddit.com/r/LocalLLaMA/comments/17vcr9d/llm_comparisontest_2x_34b_yi_dolphin_nous/

[–] bortlip@alien.top 1 points 11 months ago

It's not really about fairness though, it's about knowing where things stand.

I've used GPT 4 a lot so I have a rough idea of what it can do in general, but I've almost no experience with local LLMs. That's something I've only played a little with recently after seeing the advances in the past year.

So, I don't really see it as a question that disparages local LLMs, so I don't see fairness as an issue - it's not a competition to me.