LocalLLaMA

11 readers

4 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 2 years ago

MODERATORS

communick@poweruser.forum

Top 3B model is a distilled llama 7B (huggingface.co)

submitted 2 years ago by Aaaaaaaaaeeeee@alien.top to c/localllama@poweruser.forum

2 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] bot-333@alien.top 1 points 2 years ago (1 children)

I see that their distilled model is much worse than StableLM 3E1T, so the finetuning improved a lot. Unfortunately they didn’t release the datasets(Would that still be considered Open Source?). Also I’m pretty sure my StableLM finetunes are better in the Open LLM Benchmarks, they just don’t allow StableLM models to be submitted.

permalink
fedilink
source
hideshow 2 child comments

[–] Amgadoz@alien.top 1 points 2 years ago

Are your finetunes full training or lora/qlora?

permalink
fedilink
source
parent