this post was submitted on 28 Nov 2023
1 points (100.0% liked)

LocalLLaMA

1 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago
MODERATORS
communick@poweruser.forum
1
55B Yi model merges (huggingface.co)
submitted 9 months ago by Aaaaaaaaaeeeee@alien.top to c/localllama@poweruser.forum
14 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] candre23@alien.top 1 points 9 months ago

It's a new foundational model, so some teething pains are to be expected. Yi is heavily based on (directly copied, for the most part) llama2, but there are just enough differences in the training parameters that default llama2 settings don't get good results. KCPP has already addressed the rope scaling, and I'm sure it's only a matter of time before the other issues are hashed out.