LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

Yi-23B-Llama: Distil version of Yi-34B-Llama (huggingface.co)

submitted 11 months ago by Covid-Plannedemic_@alien.top to c/localllama@poweruser.forum

17 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] watkykjynaaier@alien.top 1 points 11 months ago (1 children)

I've completely fixed gibberish output on Yi-based and other models by setting the RoPE Frequency Scale to a number less than one, which seems to be the default. I have no idea why that works, but it does.

What I find even more strange is the models often keep working after setting the frequency scale back to 1.

permalink
fedilink
source
parent
hideshow 2 child comments

[–] Aaaaaaaaaeeeee@alien.top 1 points 11 months ago

What value specifically worked?

permalink
fedilink
source
parent