LocalLLaMA

4 readers

4 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 2 years ago

MODERATORS

communick@poweruser.forum

Are 7b models useful? (alien.top)

submitted 2 years ago by Naiw80@alien.top to c/localllama@poweruser.forum

24 comments fedilink hide all child comments

I'm struggling to get the 7b models to do something useful, obviously I'm doing something wrong as it appears many people strive for 7b models.

But myself I can not get them to follow instructions, they keep repeating stuff and occasionally they start to converse with themselves.

Does anyone have any pointers what I'm doing wrong?

you are viewing a single comment's thread
view the rest of the comments

[–] Monkey_1505@alien.top 1 points 2 years ago

For instruct specifically, certain models do better with certain things. OpenChat, OpenHermes and Capybara seem to be the best. But they will all underperform next to a good merge/finetune of a 13B model. Depending on the type of instruction one of those will be better than the others.

For repetition this seems to fall away somewhat with very long context sizes. Because of the sliding window, it can handle these context sizes, and if you use something like llamacpp the context can be reused such that you won't have to process the whole prompt each time.

7b is generally better for creative writing, however, there are as I said, specific types of instructions they will handle well.