LocalLLaMA

11 readers

4 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 2 years ago

MODERATORS

submitted 2 years ago by Naiw80@alien.top to c/localllama@poweruser.forum

24 comments fedilink hide all child comments

I'm struggling to get the 7b models to do something useful, obviously I'm doing something wrong as it appears many people strive for 7b models.

But myself I can not get them to follow instructions, they keep repeating stuff and occasionally they start to converse with themselves.

Does anyone have any pointers what I'm doing wrong?

you are viewing a single comment's thread
view the rest of the comments

[–] DarthNebo@alien.top 1 points 2 years ago (1 children)

Try to use the instruct models like Mistral. Ensure your template is the correct one a well.

[–] laterral@alien.top 1 points 2 years ago (1 children)

How do you find the right template?

[–] DarthNebo@alien.top 1 points 2 years ago

It should be model page on HuggingFace, they also have a explicit template module which you can import automatically when interacting using model-id.

Llama ones are forgiving for not using structure but the mistral-instruct is very bad if structure is not maintained