LocalLLaMA

11 readers

4 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 2 years ago

MODERATORS

submitted 2 years ago by mcmoose1900@alien.top to c/localllama@poweruser.forum

12 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] Combinatorilliance@alien.top 1 points 2 years ago (1 children)

I believe these are TheBloke's GGUF quants if anyone's interested: https://huggingface.co/TheBloke/Nous-Capybara-34B-GGUF

[–] WolframRavenwolf@alien.top 1 points 2 years ago (1 children)

Also note this important issue that affects this and all other Yi-based models:

[–] a_beautiful_rhind@alien.top 1 points 2 years ago (1 children)

So we can just skip BOS token on all these models?

[–] ambient_temp_xeno@alien.top 1 points 2 years ago

I did the gguf-py/scripts/gguf-set-metadata.py some-yi-model.gguf tokenizer.ggml.bos_token_id 144

and it's changed the outputs a lot from yesterday.