this post was submitted on 24 Nov 2023
1 points (100.0% liked)

LocalLLaMA

1 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago
MODERATORS
 

I saw a conversation about that. It has been suggested that one has to change the tokenizer to make it fully compatible with the hf ecosystem, but I don't know how to do that.

https://preview.redd.it/xmxi4xev472c1.png?width=563&format=png&auto=webp&s=9e222fb39cc7ffb59937c1218a26a47b193e9475

https://preview.redd.it/1i07b2qy472c1.png?width=563&format=png&auto=webp&s=6e5c39b52b73da1f5814ef2a886c66ab93606cc2

โ€‹

โ€‹

no comments (yet)
sorted by: hot top controversial new old
there doesn't seem to be anything here