this post was submitted on 01 Nov 2023
1 points (100.0% liked)

LocalLLaMA

3 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago
MODERATORS
communick@poweruser.forum
1
What is the difference between GGUF(new format) vs GGML models ? (alien.top)
submitted 1 year ago by TalhaZubair147@alien.top to c/localllama@poweruser.forum
3 comments fedilink hide all child comments
 

I'm using llama models for local inference with Langchain , so i get so much hallucinations with GGML models i used both LLM and chat of ( 7B, !3 B) beacuse i have 16GB of RAM.
So Now i'm exploring new models and want to get a good model , should i try GGUF format ??
Kindly give me suggestions if someone using Local models with langchain at production level .

you are viewing a single comment's thread
view the rest of the comments
[–] StrikePrice@alien.top 1 points 1 year ago

https://github.com/nomic-ai/gpt4all/issues/1370