LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

What is the difference between GGUF(new format) vs GGML models ? (alien.top)

submitted 1 year ago by TalhaZubair147@alien.top to c/localllama@poweruser.forum

3 comments fedilink hide all child comments

I'm using llama models for local inference with Langchain , so i get so much hallucinations with GGML models i used both LLM and chat of ( 7B, !3 B) beacuse i have 16GB of RAM.
So Now i'm exploring new models and want to get a good model , should i try GGUF format ??
Kindly give me suggestions if someone using Local models with langchain at production level .

you are viewing a single comment's thread
view the rest of the comments

[–] __SlimeQ__@alien.top 1 points 1 year ago

ggml is totally deprecated, so much so that the make-ggml.py script in llama.cpp now makes ggufs