this post was submitted on 26 Nov 2023
1 points (100.0% liked)
LocalLLaMA
1 readers
1 users here now
Community to discuss about Llama, the family of large language models created by Meta AI.
founded 10 months ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Same experience here. I got excellent results from quantized models of Intel-Neural-7B and Mistral-7B but bad results with quantized model of Yi-34B.
I'm not sure what the point of Neural-7B is, given that it's super censored corporate safety bot. If that's what people want they might as well just use ChatGPT, which is faster and better otherwise.
Privacy and cost Also no, 7B is as fast or faster than ChatGPT depending on ChatGPT load.