LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

submitted 11 months ago by fakezeta@alien.top to c/localllama@poweruser.forum

6 comments fedilink hide all child comments

Couldn't wait for the great TheBloke to release it so I've uploaded a Q5_K_M GGUF of Intel/neural-chat-7b-v3-1.

From some preliminary test on PISA sample questions it seems at least on par with OpenHermers-2.5-Mistral-7B

top 6 comments

sorted by: hot top controversial new old

[–] mcmoose1900@alien.top 1 points 11 months ago

For anyone wondering, you can actually rent Gaudi from Intel's Dev Cloud to finetune like this:

The blog cites $10/hour for 8 HPUs.

[–] metalman123@alien.top 1 points 11 months ago

Intel has entered the game. Things are getting interesting.

If we ever get access to a mistral or yi 70b± model I think a lot of companies are going to be in trouble with their current models.

[–] fakezeta@alien.top 1 points 11 months ago

Added also to Ollama library: in case of need.

[–] AdamDhahabi@alien.top 1 points 11 months ago (1 children)

Interested to know how it scores for RAG use cases, there is a benchmark for that https://github.com/vectara/hallucination-leaderboard

Up to now, Mistral underperforms Llama2.

[–] fakezeta@alien.top 1 points 11 months ago

Currently all the finetuned version of Mistral I've tested have a high rate of hallucination: this one also seems to have this tendency.

[–] fragilesleep@alien.top 1 points 11 months ago

Thank you for your work! Is it possible to download this model if I can't run Ollama? I couldn't find a download link or a HF repo.