As requested, this is the subreddit's second megathread for model discussion. This thread will now be hosted at least once a month to keep the discussion updated and help reduce identical posts.
I also saw that we hit 80,000 members recently! Thanks to every member for joining and making this happen.
Welcome to the r/LocalLLaMA Models Megathread
What models are you currently using and why? Do you use 7B, 13B, 33B, 34B, or 70B? Share any and all recommendations you have!
Examples of popular categories:
-
Assistant chatting
-
Chatting
-
Coding
-
Language-specific
-
Misc. professional use
-
Role-playing
-
Storytelling
-
Visual instruction
Have feedback or suggestions for other discussion topics? All suggestions are appreciated and can be sent to modmail.
^(P.S. LocalLLaMA is looking for someone who can manage Discord. If you have experience modding Discord servers, your help would be welcome. Send a message if interested.)
Previous Thread | New Models
Deepseek coder 34b for code
OpenHermes 2.5 for general chat
Yi-34b chat is ok too, but I am a bit underwhelmed when I use it vs Hermes. Hermes seems to be more consistent and hallucinate less.
It’s amazing that I am still using 7b when there are finally decent 34b models.
Did you notice a big difference between Deepseek coder 34B and it's 7B version? What are the system requirements for 34B? It looks to be around 70GBs in size..
I honestly haven’t tried the 6.7b version of Deepseek yet, but I’ve heard great things about it!
You can run 34b models in q4 k m quant because it’s only ~21 GB . I run it with one 3090.