this post was submitted on 12 May 2026
197 points (93.8% liked)
Technology
84930 readers
4107 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
They can also make you smarter if you use them right. Key is to use local models and not giving the techbros any money.
That's on my to-do list. I'm currently reworking my entire build because I realized I had enough last generation parts to build a media server. Once I have windows set up to only run on VM and get my stuff moved and backed up I'm going to install an LLM
I recommend Qwen3.6, either the 27B dense or the 35B MoE model. Both outstanding for local models.
What hardware are you using?
I am using qwen3.5 9b. And it is barely working.
I have a Radeon RX 7800 XT.
Qwen 3.5-9b is blazingly fast on it. However while it’s its impressive for its size, it has its limitations. Complex tasks with several steps are too much for it.
So now I run the 3.6-35B model with llama.cpp It’s too big for my VRAM so I had to split it: everything that doesn’t fit on the graphics’s card runs in the normal RAM. That slows everything down, but with the right flags I get a bit over 20 tokens/s.
If you have problems with speed and you’re using ollama I would replace it with something faster like llama.cpp.
Keep telling yourself that
Sure! How much experience do you have with LLMs?