778
Generative AI Is an Engineering Disaster. A shockingly inefficient trillion-dollar project.
(www.theatlantic.com)
this post was submitted on 15 Jul 2026
778 points (97.6% liked)
Technology
87367 readers
4166 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
It has been some time since my initial comment so at the time I was mainly using LM studio. Qwen 3.6 a3b is the MOE and it does work well on my card, but the dense model that is more intelligent/capable is the Qwen 3.6 27b which doesn't fit on the card and does get offloaded, but offloading cuts the speed down to like 1/tps.
I have since found a version of the 27b model that is "quantized," for lack of a better term, differently and has to be run through TabbyAPI which gets back to 30ish tps. It can't offload so it must fit fully on the card which keeps the speed high. Might be worth a look if you're interested, the only downside is that with my 16gb card the context limit has to be kept pretty low ~40k if I remember correctly