Fuck
this post was submitted on 27 Aug 2026
389 points (97.6% liked)
Technology
87612 readers
4114 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
The core llama.cpp maintainers also work at HF and will now work for Nvidia I guess. Llama.cpp is a pretty significant part of the local LLM stack, especially since other tools like Ollama and LMStudio are just GUIs built on top.
Local LLMs have gotten to the point where they are a serious threat to Anthropic and OpenAI, and Nvidia has a lot of skin in the game. If Nvidia wanted to do some serious damage to local LLMs, they are now in a position to do so.
Iβm also imagining they may try to squeeze out support for other GPU vendors. Iβm using an AMD 7900 XTX to run Qwen 3.8 27B that I downloaded from HF to run on llama.cpp, which currently works like a dream. The 7900 XTX is the only sanely priced 24GB GPU left in 2026 (under $1k vs. $2k, $3k, $4k for Nvidia 24-32GB cards). Combined with OpenCode or Pi, a setup like this basically eliminates the need to use Anthropic or OpenAI products in the same way Jellyfin eliminates the need to use streaming services.
Iβm sure Nvidia and their buddies donβt like one thing Iβve said in this comment and may very well be plotting to put a stop to it, so the community may need to step up our game and get our eggs out of the big tech basket.
Well the good news is they can't take away from you what you already have. It being an open source project, I'm assuming if they do anything to deliberately gut AMD performance, it'll get forked.
Also
The 7900 XTX is the only sanely priced 24GB GPU left in 2026 (under $1k vs. $2k, $3k, $4k for Nvidia 24-32GB cards).
Not on sale anymore, at least not at any vendor in my country, I searched an aggregate pricing website. Amazon has a few used ones left of some models, but that's probably a 2 or 3 digit figure across SKUs. Hold on to yours with an iron grip.
What kind of tok/s are you getting with it on Qwen 3.8 27B and how's the output quality? I may consider getting one if I can find one used or import from abroad.
I got mine used, around 600 imperial credits. Look for ads that provide proof of working and benchmarks (like FurMark)
Agreed, I was referring more to future updates. Obviously weβre good with what is available now.
I canβt speak to pricing and availability outside the U.S., but it looks like the one I got went up $100:
https://www.newegg.com/asrock-radeon-rx7900xtx-24g-radeon-rx-7900-xtx-24gb-graphics-card-triple-fans/p/N82E16814930084.Β
I traded in my 3070 and my final price was in the 700s. Last I looked, used ones were going for $800 on eBay vs. $1200 for a used 3090.
I run 3.8 27B at q4 with q4 context up to 200k. Decode is generally in the 30s and pp starts in the 700s and drops to the 400s as context approaches 200k. I use mostly Sonnet 5 at work and I would rate this setup with the OpenCode desktop app as pretty comparable overall for coding at least. Letβs just say I have no reason to use any cloud models, not that I would do that voluntarily outside of being compelled to at work.
what if the LLAMA.CPP devs working at Nvidia improves CUDA support and keeps other vendor support.
It would be great but they seem to favour server computing as that has the greatest margins. I have a feeling being the biggest company on the planet at $5tn is not enough.
It was only a matter of time
Please tell me this is a rumor
It's basically a done deal
Fuck
So they are just slapping random price tags on things now. It's a database of AI questions. They have paid 13 billion dollars for a database, and everyone's acting like that's a perfectly rational sensible thing to do. No one in the financial industry has any sense anymore.
A company that's only product is something that people either don't want or actively hate has paid an eye-watering amount of money for a database to train their product on, this training will have no effect whatsoever on whether people want it.
I have this nice bridge with lots of training examples if anybody's interested, 100 trillion dollars please
not everyone has a hate boner for AI
It's an expensive toy that might become a viable product in a decade that doesn't mean it's a valuable company today. Irresponsible spending like this is exactly what led to the dot com bubble.
for a database to train their product on
Isn't hugging face a database of already trained models?
Partially. But they also host datasets.
ELI5 huggingface?
Its a big box of toys but replace the toys with downloadable AI models and the box is a website
ngl the first image on their website is pretty much all the explanation you need
hosts all the ai models and is the primary resource for downloading them for pretty much all kinds, like depthmaps, masking too, also llms, inage generators, etc.
Remember, the whole AI scheme exists to take the very concept of ownership from us. This takeover is hostile.
Why do we need a centralized hub for AI models anyway? I've never figured this out. We flock to these big "friendly" fucking companies with obviously unsustainable business models trying to own and sell things that should be shared in a decentralized, democratized mesh anyway. AI models should all be distributed magnet links, not hosted files. Why do we do this to ourselves?
The way things are going at Huggingface now, I imagine we probably will have to start building the infrastructure we need to handle AI models as distributed links sooner rather than later. Let the enshittification begin, we'll move on to a different tactic for sharing AI models while cheerfully they squeeze cash out of people and businesses too lazy to adapt. Everything working as it should, I guess.
For real, this is the perfect application for torrent. If I were building a registry that had to have sustainable cost for an open source community it would use torrents as a file transfer backend. If I were building something that I was planning to sell to a monopoly firm that has monopoly money... HTTP and cloud storage all thw way! π
load more comments
(6 replies)
They should put up a back up torrent of all the currently available models before transferring ownership. Thus opening the doors for future websites.
NVIDIA is probably trying to shutdown the LLM-at-home market (which means you don't need huge clouds of graphics cards or even an Internet connection). Sad day for people who like to control their data.
load more comments
(3 replies)
Ed Zitron is going to have an aneurysm.
Ahhhh, this is what they meant by "the circular economy"
It's more of a reverse funnel system. A pyramid scheme, if you will.
Circular like this 
We just can't have nice things can we.
I fail to understand the use of the term open source when the resource itself can be bought by oligarchs. Like organic vegetables, or clean coal.
I've downloaded a terabyte worth of models and didn't pay a cent. storage and bandwidth cost money
I just put together a budget starter PC for local inference. I guess this is my cue to set up and get downloading.
Buys what?
Buys the ability to control open source model distribution.
Oh no.
view more: next βΊ