this post was submitted on 17 Apr 2026
57 points (98.3% liked)
Privacy
48080 readers
527 users here now
A place to discuss privacy and freedom in the digital world.
Privacy has become a very important issue in modern society, with companies and governments constantly abusing their power, more and more people are waking up to the importance of digital privacy.
In this community everyone is welcome to post links and discuss topics related to privacy.
Some Rules
- Posting a link to a website containing tracking isn't great, if contents of the website are behind a paywall maybe copy them into the post
- Don't promote proprietary software
- Try to keep things on topic
- If you have a question, please try searching for previous discussions, maybe it has already been answered
- Reposts are fine, but should have at least a couple of weeks in between so that the post can reach a new audience
- Be nice :)
Related communities
much thanks to @gary_host_laptop for the logo design :)
founded 6 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
IMHO LLM usage isn't coherent with independence. That being said I wrote quite a bit on self-hosting LLMs. There are quite a few tools available, like ollama itself relying on llama.cpp that can both work locally and provide an API compatible replacement to cloud services. As you suggested though typically at home one doesn't have the hardware, GPUs with 100+GB of VRAM, to run the state of the art. There is a middle ground though between full cloud, API key, closed source vs open source at home on low-end hardware : running STOA open models on cloud. It can be done on any cloud but it's much easier to start with dedicated hardware and tooling, for that HuggingFace is great but there are multiples.
TL;DR: closed cloud -> models on clouds -> self-hosted provide a better path to independence, including training.
Yeah, me too :)
https://bobbyllm.github.io/llama-conductor/
https://codeberg.org/BobbyLLM/llama-conductor
I'm thinking about coding a >>cloud side car at the moment, with the exact feature you mentioned...but...that's scope creep for what I have in mind.
Irrespective of all that, I agree: an open cloud co-op could be a good way to have SOTA (or near SOTA - GLM 5.1 is about as close as we have right now) access for when needed.
(Not teaching you to suck eggs, so this comment is for the lay-reader):
For coding, you can do some interesting stuff where the cloud model is the "general" and the locally hosted LLM is the "soldier" that does the grunt work. We have some pretty decent, consumer-level-hardware runnable "soldiers" now (I still like Qwen 3 coder)...they just don't quite have the brains to see the full/big picture for coding.