this post was submitted on 24 Jun 2026
139 points (81.2% liked)

Selfhosted

60366 readers
786 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

  1. Be civil.

  2. No spam.

  3. Posts are to be related to self-hosting.

  4. Don't duplicate the full text of your blog or readme if you're providing a link.

  5. Submission headline should match the article title.

  6. No trolling.

  7. Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
ayyy@sh.itjust.works
curbstickle@anarchist.nexus
curbstickle_lw@lemmy.world
139
Do you host your own AI? (aussie.zone)
submitted 1 week ago by SuspiciousCarrot78@aussie.zone to c/selfhosted@lemmy.world
205 comments fedilink hide all child comments
 

Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

you are viewing a single comment's thread
view the rest of the comments
[โ€“] e0qdk@reddthat.com 3 points 1 week ago (1 children)

If you just pulled the default version of qwen3.5 from ollama's repo you downloaded a mediocre one that only uses ~6GB.

Check ollama show qwen3.5 and see if you get something like this in the result:

  Model
    architecture        qwen35    
    parameters          9.7B      
    context length      262144    
    embedding length    4096      
    quantization        Q4_K_M

This is the default version I got when I first tried using ollama without any experience. It worked, but it's a heavily quantized, lower parameter version of the model -- i.e. it's pretty dumb -- compared to what you can actually run on your hardware.

[โ€“] Steve@startrek.website 2 points 1 week ago

I will check it later. I loaded whichever one cluade suggested lol