openLLM4All

joined 10 months ago
sorted by: new top controversial old
Question about the 'economics' of running a LLM locally? in c/localllama@poweruser.forum
[–] openLLM4All@alien.top 1 points 10 months ago

I've used rundpod in the past but got a bit frustrated with it when I couldn't have just a desktop to run whatever tools I wanted in the same box. I shifted to using VMs rather than runpod which has been nice switching between a text generation ui, lm studio, etc. on the same rented box.

What’s recommended hosting for open source LLMs? in c/localllama@poweruser.forum
[–] openLLM4All@alien.top 1 points 10 months ago

I noticed TheBloke was using Massed Compute to quantize models. I've been poking around and using their hardware a bit more

Free ChatGPT (not the paid version) locally? in c/localllama@poweruser.forum
[–] openLLM4All@alien.top 1 points 10 months ago

I personally like airoboros as a gpt replacement. The uncensored bits can be really fun to play around with.

I use infermatic.ai to play around with prompting against that model

Alternatives to chat.lmsys.org? in c/localllama@poweruser.forum
[–] openLLM4All@alien.top 1 points 10 months ago

I'm not sure what the limit is on Text Generation UI which is fully local.

I don't think infermatic.ai has a limit either.