pablines

joined 10 months ago

What kind of specs to run local llm and serve to say up to 20-50 users in c/localllama@poweruser.forum

[–] pablines@alien.top 1 points 9 months ago

Hugging face text inference can handle concurrency you just need to power with gpus

Rocket 🦝 - smol model that overcomes models much larger in size in c/localllama@poweruser.forum

[–] pablines@alien.top 1 points 10 months ago

Woooooooow!