LocalLLaMA

1 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago

MODERATORS

submitted 10 months ago by decruz007@alien.top to c/localllama@poweruser.forum

15 comments fedilink hide all child comments

Use case is that I want to create a service based on Mistral 7b that will server an internal office of 8-10 users.

I’ve been looking at modal.com, and runpod. Are there any other recommendations?

you are viewing a single comment's thread
view the rest of the comments

[–] openLLM4All@alien.top 1 points 10 months ago

I noticed TheBloke was using Massed Compute to quantize models. I've been poking around and using their hardware a bit more