this post was submitted on 12 Nov 2023
1 points (100.0% liked)

LocalLLaMA

3 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago
MODERATORS
communick@poweruser.forum
1
What’s recommended hosting for open source LLMs? (alien.top)
submitted 1 year ago by decruz007@alien.top to c/localllama@poweruser.forum
15 comments fedilink hide all child comments
 

Use case is that I want to create a service based on Mistral 7b that will server an internal office of 8-10 users.

I’ve been looking at modal.com, and runpod. Are there any other recommendations?

you are viewing a single comment's thread
view the rest of the comments
[–] sshh12@alien.top 1 points 1 year ago

Huge fan of modal, have been using them for a couple serverless LLM and Diffusion models. Can be definitely on the costly side, but like that the cost directly scales based on requests and setup is trivial.

​

recent project with modal: https://github.com/sshh12/llm-chat-web-ui/tree/main/modal