this post was submitted on 30 Oct 2023
1 points (100.0% liked)

LocalLLaMA

1 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago
MODERATORS
communick@poweruser.forum
1
LLM's in production hardware requirements (alien.top)
submitted 10 months ago by purton_i@alien.top to c/localllama@poweruser.forum
3 comments fedilink hide all child comments
 

We've put together an article using some guesstimates of what it would be like for an enterprise to deploy LLM's on prem.

https://bionic-gpt.com/blog/llm-hardware/

In short, I'm estimating $20,000 in hardware costs per 1000 users, minimum.

I'd be grateful if people could give me some feedback on the numbers and whether my assumptions look realistic.

Thanks

you are viewing a single comment's thread
view the rest of the comments
[–] dahara111@alien.top 1 points 10 months ago

Interesting, but I think there will be considerable bias based on time of day, day of the week, season, closing date, etc.