ggerganov

joined 1 year ago
sorted by: new top controversial old
Need help setting up a cost-efficient llama v2 inference API for my micro saas app in c/localllama@poweruser.forum
[–] ggerganov@alien.top 1 points 11 months ago (1 children)

I just wrote a post today about serving 7B models with `llama.cpp` from cheap AWS instances - might be useful:

https://github.com/ggerganov/llama.cpp/discussions/4225