this post was submitted on 21 Nov 2023
1 points (100.0% liked)

LocalLLaMA

3 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago
MODERATORS
communick@poweruser.forum
1
ExLlamaV2: The Fastest Library to Run LLMs (towardsdatascience.com)
submitted 11 months ago by alchemist1e9@alien.top to c/localllama@poweruser.forum
22 comments fedilink hide all child comments
 

Is this accurate?

you are viewing a single comment's thread
view the rest of the comments
[–] JoseConseco_@alien.top 1 points 11 months ago

So how much vram would be required for 34b model or 14b model? I assume no cpu offloading right? With my 12gb vram, I guess I could only feed 14bilion parameters models, maybe even not that.