this post was submitted on 22 Nov 2023
1 points (100.0% liked)

LocalLLaMA

3 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago
MODERATORS
communick@poweruser.forum
1
I’m extremely confused about system requirements. Some people are worried about ram and others about vram. I have 64gb of ram and 12gb vram. What size of model can I run? (alien.top)
submitted 11 months ago by A0sanitycomp@alien.top to c/localllama@poweruser.forum
14 comments fedilink hide all child comments
 

From what I’ve read mac somehow uses system ram and windows uses the gpu? It doesn’t make any sense to me. Any help appreciated.

you are viewing a single comment's thread
view the rest of the comments
[–] ModeradorDoFariaLima@alien.top 1 points 11 months ago

Imo running any AI on anything other than VRAM makes it so slow it's unusable. So, you can play around with 13b quantized models.