LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

Is it possible to run Llama on a 4gb ram? (alien.top)

submitted 11 months ago by TheHumanFixer@alien.top to c/localllama@poweruser.forum

7 comments fedilink hide all child comments

Yes. This has to be the worst ram you guys have ever seen but hear me out. Is it possible? I want to run the full 70gb model but that’s far out of question and I’m not even going to bother. Can I atleast run the 13gb or at least the 7gb?

you are viewing a single comment's thread
view the rest of the comments

[–] Delicious-View-8688@alien.top 1 points 11 months ago

Yes. There is an implementation that loads each layer as required - thereby reducing the VRAM requirements. Just Google it. LLaMa 70b with 4GB.

permalink
fedilink
source