LocalLLaMA

1 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago

MODERATORS

communick@poweruser.forum

Wich Llama can I run with a M3 pro 36GO ? (alien.top)

submitted 9 months ago by Sufficient-Seat-5067@alien.top to c/localllama@poweruser.forum

3 comments fedilink hide all child comments

Hi everyone, just received my new macbook pro with M3 pro (36GO) and I'm willing to have fun with a local Llama now that I finally have a machine able to run it !

Main question is : wich version can I run without burning my chimp ? 13b ? 70b ?

Also if you have any usefulls ressources you can suggest to get into this game feel free to share I'm a real beginner in LLM. I'm a young python dev for fun and a javascript dev for food and mainly want to have a local llama to assist me on web dev !

Thanks to guys like Alex Ziskind I'm aware it's not really reliable for coding assist for now but I'm really curious about what it can do, if you have better tool I could set up on my machine to assist me feel free to share and I'm really willing to devour some ressources about this subject.

Thank you all for your help and advises !

you are viewing a single comment's thread
view the rest of the comments

[–] ThisGonBHard@alien.top 1 points 9 months ago (1 children)

I dont know if Exllama 2 supports Mac, but if it does, 70B.

[–] FlishFlashman@alien.top 1 points 9 months ago

It doesn't