LocalLLaMA

4 readers

4 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 2 years ago

MODERATORS

communick@poweruser.forum

MBP M3 max for Local LLama? (alien.top)

submitted 2 years ago by itsnotmeyou@alien.top to c/localllama@poweruser.forum

4 comments fedilink hide all child comments

I want to buy a laptop or build a machine powerful enough that I can run these LLMs locally. I am ok with either investing in desktop or MBP though MBP was lucrative with a capability to run these models on laptop itself. Any pointers would be helpful. I tried researching but so much of info is there that I got spooked. Any initial pointers would really help. Thank you!

you are viewing a single comment's thread
view the rest of the comments

[–] FlishFlashman@alien.top 1 points 2 years ago

Apple Silicon Macs are great options for running LLMs, especially so if you want to run a large LLM on a laptop. With that said, there aren't big performance differences between the M1 Max & M3 Max, at least not for text generation, prompt processing does show generational improvements.. Maybe this will change in future versions of MacOS if there are optimizations to unlock better Metal Performance Shader performance on later GPU generations, but for right now, they are pretty similar.

Apple Silicon Macs aren't currently a great option for training/fine tuning models. There isn't a lot of software support for GPU acceleration during training on Apple Silicon.