LocalLLaMA

1 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago

MODERATORS

communick@poweruser.forum

Best bang for buck for MacBP? (alien.top)

submitted 9 months ago by palpapeen@alien.top to c/localllama@poweruser.forum

2 comments fedilink hide all child comments

I know there are a bunch of threads here for picking up your Macbook Pro, but I can't find exactly what I'm looking for.

I'm renewing my laptop (finally) from a 2019 MBP with 16GB RAM, and I want to run a lot of AI models on my new one, without destroying my wallet. Optimally, trying not to go over 3k (with a 10% discount) on MBPs.

I'm hesitating between an M3 Max with 32 GB (https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space-black-apple-m3-pro-with-12-core-cpu-and-18-core-gpu-18gb-memory-1tb) and taking a slightly older M2 Max for decently cheaper and potentially more RAM.

What do you think is better here? I'm inclined towards the M3 just because I want to take a very recent model so that I can keep it for at least 3 years.

I found https://github.com/ggerganov/llama.cpp/discussions/4167#user-content-fn-3-99cf8a0cea6a624f72b056d3ccbe1b51 which shows that M3 Max and M2 Max with similar specs (GPU cores) perform similarly, which seems to indicate I should rather take an M2 Max. But what would be cons of doing this?

you are viewing a single comment's thread
view the rest of the comments

[–] watkykjynaaier@alien.top 1 points 9 months ago (1 children)

I'm on M1 Max with 32gb, with GGUF in LM Studio you can run the 34b Yi finetunes well, but that's as high as you can go for now. The 3 bit 70b quants will technically run but not in any useful way. As others have noted, RAM is the make or break factor here. Get as much as you can, the processor generation is much less important.

[–] palpapeen@alien.top 1 points 9 months ago

Thanks! I may go for an M1 max with 64gb then. It seems very promising