LocalLLaMA

1 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago

MODERATORS

communick@poweruser.forum

M3 Max (alien.top)

submitted 10 months ago by akashocx17@alien.top to c/localllama@poweruser.forum

3 comments fedilink hide all child comments

Seems like M3 Max is best suited for Large Language Model training. With 128 gb unified memory essential let us train models on billions of parameters! Pretty interesting.

top 3 comments

sorted by: hot top controversial new old

[–] CodeGriot@alien.top 1 points 10 months ago

You probably need to wait for the Mac Studio refresh announcements for something more clearly relevant to LLM devs. Hopefully those will have 256GB or more unified memory configs, but likely something for 2024.

That said, it's handy to be able to run inference on a q8 70b model on your local dev box, so the 96GB & 128GBs are interesting for that.

[–] SomeOddCodeGuy@alien.top 1 points 10 months ago (1 children)

Get an M1 Ultra Mac Studio 128GB for $1300 less. It has 800Gb/s memory bandwidth vs the 300Gb/s on the M3 Max. Both will have the same 97GB of VRAM to play with, but you'll utilize far more of the processor on the M1 Ultra.

[–] Infinite100p@alien.top 1 points 10 months ago

300GB/s memory bandwidth is the cheaper M3 Max with 14-core CPU and 30-core GPU.

The only CPU that can have 128GB RAM is M3 Max with 16-core CPU and 40-core GPU, and that one has 400GB/s memory bandwidth).

What are your thoughts for someone who needs a dev laptop anyway. For dev a $3200 version is enough. 128Gb one would cost me $5k.

Is almost $2k extra worth it?