LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

Is m1 max macbook pro worth? (alien.top)

submitted 11 months ago by PuzzledWhereas991@alien.top to c/localllama@poweruser.forum

3 comments fedilink hide all child comments

Hi there, Im looking to buy an apple laptop and I saw a macbook pro m1 max with 64gb ram and 2TB ssd for 2400 usd Will this computer be able to run the big models at reasonable speed?

I was going to buy the simple macbook air m1 8gb ram for 700usd but I saw this and I always wanted to play with LLMs but never could.

Any advice is appreciated, thanks

you are viewing a single comment's thread
view the rest of the comments

[–] irregardless@alien.top 1 points 11 months ago

Totally feasible to run LLMs at useful speeds. I'm running a 64gb 10/32 M1 Max. With LM Studio, I typically get

3-4 T/s using q5_k_m quants of ~70B models
6-9 T/s from q5_* and q6_k quants of ~30G models
25-30 T/s from q6_k and q8 quants of 7B models
around 20 T/s from unquantized fp16 7B models

And this is my daily work and play machine, so I usually have all sorts of browser tabs and applications open simultaneously while running the models. From a fresh boot, it's cool to be able to load an entire model into memory and still be able to do "normal" work without having to use any swap space at all.