Ruin-Capable

joined 1 year ago
sorted by: new top controversial old
Speed and energy use - RAM vs Mac Studio vs RTX cards in c/localllama@poweruser.forum
[–] Ruin-Capable@alien.top 1 points 11 months ago

Ooh... now I've got another model to play with. :D

Speed and energy use - RAM vs Mac Studio vs RTX cards in c/localllama@poweruser.forum
[–] Ruin-Capable@alien.top 1 points 11 months ago (2 children)

What model are you going to run that can accept 100GB of context?

Running full Falcon-180B under budget constraint in c/localllama@poweruser.forum
[–] Ruin-Capable@alien.top 1 points 11 months ago (4 children)

I wonder if you could get it running on two Mac Studio Ultras with 192GB of RAM each. With fewer nodes you'd reduce the communication overhead quite a bit.