this post was submitted on 09 Nov 2023
1 points (100.0% liked)

LocalLLaMA

3 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago
MODERATORS
communick@poweruser.forum
1
8x P40/P100 (PCIE) servers (alien.top)
submitted 1 year ago by Training_Wait_3904@alien.top to c/localllama@poweruser.forum
4 comments fedilink hide all child comments
 

Are these worth bothering with? Or money better spent on 2x3090 or A6000?

top 4 comments
sorted by: hot top controversial new old
[–] a_beautiful_rhind@alien.top 1 points 1 year ago

3090s are faster. P40s are mostly stuck with GGUF. P100s are decent for FP16 ops but you will need twice as many.

All depends on what you want to do. 8 cards are going to use a lot of electricity and make a lot of noise.

[–] IntrovertedFL@alien.top 1 points 1 year ago

Have a look at this post, might help you on making a decision - https://www.reddit.com/r/LocalLLaMA/comments/17phkwi/powerful_budget_aiworkstation_build_guide_48_gb/

[–] AutomaticDriver5882@alien.top 1 points 1 year ago

Don’t get those the token rate will be crazy slow

[–] NoWarrenty@alien.top 1 points 1 year ago

https://preview.redd.it/p39inm44cczb1.jpeg?width=4032&format=pjpg&auto=webp&s=0b0448e1c82af4ce9ebefb8f218b1b1dd32f3f58

I like my 4x3090 Frankenstein server. 70b awq model runs at 27t/s. Cost below 4k€