this post was submitted on 09 Nov 2023
1 points (100.0% liked)

LocalLLaMA

1 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago
MODERATORS
communick@poweruser.forum
1
8x P40/P100 (PCIE) servers (alien.top)
submitted 10 months ago by Training_Wait_3904@alien.top to c/localllama@poweruser.forum
4 comments fedilink hide all child comments
 

Are these worth bothering with? Or money better spent on 2x3090 or A6000?

top 4 comments
sorted by: hot top controversial new old
[–] a_beautiful_rhind@alien.top 1 points 10 months ago

3090s are faster. P40s are mostly stuck with GGUF. P100s are decent for FP16 ops but you will need twice as many.

All depends on what you want to do. 8 cards are going to use a lot of electricity and make a lot of noise.

[–] IntrovertedFL@alien.top 1 points 10 months ago

Have a look at this post, might help you on making a decision - https://www.reddit.com/r/LocalLLaMA/comments/17phkwi/powerful_budget_aiworkstation_build_guide_48_gb/

[–] AutomaticDriver5882@alien.top 1 points 10 months ago

Don’t get those the token rate will be crazy slow

[–] NoWarrenty@alien.top 1 points 10 months ago

https://preview.redd.it/p39inm44cczb1.jpeg?width=4032&format=pjpg&auto=webp&s=0b0448e1c82af4ce9ebefb8f218b1b1dd32f3f58

I like my 4x3090 Frankenstein server. 70b awq model runs at 27t/s. Cost below 4k€