overview for asdfgbvcxz3355

1

Anyone know what the speed difference between 2x 4090 vs 2x 3090 would be for larger LLMs (70B-120B) (alien.top)

submitted 9 months ago by asdfgbvcxz3355@alien.top to c/localllama@poweruser.forum

2 comments fedilink

Probably stupid of me but I just bought a second 4090 for larger LLMs and was wondering if that was any faster than just having bought two 3090s

1

Does a 7900xtx work for LLM? (alien.top)

submitted 9 months ago by asdfgbvcxz3355@alien.top to c/localllama@poweruser.forum

0 comments fedilink

Im trying to run 70b Llms, I already have a 4090 and a friend of mine can sell me his 7900xtx for $500. Would an amd card work or should I try and get a nvidia card?

Buying a p40 for 70b-120b in c/localllama@poweruser.forum

[–] asdfgbvcxz3355@alien.top 1 points 10 months ago (6 children)

Also are there better alternatives to the p40 that don't cost a crazy amount?

1

Buying a p40 for 70b-120b (alien.top)

submitted 10 months ago by asdfgbvcxz3355@alien.top to c/localllama@poweruser.forum

10 comments fedilink

My current specs are 7950x cpu, 96gb 6000 mhz ram and a rtx 4090. Would a buying a p40 make bigger models run noticbly faster? If it does is there anything I should know about buying p40's? Like do they take normal connectors or anything like that. I'm pretty sure I'll have to buy a cooler for it but idk what else.