this post was submitted on 28 Nov 2023
1 points (100.0% liked)
LocalLLaMA
1 readers
1 users here now
Community to discuss about Llama, the family of large language models created by Meta AI.
founded 10 months ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I just got a P100 for like $150, going to test it out and see how it does with its FP16 vs P40 for SD and exllama overflow.
4060 is faster but its multiple times as expensive. For your sole GPU you really need 24gb+. The AMD are becoming somewhat competitive but still have some hassle and slowness.
CPU is going to give you 3t/s, its not really anywhere near, even with the best procs. Sure get it for other things in the system, but don't expect it to help much with ML. I guess newer will get you faster ram but it's not enough.