LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

Something wrong with older AMD high VRAM cards? (alien.top)

submitted 11 months ago by GermanK20@alien.top to c/localllama@poweruser.forum

1 comments fedilink hide all child comments

I've read a lot of hardware discussions here, but never a mention of cards like AMD 102D1631200 Radeon Instinct MI60 32GB HBM2 . Their VRAM and bandwidth are certainly fine, the price more than fine (used only), is there a devil in the details?

top 1 comments

sorted by: hot top controversial new old

[–] ccbadd@alien.top 1 points 11 months ago

I'd just be worried they will drop support for them in ROCm 6.0. They dropped the MI-50's already. Technically you can still run them and the other MI25 but ROCm is kernel specific so before long you might have to maintain a system with an old kernel to have it working. I have a pair of MI100s and while they do work fine, they are slower than NVidia 3090s when used with llama.cpp, exLLama, and Koboldcpp for some reason. It looks like with the new release of flashattention-2 the MI210 is the oldest card they support which I find very frustrating. I also have a couple W6800's and they are actually as fast or faster than the MI100s with the same software and about the same price and have built in cooling.