LocalLLaMA

1 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago

MODERATORS

communick@poweruser.forum

The Acer Intel A770 16GB GPU is now $250. You won't find a better new 16GB GPU for less. (alien.top)

submitted 10 months ago by fallingdowndizzyvr@alien.top to c/localllama@poweruser.forum

22 comments fedilink hide all child comments

Amazon has the Acer A770 on sale for $250. That's a lot of compute with 16GB of VRAM for $250. There is no better value. It does have it's challenges. Somethings like MLC Chat run with no fuss just like on any other card. Other things need some effort like Oob, Fastchat and BigDL. But support for it is getting better and better everyday. At this price, I'm tempted to get another. I have seen some reports of running multi-GPU setups with the A770.

It also comes with Assassins Mirage for those people that still use their GPUs to game.

https://www.amazon.com/dp/B0BHKNK84Y

you are viewing a single comment's thread
view the rest of the comments

[–] JFHermes@alien.top 1 points 10 months ago (2 children)

3090's draw 350W as per google. Arc a770 draws 225.

I guess people here normally go with a dual 3090 setup. That's 700 watt for 48gb of VRAM. This comes out to 14.58 watt per gigabyte of VRAM.

Lets assume you manage to cool properly, you could probably run 4 A770 for 64gb of VRAM which sounds pretty nice. This watt to VRAM gigabyte ratio is 14.06 which is actually better than the nvidia cards. Also noteworthy, the pins are 1x8 pin & a 1.6 pin if I'm reading correctly. So you would have to be careful what mobo you went with but I'm pretty sure that's doable.

It all comes down to driver support, but it could be a really nice rig that would you to run more complex models on the cheap. I'm not taking into account driver support which Nvidia obviously has an advantage in, this would affect the processing speeds of the A770. I would say however, the addition of cheap VRAM would probably be worth the extra processing times so long as things actually worked.

[–] CheatCodesOfLife@alien.top 1 points 10 months ago

My 3090's don't draw 350W for inference, more like ~200 tops.

I did manage to draw 350W from one by running whisper to subtitle something though.

[–] AnomalyNexus@alien.top 1 points 10 months ago

There is also the issue of pcie slots. Currently running a second card in a x4 slot and it’s noticeably slower. Getting four full speed x16 slots is going to be some pretty specialised equipment. All the crypto rigs are slow slots to my knowledge since it doesn’t matter there

It is good to see more competitive cards in this space though. Dual 770 could be very accessible