LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

New APU’s close to Gpu processing, but with unlimited memory? (alien.top)

submitted 11 months ago by bkm_s@alien.top to c/localllama@poweruser.forum

15 comments fedilink hide all child comments

If cpu processing is slow af, and gpu takes $$$$ to get enough memory for larger models; I am wondering if an APU could deliver some of that gpu speed, but using cheaper ram to get the larger models in memory; with 128gb of ram, that’s the equivalent of 6x 30/4090s, without allowing for overhead at least!

Wondering if anyone has got any current apu benchmarks vs cpu/gpu? Do you know if the GPU side of APU architecture can be used to get an increase over traditional CPU results?

I’ve been seeing a lot of claims that the ryzen 8000 series is going to be competing with low end Gpus, some people think all the way up to 3060.

If it’s possible to do, it might be the new best way to get large models working for cheap?

you are viewing a single comment's thread
view the rest of the comments

[–] AnomalyNexus@alien.top 1 points 11 months ago (2 children)

It’s vaguely how the Mac’s work.

The current APUs are still quite slow but maybe it’ll change. Also in most cases you need to designate memory as gpu specific. So not quite shared

[–] RayIsLazy@alien.top 1 points 11 months ago (1 children)

The new oryon cpu from Qualcomm looks pretty good, pretty much better than m2 but for windows.

[–] FlishFlashman@alien.top 1 points 11 months ago

A chip that won't be available for ~6 months will be better than a chip that came out a year ago? Amazing ;)