LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

Inferencing with AND X3D Processors (alien.top)

submitted 11 months ago by ccbadd@alien.top to c/localllama@poweruser.forum

6 comments fedilink hide all child comments

With the proof of concept done and users able to get over 180gb/s on a PC with AMD's 3d vcache, it sure would be nice if we could figure a way to use that bandwidth for CPU based inferencing. I think it only worked on Windows but if that is the case we should be able to come up with a way to do it under Linux too.

you are viewing a single comment's thread
view the rest of the comments

[–] FlishFlashman@alien.top 1 points 11 months ago (1 children)

180GB/s isn't really all that fast.

[–] ccbadd@alien.top 1 points 11 months ago

Maybe, but it's a lot faster than what we can do right now and its only the start.