diamond_jackie07

joined 1 year ago

[Project] LLM inference with vLLM and AMD: Achieving LLM inference parity with Nvidia in c/machinelearning@academy.garden

[–] diamond_jackie07@alien.top 1 points 1 year ago

Will report back on 13b performance ASAP

permalink
fedilink
source
context

[Project] LLM inference with vLLM and AMD: Achieving LLM inference parity with Nvidia in c/machinelearning@academy.garden

[–] diamond_jackie07@alien.top 1 points 1 year ago (1 children)

I tried on this config - Ryzen 9 7950x MI210. I got this result Throughput: 129 requests/min, 1028.89 tokens/s on llama2-7b. Which is even better than the performance they cite on the post

permalink
fedilink
source