this post was submitted on 24 Nov 2023
1 points (100.0% liked)
LocalLLaMA
1 readers
1 users here now
Community to discuss about Llama, the family of large language models created by Meta AI.
founded 10 months ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I've noticed this extensively when running locally on my 8gb rx580. And the issue is pretty bad.. I've run exactly the models you stated.
But when I run on (big) cloud GPU on vast.ai (eg on rtx 3090 or A6000) the problem vanishes..
vast.ai is pretty cheap ($10 deposit)you can experiment on there and see.