this post was submitted on 29 Nov 2023
1 points (100.0% liked)
LocalLLaMA
1 readers
1 users here now
Community to discuss about Llama, the family of large language models created by Meta AI.
founded 10 months ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Something is wrong with your environment. even P40s give more than that.
Other option is you don't get enough tokens to get proper t/s speed. What was the total inference time?