quaquaversal_

joined 2 years ago
sorted by: new top controversial old
How much more stupid is the 120B goliath Q3_K_M than the larger options? in c/localllama@poweruser.forum
[–] quaquaversal_@alien.top 1 points 2 years ago (1 children)

What's the tok/s for each of those models on that system?

Edit: also, if you don't mind my asking, how much context are you able to use before inference degrades?