kitkatmafia

joined 1 year ago
sorted by: new top controversial old
What is the best multi-purpose model available right now? in c/localllama@poweruser.forum
[–] kitkatmafia@alien.top 1 points 11 months ago

Wizard-lm 13b

1
Is anyone using vLLM for inference? Are there any faster inference framework for LLama based models? (alien.top)
 

Given you have a V100 gpu at your disposal - just curious what different folks here will use for inference Llama based 7b and 13b models. Also would you use fastchat along with vLLM for conversation template?