InterestingBasil

joined 10 months ago
sorted by: new top controversial old
Finetuned llama2 deployment with vllm in c/localllama@poweruser.forum
[โ€“] InterestingBasil@alien.top 1 points 10 months ago

This is my setup and have no issues. Quantization is not well supported by VLLM

Structured Output with Zephyr in c/localllama@poweruser.forum
[โ€“] InterestingBasil@alien.top 1 points 10 months ago

Fine tune of course