What is everyone's experiences so far with DPO trained versions of their favorite models? Been messing around with different models and my two new favorite models are actually just the DPO versions of my previous favorite models (causalLM 14b and openhermes 2.5 7b). Links below for the models in question.
CausalLM 14B-DPO-alpha - GGUF: https://huggingface.co/tastypear/CausalLM-14B-DPO-alpha-GGUF
NeuralHermes 2.5 Mistral 7B - GGUF: https://huggingface.co/TheBloke/NeuralHermes-2.5-Mistral-7B-GGUF
The former runs at 30 t/s for me with koboldcpp-rocm on a 6900 XT, and the latter at 15 t/s, both at Q6K. I don't have a favorite between these two models, they seem to be better at different things and trade blows in all the logic + creative writing tasks I've tested them in, despite causalLM being a larger model. I'm looking forward to seeing what nousresearch/teknium and CausalLM are bringing next.
Out of topic here but is it just me or open hermes somehow speaks in russian unprovoked frequently?
Interesting. I'm using oobabooga and that never happened to me. I actually don't recall it ever outputting anything but English...
If you are using llama.cpp, you might want to give it a grammar which forces ASCII output.
High repetition penalty? One model I merged suddenly started speaking Spanish in one summarisation task lol
I get polish or something