durden111111

nice. From my tests it seems to be about the same as LLava v1.5 13B and Bakllava. I'm starting to suspect that the CLIP-Large model all of these multi-model LLMs are using is holding them back.

I found it to be worse than openhermes 2.5. It just gives shorter, more robotic responses

Openhermes 2.5 still feels significantly better imo

Hopefully we get GGUFs soon

Text Gen UI for general inference

llama.cpp server for multimodal

I'm wondering too. Openhermes 2.5 works fine for me on Oobabooga but it just stops outputting any tokens once it reaches 4k context despite having everything set for 8k (I'm running GGUF offloaded to gpu).

70B

eww