Hi everybody, that gonna be my first post ever I think :)
I constantly looking for local models that would provide answers to questions like
"Write a story about ABC. The story should include xyz."
I currently use MxLewd-L2-Q5-GUFF as one that basically follows my prompt, answer long reply (I usually set up a new token generation count at 1024) and have no issues with situations when the main character is something else than usual (a cow for example)
Yesterday I tested 70B like Twix, Dawn, and lvlz (exl2 2. x quantization allows me to load it to vram ) and only Opus eventually reached a similar level of creativity and following prompt as MxLewd, but there were some flaws (it gave up when I should write about cow :D so I expect it's limited to human-like scenarios only)
As my system is very decent, so I'm open to many options
But with I prefer to load stuff to Vram (24vram, 128ram).
For now, I gave up with gptq and awq as those lose to 5K_M GUFF and I decided to explore the wonders of exl2 as it works really nicely.
Thanks, everyone