Hi everyone, I'd like to share something that I've been working on for the past few days: https://huggingface.co/nsfwthrowitaway69/Venus-120b-v1.0
This model is the result of interleaving layers from three different models: Euryale-1.3-L2-70B, Nous-Hermes-Llama2-70b, and SynthIA-70B-v1.5, resulting in a model that it larger than any of the three used for the merge. I have branches on the repo for exl2 quants at 3.0 and 4.85 bpw, which will allow the model to run in 48GB or 80GB of vram, respectively.
I love using LLMs for RPs and ERPs and so my goal was to create something similar to Goliath, which is honestly the best roleplay model I've ever used. I've done some initial testing with it and so far the results seem encouraging. I'd love to get some feedback on this from the community! Going forward, my plan is to do more experiments with merging models together, possibly even going even larger than 120b parameters to see where the gains stop.
We need a benchmark specifically for nsfw content generation. Because I have a theory that I think I may try and prove: nsfw content, at least of a textual nature, can hold it's own against human authors even with a 7b model...
Rwkv for example. Its just a toy for most things. But give it a couple lines of erotica and it will spit out high quality smut until its context runs out.
My theory is that the internet is full of erotic text content, and that such content exhibits less variety between outputs than other kinds of written material. Together, this ensures that even an underpowered half assed llm will likely be capable of creating half decent porn texts assuming it is uncensored
Would love to see some sample outputs from this monstrosity (I love that somebody made this but I feel guilty consuming so much electricity to create nsfw lmao)