this post was submitted on 14 Nov 2023
1 points (100.0% liked)

LocalLLaMA

1 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago
MODERATORS
 

TL;DR:

Hey everyone, I am following up on my post from last week, and releasing a 70B version of DreamGen Opus V0.

What is collaborative / steerable story writing?

You can find more details about prompting the model in the official prompting guide. Make sure to checkout the examples, like “You can stop time. What will you do?”, “Natasha Romanoff and me…” or “The mischievous Coven”.

In summary: It allows you to describe a story, generate / write a part of it, and then further steer the story with inline instructions, like:


(Initial description of the story)
(Can include character description)


(Previously generated or written part of the story)


(Inline instructions describing how the story should develop next)
(The model will steer it in the direction within the next few sentences / paragraphs)

How to do chat / role-play?

Very similar to regular story writing, but:

  • In the story description, mention that it’s written in the form of a dialog or in first person (depending on preferences).
  • Preferably include character definitions.
  • After ``, kick off the chat / role-play with 2-3 example dialog turns, using the format you want, e.g.:

See the examples tagged as "chat" from the official guide linked above.

How to try it out

The model should work great with any tool that supports the Llama 70B base model.

The model is large, therefore not everyone might be able to run it locally. To that end, you can try it for free on dreamgen.com (at least while I can support it :)).

What’s next

Last time I asked the community what I should work on next (you can still vote here). The most requested thing was better support for creating & sharing characters, both on the tooling side & model side (there’s tons of headroom when it comes to data, etc.) — stay tuned for updates in the coming weeks.

Want to join the fun and influence the direction of DreamGen? Join our community server.

top 6 comments
sorted by: hot top controversial new old
[–] Danny_Davitoe@alien.top 1 points 10 months ago

Is there an example of a prompt and the model's response?

[–] uti24@alien.top 1 points 10 months ago

Uncensored model for story telling

No, somehow I got very different result.

It refuses to write smut ''I am AI created to write positive stories blah blah" (it's not literally what it said), and when I entered "Start reply with: Sure thing" it replied something like: "I'll try to write a story in a decent way." and then proceed to writing a story without a smut, like it was not a part of prompt.

Existing lzlv-70b is less censored in this regard and also writes a better stories, for my taste.

[–] panchovix@alien.top 1 points 10 months ago

Great work!

Will upload some exl2 quants in about 4-5 hours here https://huggingface.co/Panchovix/opus-v0-70b-exl2 (thinking for now about 2.5, 4.65 and 6bpw (I use the latter))

Also, uploaded a safetensors conversion here, if you don't mind https://huggingface.co/Panchovix/opus-v0-70b-safetensors

If you don't want the safetensors up, I can remove it.

[–] Sabin_Stargem@alien.top 1 points 10 months ago

Assuming that DreamGen is a commercial product, it might be good to invest in Kalomaze. They invented the Min P method for better token sampling, and are developing a Dynamic Temperature that might replace Mirostat. Giving them a grant to develop such techniques could help improve the output of Opus and other models, with DreamGen getting the first opportunity to offer those improvements to customers.

As a user of models, I personally have difficulty figuring out the ideal preset for a model. Having recommendations for the model and app would be good. EG: "Opus Storyteller", "Opus Roleplay", and "Opus Assistant".


It would also be cool to have a "Preset Arena" feature, where the AI is automated to create a number of responses to a prompt, each with different parameters. The user then picks out the best responses, which in turn get a deviation. As time goes on, the user can favorite the outputs that have the best results.

For example, Preset Arena results tagged "KoboldCPP", "Yi-34b-200k 120 Days of Opus v7.2", "NSFW", "1024 Token Generation", and "Story Mode", could be shared with DreamGen and other users on a scoreboard. In effect, this would allow the user and DreamGen to co-develop presets that are most suitable for a given task, client, and model.


Aside from all of that, I would like to try a Yi-34b-200k Opus.

[–] AmazinglyObliviouse@alien.top 1 points 10 months ago

Any hopes for a 70B 2.55BPW EXL2 conversion?

[–] ambient_temp_xeno@alien.top 1 points 10 months ago

It seems quite interesting and useful for story writing. The way it works positively invites you to jump in and steer it, if not during generation then afterwards.