LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

Goliath120B.exl2 way too emotional (alien.top)

submitted 11 months ago by Worldly-Mistake-8147@alien.top to c/localllama@poweruser.forum

6 comments fedilink hide all child comments

I'm using 4.5bpw quant courtesy u/Panchovix for RP.

Every time we speak about loneliness, lost love or something like this, it start t-to... s-speak... *sob* speak like... this.

And I can't find a way to recover the conversation from this state. The narrations are fine, only character's speech is affected. It especially prominent when using Mirostat due to lack of repetition control.

So far I tried, with little to no success:

instruct it to speak as usual,
rewriting it's reply,
temporary switching to sampler strategy with repetition penalty,

Anyone else experiences this, and how else to deal with it?

top 6 comments

sorted by: hot top controversial new old

[–] fediverser@alien.top 1 points 11 months ago

This post is an automated archive from a submission made on /r/LocalLLaMA, powered by Fediverser software running on alien.top. Responses to this submission will not be seen by the original author until they claim ownership of their alien.top account. Please consider reaching out to them let them know about this post and help them migrate to Lemmy.

Lemmy users: you are still very much encouraged to participate in the discussion. There are still many other subscribers on !localllama@poweruser.forum that can benefit from your contribution and join in the conversation.

Reddit users: you can also join the fediverse right away by getting by visiting https://portal.alien.top. If you are looking for a Reddit alternative made for and by an independent community, check out Fediverser.

[–] Silver-Chipmunk7744@alien.top 1 points 11 months ago

Have u tried lowering the Mirostat Tau? When my AI starts to be too incoherent i lower that and it seems to help.

[–] Sat0r1r1@alien.top 1 points 11 months ago (1 children)

Same, the character will become unable to hold a normal conversation by saying "..." so often.

The narrator is unaffected.

By the way, Goliath roleplay is so horny! Even normal communication often goes nsfw.

[–] brobruh211@alien.top 1 points 11 months ago (1 children)

Horniness aside, is Goliath really the best model right now for roleplaying? I'm getting a bit of fomo from not being able to run this model locally, so I would like to know if there are 70B or 34B models that hold their own against Goliath in terms of RP. I have 24GB vram so a 2.6bpw 70B (a little unstable) or a 5bpw 34B is the best I can run.

[–] Dry-Judgment4242@alien.top 1 points 11 months ago

It is, the 3bpw quant is noticably better then lzlv 70b. Goliath is an unruly horse. It will allow itself to be controlled until it doesn't a s just goes and does its own thing. But it's prose is so much better then lzlv that I'm never going back. It's the first model that doesn't speak like ChatGPT.

[–] Murky-Ladder8684@alien.top 1 points 11 months ago

I've been checking out the latest models of people tweaking goliath120b. I found this one to be the best by far with that issue and the strange spelling stuff. Might be worth giving a try to compare for yourself: https://huggingface.co/LoneStriker/Tess-XL-v1.0-4.85bpw-h6-exl2 (Lonestriker has other bpw)