this post was submitted on 04 Dec 2023
1 points (100.0% liked)

LocalLLaMA

3 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago
MODERATORS
communick@poweruser.forum
1
Goliath120B.exl2 way too emotional (alien.top)
submitted 11 months ago by Worldly-Mistake-8147@alien.top to c/localllama@poweruser.forum
6 comments fedilink hide all child comments
 

I'm using 4.5bpw quant courtesy u/Panchovix for RP.

Every time we speak about loneliness, lost love or something like this, it start t-to... s-speak... *sob* speak like... this.

And I can't find a way to recover the conversation from this state. The narrations are fine, only character's speech is affected. It especially prominent when using Mirostat due to lack of repetition control.

So far I tried, with little to no success:

  • instruct it to speak as usual,
  • rewriting it's reply,
  • temporary switching to sampler strategy with repetition penalty,

Anyone else experiences this, and how else to deal with it?

you are viewing a single comment's thread
view the rest of the comments
[–] Murky-Ladder8684@alien.top 1 points 11 months ago

I've been checking out the latest models of people tweaking goliath120b. I found this one to be the best by far with that issue and the strange spelling stuff. Might be worth giving a try to compare for yourself: https://huggingface.co/LoneStriker/Tess-XL-v1.0-4.85bpw-h6-exl2 (Lonestriker has other bpw)