uti24

Maybe you need to update ooba, but I can confirm, today I loaded and played a bit with deepseek 67b gguf 4_K_M on windows, run on CPU.

Seems I am doing something wrong with this one.

I got abismal results with 4_K_M: it had silly grammatical errors and typos, it also did not stick to prompt, so I don't know.

For not biggest models are the best, so there is no best model for CPU, there is best model you are ready and willing to wait an answer.

Like Goliath-120B is great, and I am using it on i5-12400, having 0.4 tokens/second and I don't want anything less now.

SSD is at 100% reading 1~2gb/s

If your SSD swapping then model does not fit into RAM.

Use smaller quant, like 4_K_M from your own link.

That is pretty good! What quant do you use?

Have you tried Goliath-120B, how fast it runs? It might be even better that a Falcon-180B, so might be worth trying.

Sounds like you run it on CPU. If you using oobabooga you have to explicitly set how many layers you offload to GPU and by default everything runs on CPU (at least gguf models)

By the way, is there a demo exists or something?

I guess they might make Grok open source if it is not very good and just comparable to open source models and leave it for themselves if it is better.

rest are noise

But why? If you increase health restoring parameter it would matter on result.

Also, why it used logs then, don't seem to be right.

Well it gave me very weird results on my updated prompt.

Like it said result is 4 or something also using logs.

Prompt:

I have 100 hit points, in 1 minute of fighting with monsters I lose 7 hit points, but also every minute I regenerate 2 hit points. How long I can fight with monsters?

Ansver:

To determine how long you can fight with monsters, we need to consider the net loss of hit points per minute. You lose 7 hit points per minute but also regenerate 2 hit points per minute.

Net loss per minute = Loss per minute - Regeneration per minuteNet loss per minute = 7 - 2 = 5 hit points

Now, we need to find out how many minutes it will take for you to lose all 100 hit points with a net loss of 5 hit points per minute.

Let's use the formula:

Time (in minutes) = Total hit points / Net loss per minute

Time (in minutes) = 100 / 5 = 20 minutes

So, you can fight with monsters for 20 minutes before losing all your hit points.

The answer is: 20.

Actually, .. I am impressed!

But same time:

I have 100 hit points, in 1 minute of fighting with monsters I lose 7 hit points, but also every minute I regenerate 2 hit points. I also kill 1 monster every 4 minutes I kill a monster and my current hit points increase by 1. I also need to have 20% of my hit points remain, so I can return to the base. How long I can fight with monsters?

It can not solve it by a long shot. Tried multiple times. GPT-3.5 solved it flawlessly.

Oh we definably need GGUF variant of this model, I love Goliat-120B (I event think it might be better that Falcon-180B) and would love to run this model.

Is this model any good for SFW stuff?

Every uncensored llm I tried worked fine with SFW stuff.

If you are talking about story telling they might be even better that SFW models. And I also never seen NSFW/uncensored models to write NSFW stuff unless explicitly asked to do so.