overview for

How bottlenecked are LLMs by CPU clock? (Budget options to host multiple GPUs) in c/localllama@poweruser.forum

[–] _Erilaz@alien.top 1 points 9 months ago

8004 has six DDR5 channels afaik. That takes care of the memory bandwidth. The only issue would be an SP6 motherboard.

How bottlenecked are LLMs by CPU clock? (Budget options to host multiple GPUs) in c/localllama@poweruser.forum

[–] _Erilaz@alien.top 1 points 9 months ago

3090 doesn't support PCIE 5.0, only 4.0

The 4090 does, and it makes some sense to use them in x8 5.0 configuration, but only if you have a pallet of these GPUs.

Your settings are (probably) hurting your model - Why sampler settings matter in c/localllama@poweruser.forum

[–] _Erilaz@alien.top 1 points 9 months ago

Which 34B? CodeLLaMAs or Yi-34B?

Your settings are (probably) hurting your model - Why sampler settings matter in c/localllama@poweruser.forum

[–] _Erilaz@alien.top 1 points 10 months ago (2 children)

Hmm... With what models?

Your settings are (probably) hurting your model - Why sampler settings matter in c/localllama@poweruser.forum

[–] _Erilaz@alien.top 1 points 10 months ago (4 children)

What settings did you use? I didn't run into this issue, in fact, min-P sampling helps taming the chaotic nature of some models without putting them on rails.

Would automatic temperature work? in c/localllama@poweruser.forum

[–] _Erilaz@alien.top 1 points 10 months ago

Isn't DynaTemp just that?