andromedians

joined 9 months ago
sorted by: new top controversial old
Best model-setup for CPU-only? in c/localllama@poweruser.forum
[–] andromedians@alien.top 1 points 9 months ago

Well, there is a good chance the household will have a system with the powerful Nvidias at some point…by then will have familiarized with the whole thing anyway. Should you have any guides that are very practical, they are appreciated.

Best model-setup for CPU-only? in c/localllama@poweruser.forum
[–] andromedians@alien.top 1 points 9 months ago

i now know…what i was looking for was if maybe some disallowed, more powerful versions are on there…nay…time to start exploring soon

Best model-setup for CPU-only? in c/localllama@poweruser.forum
[–] andromedians@alien.top 1 points 9 months ago (2 children)

Thanks, was preparing a bit already. Have some variations of the software- and Wizard-Vicuna13B-Uncensored.Q5_K_M, which stood out to me for some reason…lzlv (q5km) from a review in this sub, a 70b one (the best). Can alternate depending on need. All the ones from gpt4all, and 7b(+unfiltered) q4 Lora-s, which for some reason are the only ones hosted on torrent sites.

Best model-setup for CPU-only? in c/localllama@poweruser.forum
[–] andromedians@alien.top 1 points 9 months ago

All right - the wait emphasises the quality of the question then. Will have a go - thanks!

Best model-setup for CPU-only? in c/localllama@poweruser.forum
[–] andromedians@alien.top 1 points 9 months ago

The Chinese variations to be more specific, made to run all-core turbo. Have a E5-2696 V3 and a dual E5-2687 V4. Plus twelve E5-2697 V3 for later, an experimental project. They should all be great value - as good as it gets.

1
Best model-setup for CPU-only? (alien.top)
 

Have got some server hardware: One i use for games @ 18c36t 3,2Ghz 128Gb RAM (GTX970 so GPU processing is a no-go i assume) the other similar, but will have 256Gb. What’s best for these?

i’m only starting out and don’t understand the terms and measurements yet, but i am in the process and preparing the softwares to try. i would like to focus around the best options available to me.

Thanks