overview for Arkonias

used 3090 vs 4060ti on 15+ yrs old rig help in c/localllama@poweruser.forum

[–] Arkonias@alien.top 1 points 10 months ago

3090 on old hardware sounds like bottleneck hell.

Use the black friday sales to upgrade your Motherboard, Ram, CPU, and PSU.

What is the best 7B model for reading comprehension and instruction following at the moment? in c/localllama@poweruser.forum

[–] Arkonias@alien.top 1 points 10 months ago

OpenHermes 2.5 Mistral has been quite good for me. It follows the CYOA prompt that I use well.

Running LLM on my personal pc in c/localllama@poweruser.forum

[–] Arkonias@alien.top 1 points 10 months ago

Save the $$$ for a few months and go and buy a used 3090 or two. It'll be worth it in the long run, and save any headaches of trying to frakenstein a bunch of 8 GB cards together.

codellama gives really bad results, am I doing something wrong? in c/localllama@poweruser.forum

[–] Arkonias@alien.top 1 points 10 months ago

Make sure you are using the Codellama Instruct preset. Also, work on your prompting:

"You are tasked to write a python script that takes an email address...."

LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B in c/localllama@poweruser.forum

[–] Arkonias@alien.top 1 points 10 months ago

Censorship is bad. AI is for porn.