overview for daHaus

SlimOrca-13B in c/localllama@poweruser.forum

[–] daHaus@alien.top 1 points 11 months ago (1 children)

Is there a technical reason they begin with "You are" and not "You're" or is it just an oversight?

For H.264 and H.265 media, what bitrate and file size is as close to original quality as possible or the same as original? in c/datahoarder@selfhosted.forum

[–] daHaus@alien.top 1 points 11 months ago

18-21

Are NAS-rated drives REALLY worth it? in c/datahoarder@selfhosted.forum

[–] daHaus@alien.top 1 points 11 months ago

Assume it's all marketing unless proven otherwise

Quantizing 70b models to 4-bit, how much does performance degrade? in c/localllama@poweruser.forum

[–] daHaus@alien.top 1 points 11 months ago

This seems like something that would be difficult to predict considering how fundamental what your changing is. The method you use to quantize it and how refined it is also matters a great deal.

Is it just me or is prompt engineering basically useless with smaller models? in c/localllama@poweruser.forum

[–] daHaus@alien.top 1 points 11 months ago

Try this for a prompt and go from there: "Describe yourself in detail."

Local LLMs Unable to Sort Lists in c/localllama@poweruser.forum

[–] daHaus@alien.top 1 points 11 months ago

What's your context and batch size? Using llama.cpp 1024 B context & 1024 B batch it acted like you described.

With a context and batch size of 1536B it performed as expected. openorca/mistral 7B

ABCs of r/DataHoarder in c/datahoarder@selfhosted.forum

[–] daHaus@alien.top 1 points 11 months ago

Congratulations! You've graduated from Windows. Time to upgrade.

What is the best 7B model for reading comprehension and instruction following at the moment? in c/localllama@poweruser.forum

[–] daHaus@alien.top 1 points 11 months ago

IMO mistral-7B-orca, jazzing it up by telling it it's a subject matter expert and not to guess seems to help too

https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca