overview for daHaus

SlimOrca-13B in c/localllama@poweruser.forum

[–] daHaus@alien.top 1 points 2 years ago (1 children)

Is there a technical reason they begin with "You are" and not "You're" or is it just an oversight?

For H.264 and H.265 media, what bitrate and file size is as close to original quality as possible or the same as original? in c/datahoarder@selfhosted.forum

[–] daHaus@alien.top 1 points 2 years ago

18-21

Are NAS-rated drives REALLY worth it? in c/datahoarder@selfhosted.forum

[–] daHaus@alien.top 1 points 2 years ago

Assume it's all marketing unless proven otherwise

Quantizing 70b models to 4-bit, how much does performance degrade? in c/localllama@poweruser.forum

[–] daHaus@alien.top 1 points 2 years ago

This seems like something that would be difficult to predict considering how fundamental what your changing is. The method you use to quantize it and how refined it is also matters a great deal.

Is it just me or is prompt engineering basically useless with smaller models? in c/localllama@poweruser.forum

[–] daHaus@alien.top 1 points 2 years ago

Try this for a prompt and go from there: "Describe yourself in detail."

Local LLMs Unable to Sort Lists in c/localllama@poweruser.forum

[–] daHaus@alien.top 1 points 2 years ago

What's your context and batch size? Using llama.cpp 1024 B context & 1024 B batch it acted like you described.

With a context and batch size of 1536B it performed as expected. openorca/mistral 7B

ABCs of r/DataHoarder in c/datahoarder@selfhosted.forum

[–] daHaus@alien.top 1 points 2 years ago

Congratulations! You've graduated from Windows. Time to upgrade.

What is the best 7B model for reading comprehension and instruction following at the moment? in c/localllama@poweruser.forum

[–] daHaus@alien.top 1 points 2 years ago

IMO mistral-7B-orca, jazzing it up by telling it it's a subject matter expert and not to guess seems to help too

https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca