LocalLLaMA

1 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago

MODERATORS

communick@poweruser.forum

Is there any work being done on LLMs trained on a subset of knowledge? (alien.top)

submitted 10 months ago by fy20@alien.top to c/localllama@poweruser.forum

3 comments fedilink hide all child comments

I see there is progress being made on smaller LLMs that have fewer parameters, but as I understand they are just trying to optimize how much information can be fit in a given parameter size. Is there work being done on LLMs that are trained on less information? For example say I want to chat with a PDF, I don't care for my LLM to speak French, be able to write Python or know that Benjamin Franklin wrote a paper on flatuence (all things RWKV v5 World 1.5B knows).

you are viewing a single comment's thread
view the rest of the comments

[–] vatsadev@alien.top 1 points 10 months ago

"I want to chat with a PDF, I don't care for my LLM to speak French, be able to write Python or know that Benjamin Franklin wrote a paper on flatuence (all things RWKV v5 World 1.5B knows)."

This is Prime RAG, bring snippets in, make the model use them. The more knowledge the model has, the better it gets for your usecase as well, as it knows more stuff.

Also, nice using rwkv v5, hows it work for you?