this post was submitted on 10 Nov 2023
1 points (100.0% liked)
LocalLLaMA
1 readers
1 users here now
Community to discuss about Llama, the family of large language models created by Meta AI.
founded 10 months ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
It does but it's not going to have perfect recall of that knowledge. For example you can have a dataset about random product hexyhexy that is unrecognized by the base model. Let's say hexyhexy is a game and you trained qlora on a documentation and tips&tricks for that game. End result will be that now model recalls knowledge about that game correctly around 50-70% of the time. It does know something it didn't know before, but you wouldn't make your life dependent on it.