LocalLLaMA

1 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago

MODERATORS

communick@poweruser.forum

Using Mistral Openorca to create a knowledge graph from a text document (towardsdatascience.com)

submitted 9 months ago by WaterdanceAC@alien.top to c/localllama@poweruser.forum

23 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] Some_Endian_FP17@alien.top 1 points 9 months ago (1 children)

Thanks for this. I've only worked with RAG on OpenAI models and there's a lot of prompt finetuning needed to get decent results. A KG helps define the semantic elements and relationships between document fragments and the user query for RAG.

That said, I'm still relying on the vector database to do most of the heavy lifting of filtering relevant results before feeding them into an LLM. Having an LLM clean up or summarize the user query and create a KG from the vector database's response could lead to more accurate answers.

[–] laca_komputilulo@alien.top 1 points 9 months ago

Having an LLM clean up or summarize the user query and create a KG from the vector database's response could lead to more accurate answers.

That is the promise. Of course, you still need to figure out for your app domain if doing a concept-level, chunk level, or some in-between option like CSKG is the right application.

One thing I find helpful with prompt design is to spend less attention on writing instructions, replacing them with specific examples instead. This replaces word-smithing with in-context learning samples. You build up the examples iteratively, running the same prompt through more text, fixing it and adding onto the example list.... until you reach your context budget for the system prompt.