Community to discuss about Llama, the family of large language models created by Meta AI.
If you are interested in knowledge graphs, I did a whole bunch of research and work on fine-tuning Inkbot to create knowledge graphs. The structure returned is proper YAML, and I got much better results with my fine-tune than using GPT4.
https://huggingface.co/Tostino/Inkbot-13B-8k-0.2
Here is an example knowledge graph generated from an article about the Ukraine conflict: https://gist.github.com/Tostino/f6f19e88e39176452c1a765cb7c2caff
Is your approach to constructing the F/T dataset written up anywhere?
Thanks for sharing the model!
See the info I just posted here: https://www.reddit.com/r/LocalLLaMA/comments/186qq92/comment/kbbpnel/?utm_source=share&utm_medium=web2x&context=3
I haven't written up anything more comprehensive yet.
Still, it's sort of cool for us non programmers to be able to do this: https://poe.com/s/MLqxYzcczvnfnUkozR52
Agreed that it is quite cool, but you don't need to be a programmer to use a custom model.
Inkbot works just fine with ooba or sillytavern if you want to use a UI, TheBloke has done quants.
Great work! Would you mind sharing the datasets you used and/or how you augmented the data for training?
I'll give you some better examples, just didn't have time right then. Give me a few.
It was trained on a whole bunch of prompts asking for each task, so it's not reliant on the exact wording from one of them in training to work. Set the task in the meta section as "kg", and the model will respond with a knowledge graph if you ask for one (and sometimes if you don't).
Here are a few of them:
Create a Knowledge Graph based on the provided document.
Create a Knowledge Graph based on the details in the conversation.
Your task is to construct a comprehensive Temporal Knowledge Graph
1. Read and understand the Document: Familiarize yourself with the essential elements, including (but not limited to) ideas, events, people, organizations, impacts, and key points, along with any explicitly mentioned or inferred dates or chronology
- Pretend the date found in 'Date written' is the current date
- Create an inferred chronology (e.g., "before the car crash" or "shortly after police arrived") when exact dates or times are not available
2. Create Nodes: Designate each of the essential elements identified earlier as a node with a unique ID using random letters from the greek alphabet. Populate each node with relevant details.
3. Establish and Describe Edges: Determine the relationships between nodes, forming the edges of your knowledge graph. For each edge:
- Specify the nodes it connects
- Describe the relationship and its direction
- Assign a confidence level (high, medium, low) indicating the certainty of the connection
4. Represent All Nodes: Make sure all nodes are included in the edge list
I haven't noticed a huge difference in the outcome at inference time depending on prompt used, but sprinkling in some more detailed instructions helped lower loss when training.
As far as dataset, I used a little bit of the Dolphin dataset, to not lose the usual conversational ability. A little bit of the SponsorBlock dataset as a seed, and then I improved it, and the rest is custom...I spent ~$1k or so on API calls creating it. I plan on releasing it at some point, but I want to improve some aspects of it first.
Total dataset size I used for training is ~85mb.
Alright, here are two full logs, Inkbot generated everything below the <#bot#> response.
Simple prompt: https://gist.github.com/Tostino/c3541f3a01d420e771f66c62014e6a24
Complex prompt: https://gist.github.com/Tostino/44bbc6a6321df5df23ba5b400a01e37d
So in this case, the complex prompt did perform better.
Great work, this is impressive, especially for a 13B model!
It was not an insignificant amount of work to get it working as well as it is tbh.
For example, one of the tweaks I did that had the most impact...you'll notice the node IDs are all greek letters. They were originally contextually-relevant IDs, like the name of the entity in the graph.
```
- id: Eta
event: Construction of the Eiffel Tower
date: 1889
```
would have been
```
- id: eiffel
event: Construction of the Eiffel Tower
date: 1889
```
But that lead to the model relying on context clues from that piece of text, rather than being forced to actually look up the data in the knowledge graph during training. So switching that out to use a symbol approach worked much better for relying on data in the graph, rather than model built-in knowledge.
I was planning on testing that out on my own, but then I ran into this paper: https://arxiv.org/abs/2305.08298, which made me pull the trigger and convert my whole dataset and creation process to support symbolic identifiers.
That's really interesting, thank for sharing!!
How do the querying process work for this 'knowledge graph"?
a KG can be implemented with a database, graphDB are well suite for that.
but beyond the tech, how do you actually automate query/answer?
I am looking at a library to generate query/answer tuples from a KG, in order to further fine-tune a model.
still a WIP
This is a really good question and i'd also like to understand how to use the knowledge base with an LLM
Finally, a question on this sub that is not about an "AI girlfriend" (ahem RP)
There are about a dozen + different ways to incorporate KGs into an LLM workflow with our without RAG. Some examples:
## Analyze user question, map it into KG nodes and extract connectivity links between them. Then put that info into the LLM prompt to better guide the answer.
Example: "Who is Mary Lee Pfeiffer's son and what is he known for"? (b.t.w. try this on ChatGPT 3.5)
## Use KG for better RAG relevancy.
Example: Assume your KG is not about concepts but simply links paragraphs/chunks together. This could be simple as mining links like (see Paragraph X for more detail), Doing semantic similarity between chunks, putting in structural info like (chunk is part of Chapter X, Page Y), topic or concept -based connectivity between chunks.
Then, given a user query, find the most relevant starting chunk, Apply logic for what is "more relevant" from your application to figure out which other linked chunks to pull into the context. One simple hack, using node centrality or Personalized PageRank is to pull in chunks that are indirectly connected, but have high prominence in the graph
Thank you for you answer! I've worked hard to improve my personal RAG implementation, searching (and asking here) ad nauseam to find ways to enhance the performance of the retrivial process...
i will study over this approach linked in the OP post, and your answer really helped me to take everything to a more "practical / tangibile" level.
I'll try to integrate that on my experimental pipeline (currently I'm stable on RAG fusion using "query expansion" and hybrid search using transformer, SPLADE and bm25.
i already tried an approach that need a LLM to iterate over every chunk before generating embedding, mainly to solve pronouns and cross reference between chunks.... Good results... But not good enough if analyzed in relation to the resource needed to iterate the llm over every item. Maybe the integration of this "knowledge nodes/edges generation" in my "llm" pre processing will change the pro/cons balance since, from a rapid test, the model seem able to do both text preprocessing and concept extraction in the same run.
Thanks again!
.
Finally, a question on this sub that is not about an "AI girlfriend" (ahem RP)
I had many good discussions on this sub, and I really like that community... Anyway, i got your point Lol.
Thanks for this. I've only worked with RAG on OpenAI models and there's a lot of prompt finetuning needed to get decent results. A KG helps define the semantic elements and relationships between document fragments and the user query for RAG.
That said, I'm still relying on the vector database to do most of the heavy lifting of filtering relevant results before feeding them into an LLM. Having an LLM clean up or summarize the user query and create a KG from the vector database's response could lead to more accurate answers.
Having an LLM clean up or summarize the user query and create a KG from the vector database's response could lead to more accurate answers.
That is the promise. Of course, you still need to figure out for your app domain if doing a concept-level, chunk level, or some in-between option like CSKG is the right application.
One thing I find helpful with prompt design is to spend less attention on writing instructions, replacing them with specific examples instead. This replaces word-smithing with in-context learning samples. You build up the examples iteratively, running the same prompt through more text, fixing it and adding onto the example list.... until you reach your context budget for the system prompt.
Just for some meta fun, I had Claude 2 analyze the tutorial and then creat3 a knowledge graph out of it: https://poe.com/s/V45iXNtYahh05qE7N3YU
This is solid work and shows how you can add training without you know, training.
Thanks for this, I have been looking into this for the last month solid.
Awesome work!