LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

Using Mistral Openorca to create a knowledge graph from a text document (towardsdatascience.com)

submitted 11 months ago by WaterdanceAC@alien.top to c/localllama@poweruser.forum

23 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] Inkbot_dev@alien.top 1 points 11 months ago

It was not an insignificant amount of work to get it working as well as it is tbh.

For example, one of the tweaks I did that had the most impact...you'll notice the node IDs are all greek letters. They were originally contextually-relevant IDs, like the name of the entity in the graph.

```

- id: Eta

event: Construction of the Eiffel Tower

date: 1889

```

would have been

```

- id: eiffel

event: Construction of the Eiffel Tower

date: 1889

```

But that lead to the model relying on context clues from that piece of text, rather than being forced to actually look up the data in the knowledge graph during training. So switching that out to use a symbol approach worked much better for relying on data in the graph, rather than model built-in knowledge.

I was planning on testing that out on my own, but then I ran into this paper: https://arxiv.org/abs/2305.08298, which made me pull the trigger and convert my whole dataset and creation process to support symbolic identifiers.