this post was submitted on 08 Nov 2023
1 points (100.0% liked)

LocalLLaMA

3 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago
MODERATORS
communick@poweruser.forum
1
Rag vs Vector db (alien.top)
submitted 1 year ago by troposfer@alien.top to c/localllama@poweruser.forum
9 comments fedilink hide all child comments
 

I am confused about these 2 . Sometimes people use it interchangeably. Is it because rag is a method and where u store it should be vector db ? I remember before llms there was word2vec in the beginning ,before all of this llm. But isn’t the hard part to create such a meaningful word2vec , by the way word2vec is now called “embeddings” right?

you are viewing a single comment's thread
view the rest of the comments
[–] bortlip@alien.top 1 points 1 year ago (1 children)

word2vec is a library used to create embeddings.

Embeddings are vector representations (a list of numbers) that represents the meaning of some text.

RAG is retrieval augmented generation. It is a method to get better answers from GPT by giving it relevant pieces of information along with the question.

RAG is done by:

  1. taking a long text splitting it into pieces
  2. creating embeddings for each piece and storing those
  3. when someone asks a question, create an embedding for the question
  4. find the most similar embeddings of sections from the long text say the top 3
  5. send those 3 pieces along with the question to GPT for the answer

A vector DB is a way to store those section embeddings and to also search for the most relevant sections. You do not need to use a vector DB to perform RAG, but it can help. Particularly if you want to store and use RAG with a very large amount of information.

[–] Life_Inspection4454@alien.top 1 points 1 year ago (1 children)

This is a very good answer, but I'll try to elaborate to make things clearer:

RAG is done by:

  1. Taking a long text and splitting it into chunks of a certain size/length.

  2. You take each chunk of text, and run it through a function which turns the text into a vector representation. This vector representation is called an embedding, and the function used is an embedding function/model. E.g. OpenAIEmbeddings(). You then generally store these vectors in a vector database (Qdrant, Weviate ++).

  3. When someone asks a question, create an embedding for the question.

  4. Since your question is a vector (embedding), and your data is represented as vectors (embeddings) in your vector db (from 2), you can then compare your question vector with your data vectors. Technically you measure distance between your question vector to vectors in your vector db. Vectors closer to your questions, is likely to contain data relevant to your question.

  5. You grab the text corresponding to the (e.g.) 3 closest vectors from your vector db. The text is often stored along with the vector for retrieval purposes. You send that text + question to your LLM (e.g. GPT-4), and implicitly say: "Answer this question based on only these 3 chunks of text." That way you sort of limit the language models knowledge to what you explicitly give it.

[–] troposfer@alien.top 1 points 1 year ago

Oh thanks in 5. You also answered 1 question in my mind, how to return back to words from floatin point numbers. Then now i understand they are created by specific embedding creator models. And I guess every result is different then other models result. So isn’t this so important like best embedding creator model and query creator model, which one is more successful right now now? And if i create an embedding in one creator model , i can’t create an embedding query with different embedding creator model to query my embedding?