This code uses txtai, the txtai-wikipedia embeddings database and Mistral-7B-OpenOrca-AWQ to build a RAG pipeline in a couple lines of code.
this post was submitted on 10 Nov 2023
1 points (100.0% liked)
LocalLLaMA
4 readers
4 users here now
Community to discuss about Llama, the family of large language models created by Meta AI.
founded 2 years ago
MODERATORS
I was already super interested in txtai, but you are the best for the wikipedia embeddings link too. I'm definitely playing with this soon
how can this be used for code generation with a github repo and its documentation?
Well for RAG, the GitHub repo and it's documentation would need to be added to the Embeddings index. Then probably would want a code focused Mistral finetune.
I've been meaning to write an example notebook that does this for the txtai GitHub report and documentation. I'll share that back when it's available.
Textai is fantastic!!
The choice of question in there is particularly insightful. All AI-related tasks should focus on spiders.
Hi David,
I'm very impressed by your work, not only the library itself but also the documentation, which is crystal clear and very well illustrated.
I'm just curious, how do you monetize your work?
Thank you, appreciate it.
I have a company (NeuML) in which I provide paid consulting services through.
Can this query my docs too?
Yes, if you build an embeddings database with your documents. There are a ton of examples available: https://github.com/neuml/txtai
This looks incredibly useful
Looks like it can work with AWQ models. Can it work with GPTQ (Exllama2) and GGUF models?
It works with GPTQ models as well, just need to install AutoGPTQ.
You would need to replace the LLM pipeline with llama.cpp for it to work with GGUF models.
See this page for more: https://huggingface.co/docs/transformers/main_classes/quantization
Im trying to wrap my head around this :)
But will this (conceptually) also work for Atlassian (Jira and Confluence) instead of wikipedia
In a way, that you can use semantic search through jira and confluence
Instructions unclear; my chat is now full of spiders.