Free and Open Source Software

17916 readers

86 users here now

If it's free and open source and it's also software, it can be discussed here. Subcommunity of Technology.

This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

founded 2 years ago

MODERATORS

Gaywallet@beehaw.org

alyaza@beehaw.org

Chat with Documents - Ollama or Other? (beehaw.org)

submitted 3 months ago by its_me_xiphos@beehaw.org to c/foss@beehaw.org

2 comments fedilink hide all child comments

Been having too much fun using LLMs hosted locally, but can't seem to get Ollama's chat with documents to work well. Lots of "what are you talking about? There are no documents here" issues. Does anyone have any recommendations to either a) figure out what's going wrong or b) Alternative locally hosted options that chat with documents works well with (GPT4all or something?)

you are viewing a single comment's thread
view the rest of the comments

[–] bazsalanszky@lemmy.toldi.eu 5 points 3 months ago* (last edited 3 months ago) (1 children)

I haven't tested it extensively, but open webui also has RAG functionality (chat with documents).

The UI it self is also kinda cool and it has other useful features like commands (for common prompts) and searching for stuff online (e.g. with searx). It works quite well with Ollama.

[–] its_me_xiphos@beehaw.org 1 points 3 months ago

Thanks for this. Time to play around and experiment.