DarkSouth

Hi there,

I’ve recent tried textgen webui with ex-llama and it was blazing fast so very happy about that.

I noticed ooba doesn’t have rag functionality to pass in documents to vectorise and query.

I’m after a similar tool with the following capabilities:

• GPU support
• Can vectorise multiple files at once
• windows or ubuntu support

Any help would be appreciated! Tried a few but they didn’t fit all 3 of the above.

Thanks.

Hey everyone,

Looking to get into some ML.

Can a GTX 4070ti with 12gb VRAM alongside 32gb ram run 13b comfortably?

I seem to read conflicting opinions on this.

Thank you!