You can do this pretty seamlessly with hybrid versions of opensearch, combining traditional search queries with vector based embeddings. Vectorize your item descriptions and then turn your free form user queries into hybrid queries that you feed to open search. It’s possible to feed complex natural language queries like “find me some cheap crocs or sandals listed in the last 12 days” and ask an LLM to transform that into a JSON format valid opensearch query. Gpt4 can do this flawlessly if you also provide a JSON Schema that matches your open search index.
I did something very similar recently and I also wasted an enormous amount of time screwing around with haystack, llama index, langchain and the like. I highly recommend avoiding these things if you are trying to build anything custom or that needs to scale or that needs to go onsite.
These frameworks all provide a few nice basic app ideas, and then wrap a laundry list of dedicated vectordbs that I predict will disappear within 2 years. Beyond that they have huge Java like OOP implementation styles that make it a huge pain to hack around or prototype without providing the scalability you’d typically like to trade for that bloat. Plus since they try to treat all the backends the same and this ends up giving more powerful multimodal data stores like Postgres and opensearch short Schrift by not supporting their more useful and sophisticated hybrid search features.
I finally discarded all of it and just started over with an api backend implemented with fastapi and opensearch and oss embeddings.
I’m also wary of OpenAI - gpt4 is great but for this use case it’s best to treat it as a prototyping mechanism and then fine tune a llama2 variant to do the query wrangling. Otherwise it’s too slow, to unreliable (service wise) and way too expensive. There’s no advantage to using OpenAI embeddings.
My 2c. Sounds like a super fun project (at least if it goes like mine has).