We've been building LLM based tools for months, but I think that there should be efficient frameworks by now that actually add value. I tried langchain a while back but I felt like it was just an over complicated overhead where it was always simpler to make everything from scratch each time. Guidance has been the only real improvement for me as it does way more than basic prompt templating, but it is in no way a full framework.
Now there are LlamaIndex, TigerLab, Langchain... but I simply don't have the time to test them all.
We need to run the models by ourselves, so no Open AI api, ideally run something compatible with TGI / VLLM. We need to connect to proper databases and vectorDB (currently using Milvus). And I'm looking for something that is actually useful and I don't have to struggle and hack the library everytime I want to do something slightly different.
Does any of you have a good recommendation? Everything changes so quickly I feel like I can't trust articles that are older than two months. So what are you currently using and what has been an overhyped crap?
AFAIK there is none, and there is a good reason why. I built a small guide for my company 2/3 months ago. Most of it is now in some version of deprecated, the field is moving extremely quickly and every general guide needs to be updated constantly. For now the only way I see is to use all the inevitably somewhat deprecated guides out there and keep your mind open for updates.