Hello, I'm working on an ERP system and would like to integrate it with an AI assistant. I would like that assistant to give answers based on the data in my system, in real time. For example, who is working where, and who is busy or free at the moment? Are there any issues, etc.?
On top of that it would be ideal if I could also issue commands/instructions and that assistant could execute corresponding actions based on them. If the assistant doesn't have enough info about a certain action he could ask a follow-up question(s) until he has.
The way I'm thinking about implementing this is something close to RAG (Retrieval Augmented Generation), where I would provide the full context to the assistant before asking the question. For instructions, I would provide the types of instructions that exist and tell the assistant that he can only use those.
The main issue with this is that LLMs have a limited context window size (they can only take N tokens - fixed) and my data from the system might exceed that.
Before I start thinking about how to implement this, I thought that I could first check if there are any existing tools on the market that do this for you.