Very much pro Open Source AI. Especially as a concept digital public good. With https://petals.dev/ being the most promising option that regard (imagine something like RAG for the arch wiki with very large models supported by the community!).
It feel very enthusiasts right now. Where I feel like I'm just on the cusp of having usable set up.
I personally really want a full Dev that just takes gitlab issues and runs codes against tests until it passes, and then cycles between attempting to explain what it doing and refactoring until that explanation is reasonably simple, then submit PR.
At the moment I am trying to use it as a copilot (ollama lama3, continue, and devonAI vscode plugins) all on my MacBook (my Linux machine were too small gpu wise, at least first time I attempted). That said it ok for questions no real luck on a decent experience for actually making anything.
The next step to me for it to move from enthusiast to hobbiest would be:
- Models that just work on my machine. I had to do a lot of trial and error just get performant models.
- Models just my use case. I don't know what model support tooling, or multimodal inputs. What models are actually optimized for programing, for actions (ala openinterpretor), for reviewing documents, etc.
- For federated (like pedals.dev) I feel like I need some sane data guardrails. I don't want my medical documents anywhere near "bittorrent style" anything, but would absolutely love to leverage it for better outcome on opensource projects without secrets file. This also feeds into point 2 to me.
- More sane RAG. Maybe even IPFS links to caches or DBs for popular data sources (like code docs for example).
I feel like there has to be a better way for this. Maybe its just selinux rules for data tags for locking down my local system and some routing config file at the root of my projects. Idk tbh