LocalLLaMA

Musk's new in-progress LLM is apparently being built with Rust and JAX.

We are using rust (webassembly) in Edgechains. Looking for contributors to help build the AI webassembly runtime!

I do a great deal of work with AI using Go.

TL;dr: I strongly recommend using Go with the llama.cpp bindings (https://github.com/ggerganov/llama.cpp).

Go is particularly well-suited for LLMs, but in a different way than most people discuss here. Firstly, Go is a better language for LLMs to learn and write. This becomes evident when comparing Go with other languages across a variety of coding tasks using ChatGPT. The code generated for Go is often much better: more code can be correctly generated per prompt, and more complicated logic is often handled well in Go, while it's difficult to generate in other languages. This is because Go has less variability; there are fewer alternative ways to do something. Go is very uniformly structured across codebases due to enforced formatting, shares most symbols with C and C++, and has a very complete standard library, making it unambiguous which libraries to include. Training and generating Go code with LLMs is far more performant than any other language I've tried, especially if you use a local model with a Go LoRA, or train your own model on Go.

Go is also well-suited for efficiently handling blocking data tasks such as loading large models from disk to RAM or GPU, or from remote RAM to local RAM, thereby achieving the best available I/O performance (often the bottleneck).

Additionally, Go is the language of choice for web service client and server implementations, working well in contexts where you are using ChatGPT, GitHub, HuggingFace, and other API-based AI and supporting services.

Finally, Go is compilable and cross-compilable to a single executable, making the distribution and remote deployment of applications much less error-prone.

I strongly recommend Go for AI work due to its interaction with all these aspects of the AI ecosystem.

All AI stuff is done in C/CPP, everything else is just a wrapper.

I use Golang for the backend of an Agent-based platform mixed with Python. I use it mainly for the RAG part, but I'm considering expanding to other functions, and I did some small projects fully in Golang and OpenAI API.

Also, there is a langchain port to Golang: https://github.com/tmc/langchaingo, which shows how much can be done with Golang in the LLM world.

I am using Zig for my project (https://www.avapls.com/). Currently, I only do inference, but so far it was awesome and I would definitely do it again.

Zig plays nicely with anything with C interface. ATM I am using GGML but I think if I really needed, I could use torch too.

I also did node.js extension for quick prototypes and so far it worked great. I intend to get back to it one day but it's currently on hold because I don't have time.

https://github.com/cztomsik/ggml-js

BTW: some people also use elixir for ML https://github.com/livebook-dev/livebook