Machine Learning

1 readers

1 users here now

Community Rules:

Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.

founded 1 year ago

MODERATORS

communick@academy.garden

[D] Rust in ML (alien.top)

submitted 11 months ago by IndependentFresh628@alien.top to c/machinelearning@academy.garden

9 comments fedilink hide all child comments

Why hasn't Rust gained the same recognition as Python in AI, ML, and DS, despite its significant ecosystem potential and notably better speed compared to Python?

Also, why haven't people actively developed Rust libraries for data analysis and processing, similar to what has been done in Python?

top 9 comments

sorted by: hot top controversial new old

[–] something_cleverer@alien.top 1 points 11 months ago

A lot ML/AI is research where iteration speed is more important than runtime performance or long term maintainability (software engineering).

A lot more programmers know Python than Rust, so it’s easier to prototype a new project in a new domain without having to learn a new language at the same time.

A lot of Python calls C libs to do the heavy lifting, and that’s where the “real” ML/AI code is written, rather than the web apps built around it.

All that said, at PostgresML, we’ve found Rust great for not just ML and AI, but also database and web application development.

https://postgresml.org/blog/postgresml-is-moving-to-rust-for-our-2.0-release

[–] deepneuralnetwork@alien.top 1 points 11 months ago

python has everything we need and is well supported across the field. why in the world would we change?

[–] jacobgorm@alien.top 1 points 11 months ago

Because python allows you to prototype and iterate quickly, whereas in Rust you have to fight the compiler every step of the way to convince it to do what you want. People have been trying to build DL frameworks in languages such as Swift and C++ (dlib, Flashlight) but none have taken off.

Python can be a pita due to stuff like lack of multi-threading, but for most things it is quick and easy to experiment in, and the amount of code you have to write is not too far off from the corresponding mathematical notation, so for now I think it will keep its position as the most popular language for AI/ML.

Before we could use python, most researchers were using Matlab, which was really holding down progress due to its closed-source nature.

[–] mtocrat@alien.top 1 points 11 months ago

Training jobs for LLMs are quite possibly the biggest programs ever run in terms of compute and yet they use python. "Better speed" is often not true in a way that is actually relevant since cpu speed is generally just not your bottleneck.

[–] doodeoo@alien.top 1 points 11 months ago

Because data scientists are not software engineers

[–] slashdave@alien.top 1 points 11 months ago

Python enables rapid prototyping. Bindings such as numpy make CPU speed less of an issue.

[–] Red-Portal@alien.top 1 points 11 months ago

Not only that, the numerical computation and high performance computing landscape of Rust is very primitive.

[–] LegacyAngel@alien.top 1 points 11 months ago

Rust hasnt even become mainstream in browsers or embedded systems yet. Its purpose. Why would it be used in ML when languages like julia arent popular?

[–] chonk-boy@alien.top 1 points 11 months ago

As some above comments mentioned, iteration speed is the most important in ML, so I don’t see python being replaced by something else soon. However, there are some Rust projects wrapped with python for better efficiency and safety (Huggingface’s Tokenizers and Safetensors for instance).