Machine Learning

1 readers

1 users here now

Community Rules:

Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.

founded 2 years ago

MODERATORS

communick@academy.garden

[D] "AI systems are always deterministic," AI teacher says. How can I reply (with examples and papers)? (alien.top)

submitted 1 year ago by shiftingsmith@alien.top to c/machinelearning@academy.garden

53 comments fedilink hide all child comments

I had a discussion in class with one of my teachers. He says that AI is and can only be always deterministic because "even a deep learning neural network is a set of equations running on a computer, and the stochastic factor is added at the beginning. But the output of a model is always deterministic, even if it's not interpretable by humans."

How would you reply? (Possibly with examples and papers)

Tysm!

you are viewing a single comment's thread
view the rest of the comments

[–] woopdedoodah@alien.top 1 points 1 year ago

Once trained, a models outputs are completely determined by its inputs at a mathematical level. This is easy to prove. You write out the equation for each output variable (it's large but possible) and observe that each parameter is constant and not a random variable. Thus the output is also constant given some inputs and weights.

Training is arbitrary but also deterministic, since the combination of the initial states, training batch order, and optimization algorithm and parameters, determines the output.

If you feed random parameters into any of these then the output is arbitrary, but not random

For example at inference time, a transformer is not using any random number, not even any pseudo random number.

I think you are mistaking the sampling procedure with the model itself. The sampling procedure is often pseudo random. The model usually is something that produces a probability distribution. That's deterministic.