this post was submitted on 19 Nov 2023
1 points (100.0% liked)

Machine Learning

1 readers
1 users here now

Community Rules:

founded 1 year ago
MODERATORS
 

I'm trying to teach a lesson on gradient descent from a more statistical and theoretical perspective, and need a good example to show its usefulness.

What is the simplest possible algebraic function that would be impossible or rather difficult to optimize for, by setting its 1st derivative to 0, but easily doable with gradient descent? I preferably want to demonstrate this in context linear regression or some extremely simple machine learning model.

you are viewing a single comment's thread
view the rest of the comments
[–] muggafugga@alien.top 1 points 11 months ago

Here's an example of fitting a 3rd order polynomial I worked with recently that might be useful for you. The code is largely part of the pytorch documentation, with an animation to illustrate the process.

https://gist.github.com/jlines/00eae9590d26b31cb98b333e537fcb31