Machine Learning

1 readers

1 users here now

Community Rules:

Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.

founded 11 months ago

MODERATORS

communick@academy.garden

[D] GANs do not have Density estimation abilities (alien.top)

submitted 9 months ago by racc15@alien.top to c/machinelearning@academy.garden

15 comments fedilink hide all child comments

Hello.

I am seeing the GAN specialization course on Coursera by instructor Sharon Zhou.

In one of the lectures, she says that a disadvantage of GANs is that it cannot do density estimation and thus is not useful for anomaly detection. (not sure if you can access it).

In the next video, she says that VAEs don't have this problem.

I am a little confused about this. Could anyone please explain what she means?

As far as I can understand from the lecture, density estimation means learning how probable/frequent particular features are in a dataset. Like, how probable is it that a dog will have droopy ears. Then, we can use this info to detect anomalies if they do not exhibit these features.

But, isn't this exactly what GANs learn? aren't GANs learning to mimic the distribution of the training data?

Also, how is a VAE different in this particular regard?

Could someone please help explain this?

Thank you.

you are viewing a single comment's thread
view the rest of the comments

[–] cwkx@alien.top 1 points 9 months ago (1 children)

If you look at the review: https://arxiv.org/pdf/2103.04922.pdf in Table 1, you'll see in the rightmost column GANs don't have any "NLL" - this stands for the negative log likelihood, or if you like the model's density fit over the distribution. Other classes of models, like VAEs, only give bounds on the density (approximate densities). Flows and autoregressive token predictors can give exact densities. The discriminator of GAN just estimates if something is real or fake, not estimating true probabilities (densities). Adversarial training can, however, be used for anomaly detection (and works quite well, e.g. GANomoly and successors).

[–] racc15@alien.top 1 points 9 months ago

thanks!

there is so much stuff to learn!