GANs indeed learn to generate samples from the data distribution.
VAEs learn how to encode samples to parameters (mean, variance) of a latent-distribution. De VAE-decoder then maps samples from that latent-distribution back to input samples. If basically an autoencoder that tries to do "input->code->reconstructed input", but with the code being a compact probability distribution instead of a point.
You can use a VAE as an outlier detectors by looking at the reconstruction error. If you e.g. have trained a cat-image VAE then if will ouput cat images. You can generate random code samples and run those through the decoder sttep and then you'll get random cat pictures. If you feed it a cat picture, encode it, and then decode it, you get something similar to your original input cat image out again. This is because it is an auto-encoder. The reconstruction error is small in this case. If you however feed it a dog image, then the encoder will try to map it to a cat-code, however, the decoder will then still always generate a cat image. In this case the input dog image and the ouput cat image will have a larger distance / reconstruction error.
There is yet another type of generative models called "flow models" that explicitely model the data density. Flow models use invertible function and allow you to evaluate the pdf directy, whereas VAEs only tell you how well it can auto-encode a sample, and it will be trained to do that (only) well for samples from the trainset.
Yes, indeed, good point. You can also use the discriminator or a GAN for anomaly detection.