0. Introduction to GANs

Generative Adversarial Networks

GANs are generative models: they create new data instances that resemble your training data

They usually fall under the category of deep unsupervised learning models.

More formally, given a set of data instances X and a set of labels Y:

• Generative models capture the joint probability p(X, Y), or just p(X) if there are no labels.
• Discriminative models capture the conditional probability p(Y | X).

GANs are just one kind of generative model. Other kinds include autoencoders or normalizing flows.

For example, models that predict the next word in a sequence are typically generative models (usually much simpler than GANs) because they can assign a probability to a sequence of words.

Fig. 1) A D model learns a boundary that separates positive examples from negative examples. A G model learns where similar examples are placed in the data manifold.

1. GAN Anatomy

Generative models have to extract more information than discriminative models. Discriminative models try to draw boundaries in the data space, while generative models try to model how data is placed throughout the space.

GANs give a framework to actually train a generative model with backpropagation and using a discriminative model as a partner.

A generative adversarial network (GAN) has two parts:

• The generator learns to generate plausible data. The generated instances become negative training examples for the discriminator.
• The discriminator learns to distinguish the generator's fake data from real data. The discriminator penalizes the generator for producing implausible results.

Fig. 2) Simplified diagram of a GANs. G: generator, D: discriminator, R: real data

The generator and the discriminator are neural networks. Their weights are updated through backpropagation based on the loss function results.

1.1 The discriminator

The discriminator is generally speaking a classifier: it could use any NN that can classify examples based on labels.