Proof (GAN optimal point)

We have mentioned optimizing GAN is optimizing JS-divergence. This is not obvious from the cost function:

We will prove it here and also prove the corresponding optimal point is:

• p = q,
• the discriminator cannot distinguish the real from the fake D*(x) = 1/2,
• V* equals -2 log2.

Proof:

If G is fixed, the optimal Discriminator D* is

Find the optimal value for V:

i.e.

Hence, from the equation above, the minimum (optimal point) for V is when p = q. (when D_JS = 0)

Optimal discriminator

The optimal value y* for

is

Proof: