26. Generative Adversarial Networks
Mar 13, 2024
In this video we cover the first video on generative machine learning models: the generative adversarial network. We describe what they are, how they work, why they are so hard to train and more.
hey everybody this is Taylor Sparks
welcome back to my materials informatics
series now today we're going to dive
into generative machine learning models
and specifically we're going to dive
into generative adversarial networks as
the first of the different models that
are out there so let's get started we
benefit in the the world of materials
informatics of being adjacent to
computer scientists who are doing all
the real heavy lifting for us in coming
up with cool new uh machine learning
models and algorithm for generating new
things right so we have the benefit of
using things like Gans vaes flow-based
models diffusion models and I'll
describe what all of those are in the
next series of videos um in this series
uh but today's video is going to talk
about the Gan the generative adversarial
network and to kick it off let's start
with this tweet this came from a tweet
from the inventor of the Gan um a guy
named Ian Goodfellow who invented it
back in 2014 when he was a PhD student I
think he was at the University of
Montreal
how awesome is that grad student
invented this um and he posted you know
this tweet showing the progress how you
know they were interested in using Gans
to create faces like you see on the left
here and he showed over the series of
just four years the incredible progress
in making these really realistic faces I
added the one on the right in 2021 and
now in 2024 my gosh they're doing videos
and all sorts of other incredible things
so Gans have proven a really really
valuable tool and in the short you know
period of 10 years years we've man like
it's been a phenomenal tool and it's
been used like crazy including in
Material Science which will be our next
video so let's dive into how Gans work
what are they how do they do what they
do so Gan stands for generative
adversarial Network right and it's the
adversarial part that really makes them
perform so well it's also the reason why
they are so frustrating to use more than
that later um but at their simplest they
start with two different neural networks
and the first is a discriminator and the
discriminator all it is is a classifier
right it's a straight up classifier
where again the classifier is jobs to
classify things like you know that's a
picture of a bird and that's a picture
of a car well this time what it's
classifying is oh that's a real data
point and that's a fake data point so
the idea is that you're going to pass
into your discriminator real samples
from some sort of distribution or
database in our case since we're doing
with Material Science that might be like
a crystal structure database or that
could be real pictures of micr
structures that could be real formulas
right and then we're also going to pass
it fake ones right and the discriminator
job is to figure out real or fake and
get it right and if it gets it wrong
then the loss function right gets
imposed to Via back propagation update
the weights of the neural network so
that it gets better at its job okay and
so the typical thing that you'll see at
the end of the discriminator something
like a sigmoid function right um and
we've talked about sigmoid functions
before they're this really great you
know sigmoid shape that allows it if
it's a negative if the final node of
your neural network for example is a
negative number then it flattens to a
zero and if it's a really positive
number it goes to a one and there's only
this little region in the Middle where
there's any ambiguity which makes them a
good classifier it's a good
discriminator um okay so these are great
but this is just a classifier this isn't
going to generate anything new instead
it tries to learn something about the
underlying distribution from which the
data is sampled now what do I mean by
that and why can't it generate something
new let's let's look at this example
which comes from computer files great
video on Gans let's consider these three
different data samples that I've
generated here now you can see that
there is a sort of underlying pattern to
all of these three individual samples
right and that's the fact that this is
sort of like a signus solal oscillation
right and so what we are asking the
classifier to do is figure out that
underlying distribution right which
might look something like this red line
in the bottom left but then if you were
to use that learned distribution
and try and generate new data points it
might look something like this the data
points are going to sit right on top of
that line because the classifier doesn't
know how to add variants to this it
doesn't know how to add the noise that
we need to make this look like a real
sample right what we want it to do is
spit out something like this where it's
more or less obeying the underlying
distribution but there's some realism to
it it looks like a real sample and a
classifier alone is not going to be able
to do that so clever people people like
Ian good fellow and Company came up with
this clever idea to what they basically
said well what if you have one model
that learns the distribution and another
that tries to generate things that match
that distribution and you pit these two
things against one another and thus was
born the generative adversarial Network
so in this model you see it looks like
what we had before partly you still have
this uh database of real materials that
get passed to a discriminator whose job
is still to figure out real from fake
but instead of starting with some
database of fake materials we're going
to use another neural network our
generator to generate fake samples right
so this could be fake compounds fake
images whatever it is we're talking
about it's going to generate fake
versions and the discriminator job is
going to be to tell the real from the
fake and the catch is this the loss
function is going to apply to both
neural networks right the loss function
in one case the discriminator wants to
do a really good job at correctly
classifying real versus pH
but the discriminator wants to do a
really good job at fooling the
discriminator such that it does a bad
job on classifying real versus fake and
so we're going to take information from
the loss function when it gets it right
or when it gets it wrong and we're going
to use that to do back propagation to
both of these models sort of separately
um but the benefit here is that once you
get this trained and working well well
then you could take random noise which
sort of represents our chemical white
space um and we'll talk about how to
impose constraints on that in a later
video on optimization but essentially in
comes random noise out comes a realistic
looking structure or micro structure
whatever it is about the materials and
as the better the generator gets and the
better the discriminator get they start
to pit one against another the classic
example is always that of like a a con
artist and a forger sorry a con artist
and a detective so the con artist might
might want to make fake paintings of you
know whatever Monae or whatever Picasso
and yet the discriminator is going to be
our detective is going to want to do a
better job of catching the con artist
and the better the detective is the
better the con artist must be and vice
versa this adversarial training allows
these two to pit against one another and
that's kind of honestly how how humans
work right when I'm teaching my kids to
speak or do different things I'm also
I'm pointing out the things that they
get wrong and we are constantly
challenge them as they learn to speak
and walk things to focus on the things
that they're not doing quite as well
right so it's a really great opportunity
but it's also what we call a minmax game
what do I mean by a minmax game well if
you look at the loss function right when
something comes out of the discriminator
and we assess whether it got it right or
wrong we can actually Define a loss
function for both our discriminator D
and our generator G so the loss function
is going to be V here and what it
essentially is showing you is two
different terms and the first term is
showing you the discriminator prediction
on real data how effective is it the
second term is going to be its
prediction on fake data how good is it
at catching The Fakes right if you dig
into here e subx where X is taken from a
distribution of data right this our X is
going to represent our real data so the
expectation when you send it real data
to the discriminator hence d ofx right
when we take log of that term we want it
to be high right in other words we want
it to correctly uh uh classify real data
however the expectation when you send it
fake data denoted as Z from some
distribution of fake data we want that
to be 1 minus the log of D of G of Z so
what is this D of G of Z well remember
the the noise Z gets passed to our
Generator First hence why it's G of Z
first that is now generating a fake
sample G of Z is the fake sample and we
then pass that to the discriminator so D
of G of Z is the evaluation on that fake
sample so we want that to be low right
we don't want it to fall for that hence
why it's one minus that log term um and
it's interesting because now we have
this scenario where the discriminator
and the generator are doing two
different things the discriminator for
example wants to maximize this V
expression while the generator wants to
minimize V okay so hence the
mathematical expression for how we write
this is min G Max D so let's see how
this looks when we actually go through
training the training steps you know
just like when you train a neural
network that it's done in Epoch where
you're showing the data continuous over
and over we have the same opportunity
here we're going to do K number of steps
think of this like the number of epochs
you're going to go through and during
each Epoch you don't train necessarily a
single image you could train multiple
images or multiple structures whatever
your representation is so we're going to
train um M number of noise samples we
start out with M number of noise samples
and we turn those into fake data by
passing them through our generator
Meanwhile we're going to take M number
of real samples so X now instead of Z
and those represent real data so we pull
those from from some real database we're
going to use both of those and we're
going to use them to generate and update
our discriminator by ascending the
gradient so we take that loss function
that we saw before right this one now we
take the derivative with respect to the
different uh Network that's in use here
so for example here we're taking the
gradient with respect to x and z our
input samples and we're seeing how the
parameters of our discriminator change
as we go through the different batch of
things so we're looking at the log of
how well it performs at discriminating
real samples and the log of one minus
how well it does at discriminating fake
samples
okay then we're going to go to our next
step because we're going to train these
models sequentially this time we're
going to look at at the noise samples
that we transformed with our generator
and we're going to update our generator
by descending the gradient right so this
time we're taking the the the gradient
right with respect to the generator the
parameters of the gr the generator and
you'll notice that when we do that
there's no generator in this first term
so we can actually ignore this first
term which is kind of interesting that
which is just another way of saying that
when we update the parameters of the
generator we don't need real samples at
all we only need Z are noise samples
that get generated that's all we need to
generate the generator so there there's
some more math that goes along with this
that some people look into for example
you can there's a proof that shows that
you can minimize G right G the the loss
function gets minimized if and only if
the distribution from which you're
sampling data your noise samples um
matches real data right so if you don't
have that then you can't perfectly
minimize it but the challenge here is
that you've got two different things
that are essentially being trained and
they it's like a tug-of-war between them
that leads to some interesting
challenges and in practice working with
Gans can be kind of a pain so for one
thing Gans are really good at what they
do if you train them and they converge
and you get it you know working
correctly they are among the most
powerful generative models that are out
there they can create really high
quality realistic fake data however they
do suffer from this uh training
instability so what's going on why is it
unstable well there's a couple reasons
first off we have what's called a
non-convex game which is just a matthy
way of saying that there's not a single
Global solution there can be local
Solutions as opposed to a global Optimum
there can be local optimums and so how
do you know if you found the best
solution or just a local one that's
non-convex right you also have this
problem of Nash equilibrium in this
non-convex game where a Nash equilibrium
comes from Game Theory and it's
essentially saying in if there's a
strategy game where there's two
different players that are competing um
Nash equilibrium says that there's no
way to benefit a single player in
changing their strategy if the other
player doesn't change their strategy in
other words the generator and the the
discriminator are pitted against one
another and there's this constant
tension as they're trying to um improve
by seeing the other one change their
approach right you run into this problem
of mode collapse where a certain
solution turns out maybe to be really
good at fooling the discriminator the
generator comes up with a clever way to
trick the discriminator and it's so
clever that the discriminator never
figures it out and so the generator
never has a reason to change something
so it might create a certain type of
structure that fooled it and then it
will just keep on making variations on
that certain type and and that's mode
collapse because we want it to explore
the whole breadth of potential Solutions
but it might get stuck in a single one
that happens there's something called
Vanishing gradient so if your
discriminator is
outperforming then your generator's
gradient vanishes and therefore your
generator can't learn which makes the
discriminator even better which is like
a feedback loop that makes this even
worse and the other way around you can
have exploding gradient if your
generator is outperforming well then
your discriminator loss escalates and
that gradient explodes and you have the
same problem so there's this real
Balancing Act and this is the challenge
with uh Gans you might be training them
for a week and then all of a sudden it
will diverge right you want it to
converge and get to your whatever you
hope it's the opt the global solution
but you don't know but you're hoping
it's a good solution as it converges but
it might diverge which is frustrating
when you've been watching it train for a
week and all a sudden you lose your
progress right there's also a lot of
sensitivity to hyperparameters your
learning rate the batch size the
architectures that you use these are all
totally tunable things and so that can
be really challenging so what can we do
to make Gans in general work better well
first off there are alternative loss
functions the discriminator uses
typically a sigmoid um and sigmoids are
good discriminators but they also
approach asmp tootes as it gets really
fake data or really real data it
approach it approaches an ASM toote I'll
show you this in my next slide uh in my
next video in the series when we talk
about applications materials that that
can be a problem because you get this
Vanishing gradient um there are
regularization techniques where you sort
of penalize it for using too many
features or trying to prevent it from
overfitting and there's also lots of
different architectures in particular
things like conditional Gans or was
steam Gans more on those in the next
video and these give additional
information to the generator or
discriminator that can make them perform
a little bit better so with our
foundation set for how Gans work let's
dive into our next video which is how
Gans have been used in Material
Science

