Beyond Interatomic Potentials - Further Acceleration of Atomic-Scale SImulations)

Machine & Design
2 followers
23 Video Views·Mar 24, 2026
AI Generated·May contain errors
how we can further speed up atomistic simulations with other types of models.
So as a first introduction, although I think most of the people here are already familiar
with interatomic potentials, I will be briefly introducing them. So an interatomic potential
is a scalar function of this form. So it is a scalar function that takes as the arguments
the positions that I called Q. Generally, they are called X, but we will see why I've chosen Q
for this presentation just for consistency. And A, the atomic types of atoms going from 1 to N.
And this is supposed to represent the potential energy of the system.
This function has a few symmetry properties. One is invariance with respect to permutations.
So formally, this is how you would write it mathematically. So exchanging two atoms
as the inputs to this function doesn't change the potential energy. And this function V,
so the potential energy, is also invariant with respect to translations, rotations,
and inversions. So operations of the E3 group. In particular, in this presentation,
even though I will not focus on machine learning interatomic potentials, I will use
some models that have already been used to machine learn interatomic potentials.
And in particular, I will be using some architectures that do not enforce the second
type of symmetry, some among the second type of symmetry that I've mentioned here.
In particular, I will be using the PET architecture. And it is one of many emerging
architectures that are not enforcing rotational and inversion symmetry. So how is this possible?
Well, symmetries are delegated to be learned approximately at training time. This makes the
architecture around three times faster. This was true a few months ago. Now that all architectures
benefit from things like Torch Compile, this number is more like 2x. And it is likely that
it will go down as further optimizations are done, because there is more to optimize
for models that enforce these symmetries, so-called equivariant models. But nonetheless,
even at the current time, these unconstrained models, as we call them, are faster. And they
reproduce correct observables with some caveats. So in around 90% of cases that we've investigated,
they reproduce, for example, in molecular dynamics, the correct observables out of the box.
And in some cases, you need to apply some corrections, like sampling a random rotation
at each molecular dynamic step, which, however, doesn't impact the cost and gives you back
exactly the correct physical observables. So that is, at the moment, about a 2x speed up already.
So now, having seen what interatomic potentials are like, we're now going to explore some
potential alternatives to get faster models for atomistic simulations. So one of them
stems from the observation that forces are actually what we're interested about most
of the time. They are what drives atomistic simulations. And actually, if you want to
calculate a force starting from an interatomic potential, you have to take a derivative.
And these are relatively efficient in the modern machine learning frameworks,
just by backpropagation, but they're not for free. They are generally two to three times
more expensive. And so you can either have a model on the left that calculates forces
as the derivatives of a potential energy surface, negative derivative with respect to the atomic
positions, or you can have a model that directly predicts the forces, so it's being trained directly
on the forces, it doesn't do any backpropagation. And now we call this type of model non-conservative
because it's not guaranteed to conserve the energy during a simulation, and it's around
two to three times faster. Now, the consequences of this lack of energy conservation are, of course,
the non-sum to zero of a work along a closed path, that's the first equation, and the lack of
symmetry of the Jacobian of the forces, which would be the Hessian of the potential energy
surface. Now, this translates into broken phonons, I don't have to show you that,
there's plenty of benchmarks in the literature that show that. Geometry optimization does not
converge, but this is only true for inaccurate models, inaccurate non-conservative models. Now,
people have trained non-conservative models that are way more accurate and that actually give
good geometry optimization outcomes. And finally, MD is the most pathological of them all because,
for example, here I'm showing some MVE molecular dynamics and I'm plotting the temperature as a
function of the time step, and you can see that Orb and Equiformer, which are non-conservative
models, essentially explode. Their molecular dynamics drifts and eventually explodes,
the temperature just keeps going up and the simulation breaks. Meanwhile, 7Net and Mason P0
are conservative and, okay, there is a small imperfection in Mason P0 in the fourth plot,
probably it's not accurate enough, but it's still kind of conserving the energy.
So, these models conserve the energy, but they are slower. Now, how do we fix this? So, we would
really like to keep the speed up and throw away the unphysicality of the simulation. Now, at
training time, actually, we can take advantage of this speed up in a quite neat way by pre-training
on the non-conservative forces, which are much faster, they give you much faster training,
and then fine-tuning on forces that are calculated by backpropulsion. This requires
your neural network, as you can see on the right, to have two heads, two prediction heads,
and you can see on the figure on the right that you can essentially spare two-thirds of the GPU
time by training this one, training an inter-atomic potential in this way, and you can reach the same
accuracy. And you can now do good molecular dynamics with a great speed up, because
we found out a method that allows you to essentially evaluate non-conservative forces
at every step, but then conservative forces every m steps as a correction, and that gives you perfect
energy conservation, as you can see here. So, this method, we call it MPS, multiple time stepping,
and it basically affords the same amount of energy conservation as a conservative model.
And in this case, we are only evaluating one conservative force every eight steps. So,
basically, it's running non-conservatively and in a physically correct way.
Now, based on these observations, we've developed PETMAD. It's a modern universal potential. It's
made for finite temperature applications, incorporates the latest advancements and
uncertainty quantification. We trained it on the math dataset, which is kind of small,
and that just comes from a simple number of parameters versus number of targets
line of reasoning, and also from the fact that BFT is very expensive, so we didn't want to
waste too many resources calculating the BFT targets. Okay. So, these are more or less the
speedup plots that compare PETMAD in black for its conservative version and in gray for its
non-conservative version, and we can see that it's quite a lot faster than other interatomic
potentials. Now, this is a picture from a few months ago. Things have evolved, but
theoretically, things are more or less a 3x speedup thanks to the lack of enforcement
of rotational equivalence and another 3x speedup due to the non-conservative forces,
and you can see that ORB reaches the same speed as the non-conservative version of PETMAD because
it's also a non-equivalent and non-conservative model, but it has a very bad reputation because
the physical observables that you can get from ORB are very wrong in most cases because
they don't allow you to do that multiple time stepping that can fix the physical observables,
and we've been able to calculate many interesting physical properties with this interatomic
potential, including on the left an uncertainty quantified melting point, for example. I don't
have the time to explain what we did in all these cases, but it works pretty well, and you can find
it on github at labcosmo.com. In the same repo, you will find many more models as well that we've
trained on many other datasets. These are for the moment in a branch, but that will be merged
in just a few days, so you have the choice of level of theory, size of the model, and so on.
Now, beyond direct forces, we can achieve even more speed up in molecular dynamics by having a
model that can predict molecular dynamics itself, and what do I mean by that? Here, I'm just
establishing a few notations, a few pieces of notation. So, for example, P, I've called P the
momenta and Q the positions. This, for example, the expression on the top left here is the
Hamiltonian. These are Hamilton's equations of motion, and this is their discretization that
people generally use when they run molecular dynamics. It's called the velocity algorithm,
which is simplistic, reversible, and explicit, so it has all the properties we want. I will not go
into details, but only perceptible, and this will come back to haunt us, we can say.
Now, what does FlashMD do? FlashMD is a model that allows you to predict positions and momenta
in molecular dynamics at a future time step, starting from the current time step, and this
can be done many time steps apart. So, this is the data that is used to train the model.
So, molecular dynamics at a given number of time steps apart. The good news for training a
FlashMD model is that it is deterministic, because it's trained on NDE molecular dynamics,
so a simple velocity-related algorithm, no thermostats, no biostats, and so on,
and that it is composable. So, once you train a FlashMD model on NDE molecular dynamics,
you can apply, then, at inference time, thermostats and biostats, and get sampling
of any ensemble that you made. Now, the bad news is that molecular dynamics is chaotic,
and so after a certain time lag, the learning exercise becomes increasingly more difficult,
and that is exponential. So, there's a quite hard barrier as to what we can learn in terms of
number of time steps for now, and the best we can do for the moment with the models that we
currently have, so the same type of GNN that people use to learn interatomic potentials,
is about 30 time steps. And another piece of bad news is that there is no Hamiltonian,
and so the simulations can have some interesting artifacts, to say the least. So, now, we speed
molecular dynamics up by a factor of N, because we don't do an evaluation at every time step,
but only one every N steps, and so we skip all the intermediate evaluations.
Now, N can go up to 30, and so you can see on the plot on the right that you can get huge speedups,
even over non-conservative forces. So, if this is the plot that you saw before,
for equivariant models, non-equivariant models, non-conservative models, FlashMD beats everything
and by a lot, of course, because it's skipping all the time steps in between, and it can give
you a speedup of up to 30x. Now, we use the structures from the math data set, and we run
with an R2Scan MLIP, so Machine Learning Interatomic Potentials, and using molecular
dynamics trajectories sampled in this way, we trained a universal FlashMD model that you can
also use. And we have many different time steps. Here, I'm showing you a FlashMD model with a
time step of 32 femtoseconds, but actually, we have one even up to 128 femtoseconds. And you
can see that the simulation is pretty much consistent, at least qualitatively, with what
you can get from a traditional Machine Learning Interatomic Potential. It's just, in this case,
32 times faster. And I can guarantee you that it's very satisfying to run, especially because
you can do, like, one nanosecond in 40 minutes. It's really impressive, because the speedup is
crazy, and you can try it on GitHub at this address. Perfect. So, what kind of results
were we able to get with FlashMD? Very quickly, this is a torsional spectrum of alanine dipeptide
in water. So, FlashMD, with molecular dynamics, you can go up to 0.5 femtoseconds, more or less,
in this system. And FlashMD was able to give us the correct torsional spectrum up to 16 femtoseconds,
for example. It was able, for example, on the right here, probably the most impressive result
here, to qualitatively get a super ionic transition in lithium thiophosphate. So,
this type of system. So, very impressive. Overall, this is an RDF, a radio distribution function of
water, oxygen, oxygen on the left, hydrogen, hydrogen on the right. Again, pretty much on the
spot. Now, densities in entity simulations suffer a bit, but with the new models, they're better
than this. And something that is still a bit shaky when using FlashMD is dynamical properties. So,
here we have diffusion in water. So, on the x-axis, we have time, and on the y-axis, we have the mean
square deviation of water molecules, actually of the oxygen atoms. And you can see that although
most FlashMD models, especially those that predict a smaller time step, are able to get
the answer more or less right, so the mean square deviation more or less right, there are a few
models, especially 8 and 15, sorry, 8 and 16 femtoseconds that are struggling a bit, and that
are off by a factor of 2. But a factor of 2 is nothing compared to, for example, coarse-grained
models that can get it off by, for example, two orders of magnitude and achieve more or less the
same speed. So, big speedup offered by FlashMD with potentially some problems on dynamical
properties. So, these are solved by FlashMD. I will just rush a bit through this slide. It's very,
the way to make molecular dynamics symplectic and energy-conserving and so physically sound
is a bit complicated, but eventually you get something that can conserve energy exactly
and can recover exactly the same observables. So, here you see the same observables. So,
the mean square deviation displacement, same thing that FlashMD was struggling with.
And here FlashMD is called direct, so it's just the direct model for molecular dynamics.
And you see that the baseline MD predicts something for a certain mean square displacement,
and the direct molecular dynamics model is off by a factor of more or less 2, as we saw before,
even though that's not always the case. And the symplectic MD model that is an iterative model
is able, as the number of iterations is increased, to recover exactly the same
answer as the baseline molecular dynamics run. And with this, I would like to conclude the talk.
So, just to conclude, breaking symmetries is possible. That can give you up to 3x speedup.
Breaking energy conservation is also possible, even though one needs to be careful to run
molecular dynamics in a certain way with a certain algorithm. That also gives you a 3x speedup over
machine learning interatomic potentials. We published the PetMath universal potential with
all these features, and you can find it at this GitHub link. And then, in the second part of the
presentation, we talked to you about the direct prediction of molecular dynamics with FlashMD
that can give you up to a 30x speedup. We have published FlashMD universal models that
predict molecular dynamics at an R2 scan level of theory, so a very good DFT. These are available
at this link. And finally, SymplecticMD, although it's more expensive, it can be used as a correction
because it enforces the mathematical structure of molecular dynamics on the predictions. And here is
a link if by any chance you're interested in training your own models. We have a trainer
CLI interface that can train very, very arbitrary models with all types of symmetries and
constraints, per atom, not per atom, all types of per-pair targets, really anything pretty much
you can imagine, and you can find it at this GitHub link. And I would thank everyone for listening.
