Run singlepoint (parallel) | Video tutorial to single-point calculation wrapper script (3/5)

Philip Zhu
41 followers
9 Video Views·Feb 11, 2025
Video tutorial to smart single-point calculation wrapper script. Github:
https://github.com/PhilipZhu/singlepoint-energy-batch
Let me show you another example.
This time we run multiple calculations in parallel.
It's the same system, 6 frames.
And we run the same single point executable with input.xyz.
Since this time we are going to launch multiple jobs at once,
it will be a little bit messy if we dump all the output into the console,
so I will redirect the output into no output.
Everything else is the same. It's exactly the same as what we did before.
And I will run one of them, and then two of them, four of them.
So, what happens is that it created...
Oh, it's already done.
It's already done.
So now it's much faster than last time,
which last time in serial we have been waiting for maybe 20 seconds,
and this is probably less than 10 seconds.
So what it did is that it created...
The first job that gets run created these run folders,
and then it started running in one of these folders.
And the second job, third job, fourth job sees that the calculation folder already exists,
so they will not attempt to create a duplicate calculation folder,
because that might be in conflict with the existing one.
And then each of them is smart enough so that they go into a folder
and sees that this folder is being run by another process,
or it has finished, or it has not been finished but is currently being run,
so it will skip that and go to the next folder.
This folder has not been touched by any other processes,
so it will start running there.
So they are smart enough to automatically, you don't need to keep track of where they are,
they automatically find folders that need to be run until all jobs are finished.
So on a supercomputer node, you can either do the same thing, run this multiple times,
launch everything parallel in background,
or you can run these lines in different compute nodes.
So usually we submit a submission script, such as run parallel sh,
we submit a script like this, launch everything in background and parallel, and wait for them.
So you can submit it on the same node or on different nodes,
as long as different compute nodes all have access to the same file system,
which contains this run folder, this calculation folder.
As long as all of these jobs have access to the same folder, calculation folder,
they will be able to tell, when they look through these calculation folders,
they will be able to tell from the existing files that they see there is a file that may indicate
that some other processes might be running in this folder already, so it will skip it.
So different jobs can automatically find a folder that needs to be run until everything is done.
A tip for submitting jobs on supercomputers is this.
After you submit the first one, you can give it a few seconds of sleep, like 5 seconds.
In this case, I will use 1 second of sleep as an example.
The reason is that because the first job creates these input folders,
so it may be good of giving it a little bit of time to let them create some folders.
After the folders have been created, you can submit as many jobs in parallel without any delay as you want,
because they won't be trying to create new folders,
because it definitely sees that the calculation folder already exists,
and they won't try to create another folder that might be in conflict with the existing one.
There will not be synchronization and timing problems after that.
So after the first one, wait for a bit, and then you can submit as many as you want, and then wait.
So this is a good practice of submitting multiple jobs in parallel in a script.
Let's try to run this example.
Okay, it's done.
So this is the end of the single-point calculation examples using the default software, which is Orca.
So in order to use this default software, you can look at the readme file,
and you need to set up Orca on your machine in your account according to the readme instructions.
Alternatively, you don't have to use the default software.
You can just set up any other energy calculator software.
In the next example, I will show how to configure the wrapper script for your custom energy calculator software.
I try to make it modular so that for any new software, you don't need to change much.
The changes you need to make are minimal, as minimal as possible.
So I try to make it like that, and we will see that in the next example.
