Comments (4)
Hey, there is perhaps a cheap trick to do that: (ie. leave the "state" in a filesystem area)
create a filename in a lockfile in a working area "calculated" based on the produced module name,
eg. $LOCKFILE=$LOCKDIR/echo $MYWANNABEMODULENAME | tr '/' '_'
then do in the spawning code the equivalent of:
lockfile $LOCKFILE ## ensure we are alone
qsub "easybuild() ; rm -f $LOCKFILE" || rm -f $LOCKFILE
lockfile $LOCKFILE ## this ensures no progress until the finish of the submitted task
rm -f $LOCKFILE
Because only the first entrant will grab the lock, any other ones will just stay on 'halt' mode awaiting;
now as soon as the lock is released, the module has been (hopefully) produced and life goes on.
Caveat of proposed design: if the "common" task is failing for whatever reason, the module won't be
built and it will be retried as many times as asked.
ps. I had to write a solution to a related problem for a local user this year... easyqsub.py anyone? ;-)
ref: http://pypi.python.org/pypi/lockfile/
from easybuild-framework.
That's one way, but working with lock files on the shared filesystem we have is a pain in the ass.
We have an API for PBS built into EasyBuild, so I'd rather use that directly to figure out which jobs were submitted before.
That way, we can submit the same build on different clusters (each of which has a different install path), without having to use some crazy naming scheme for the lock files.
And like you said, if something goes wrong and lock files stay behind for some reason, you can run into strange issues.
from easybuild-framework.
Hi to all easybuilders,
so, I am back from (my regular end-of-month sailing race) vacations and now trying to catch up;
I have been reading this page in relation to parallel builds in the FreeBSD world
http://wiki.freebsd.org/SummerOfCode2012/Parallelization_in_the_ports_collection
and come to the conclusion that handling the dependencies of packages in the
parallel fashion is the really interesting stuff (think 2 WRF builds in parallel and such);
that unavoidably implies some form of queue/dispatcher process in the same lines as described here:
https://www.sara.nl/systems/lisa/software/disparm
That's one way, but working with lock files on the shared filesystem we have is a pain in the ass.
Indeed, also at the end of the disparm page this problem is described
and even the tested workaround based on directories creation is provided.
btw. yes, we have NFS for $HOMEs like many others, too :-)
We have an API for PBS built into EasyBuild, so I'd rather use that directly to figure out which jobs were submitted before.
In that case, PBS becomes the "store" of the state and the solution would be only working on it.
Can we find a more generic approach instead?
IMHO, this issue falls in a class of more generic problems, on how to submit jobs on systems with primitive facilities (and going in this direction, would allow for generality across implementations)
to be continued...
from easybuild-framework.
Nice feedback @fgeorgatos, we'll make sure to take this into account when we're picking up on this.
from easybuild-framework.
Related Issues (20)
- make "commit" an optional parameter and template
- GNU_SOURCE (https://ftpmirror.gnu.org/gnu/) downloads failing HOT 2
- Packaging with read-only install directory crashes with `Permission denied`
- make `--job` a bit smarter when uploading test reports HOT 2
- Create an option for a developer mode HOT 3
- Change default value for module-extensions to True
- Change default value for module-depends-on to True
- Feature request/proposal: maintainers/authors optional infos in easybuild files HOT 1
- Honor dependency order when (re)building multiple ECs HOT 4
- Checksum should be allowed to be a single dictionary at the top level scope only, even for bundles/extensions (where there are top level sources as well)
- More granular `eb` exit codes (return codes) HOT 3
- Clean up SLURM/Batchsystem environment before doing builds HOT 4
- PR title created with --new-pr misses 2nd toolchain HOT 2
- Accepting EULA in a container recipe HOT 1
- Download instructions should tell user exactly what files to put where HOT 2
- `eb` command fails outside of framework source folder HOT 4
- Auto-enable Fortran oneAPI compilers HOT 1
- Add option to ignore default config files
- refactor easyconfig parameters and template constants to use a consistent naming style HOT 9
- Add `sysroot` support to intel compilers
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from easybuild-framework.