federatedcloud / nixtemplates Goto Github PK
View Code? Open in Web Editor NEWSeveral basic templates that are meant to be used as the basis for other Singularity Spec files using nixpkgs
License: BSD 3-Clause "New" or "Revised" License
Several basic templates that are meant to be used as the basis for other Singularity Spec files using nixpkgs
License: BSD 3-Clause "New" or "Revised" License
There are numerous performance improvements that this could entail:
Of course, there are some places where this won't work:
https://github.com/federatedcloud/NixTemplates/blob/develop/Dockerfile-OpenMPI#L1
To maintain similar functionality, need to switch to some sort of text templating system that can generate our Dockerfiles for us.
Could use Data.TemplateString for a typed and relatively light alternative relying on NodeJS, along with something like Snail for scripting, but see these comments
Not exactly an issue for this repo, but here it is. Closest I've gotten is to try to use the host singularity, owned by root:
singularity run --contain -B /tmp/shubtmp:/tmp -B /usr/local:/host_local --overlay nix-overlay.img shub://federatedcloud/NixTemplates:nix_alpine_openmpi_6796a60398bb890002e7010593c14cf3731613a1
Then I try to do:
/host_local/bin/singularity run shub://singularityhub/hello-world
But need to use a system that has /bin/bash
installed ... or a singularity image where the user can do sudo
. To be continued.
This could be useful for allowing MPI installed int he container to be run "on the host" by making the mpirun user's default shell the container housing MPI (which would need to be configured by the VM-provisioning software of choice).
Already built nix_alpine_base per README, cannot build nix_alpine_openmpi
:# source Docker/OpenMPI/build.sh
NIX_OMPI_IMAGE is nix_alpine_openmpi:53bf75e88ccbd0b9719d2c63ffea944e344e1bcb
Sending build context to Docker daemon 146.9kB
Step 1/35 : ARG BASEOS
--->
Step 2/35 : FROM nix_${BASEOS}_base:11a12ceb5c9cdedd3816acc783d193eeb1fe7fa6
pull access denied for nix_alpine_base, repository does not exist or may require 'docker login'
Unable to find image 'nix_alpine_openmpi:53bf75e88ccbd0b9719d2c63ffea944e344e1bcb' locally
Error response from daemon: pull access denied for nix_alpine_openmpi, repository does not exist or may require 'docker login'
Error response from daemon: No such container: nix_alpine_openmpi_53bf75e88ccbd0b9719d2c63ffea944e344e1bcb_TEST
Error response from daemon: No such container: nix_alpine_openmpi_53bf75e88ccbd0b9719d2c63ffea944e344e1bcb_TEST
Error response from daemon: No such container: nix_alpine_openmpi_53bf75e88ccbd0b9719d2c63ffea944e344e1bcb_TEST
:# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
nix_alpine_base 53bf75e 5f19395e57df 9 minutes ago 279MB
alpine 3.7 3fd9065eaf02 5 months ago 4.15MB
jupyter/all-spark-notebook latest bbfb8aad625b 8 months ago 5.24GB
cat /etc/issue
Ubuntu 16.04.4 LTS \n \l
docker --version
Docker version 17.09.0-ce, build afdb6d4
git log | head -n 1
commit 53bf75e
As done here
I do not recall where this came from or what it does.
This will be somewhat meta, as ideally we should be able to just build these images on any system with docker and/or singularity, but we're starting to see some dependencies in the build process:
Here is a list (update as necessary):
spython
(from singularity-cli) - used occasionally to generate Singularity recipes from Dockerfiles. Probably not necessary to include, but might as well if we build this.envsubst
) - used while Singularity doesn't support environment variable expansion.This would need to be a non-interactive script used by non-base images (e.g. the OpenMPI image).
Even though we are doing dev work in 'develop', if we merger in commits to 'master', a singularity image will be built for each commit in 'master', with some intelligent caveats:
If a user has a repeat tag, then the file with more recent changes (timestamp) is built. Only Singularity files that have been changed since the last commit will be built with Automatic Building in this fashion. If you want to go back int time to a particular commit, you should use a Manual Build. This means that if your push to Github has 10 changed Singularity recipe files, all 10 will be added to the queue to build, and your collection is allowed one active build at a time.
This is probably reasonable, but we should discuss.
Currently .dockerignore
ignores *.img
and *.simg
, however, most of these still seem to get picked up, as indicated by the size of the build context and the tool docker-show-context
. Not sure what is going wrong.
Specifically all Singularity recipes (and by extension / call for simplicity, Dockerfiles) should be moved to the repo root. See singularityhub/singularityhub.github.io#140 for context.
The build part can be accomplished by copying the repo dir to /tmp/<randomId>
, checkout out appropriate commit, then building the base image.
As discussed at https://stackoverflow.com/a/50747830/3096687 and pure-eval mode: nix-shell --pure --pure-eval ...
with the nix expression embedded in a simple shell script should work.
nix-shell -p <pkg_attribute_name> -I /path/to/nixpkgs
and nix-env?
Notes from output:
/OpenMPI$ ~/spython recipe Dockerfile > SingTemplate
WARNING ARG is not supported for Singularity! To get BASEOS
WARNING in the container, on host export SINGULARITY_BASEOS
WARNING ARG is not supported for Singularity! To get BASEDIR
WARNING in the container, on host export SINGULARITY_BASEDIR
WARNING ARG is not supported for Singularity! To get ADDUSER
WARNING in the container, on host export SINGULARITY_ADDUSER
WARNING $BASEDIR/config.nix doesn't exist, ensure exists for build
WARNING $HOME/.config/nixpkgs/ doesn't exist, ensure exists for build
WARNING $BASEDIR/dev-env.nix doesn't exist, ensure exists for build
WARNING $ENVSDIR/ doesn't exist, ensure exists for build
WARNING Utils/persist-env.sh doesn't exist, ensure exists for build
WARNING $ENVSDIR/ doesn't exist, ensure exists for build
WARNING $BASEDIR/ssh/config doesn't exist, ensure exists for build
WARNING ${SSHDIR}/config doesn't exist, ensure exists for build
WARNING $BASEDIR/ssh/id_rsa.mpi doesn't exist, ensure exists for build
WARNING ${SSHDIR}/id_rsa doesn't exist, ensure exists for build
WARNING $BASEDIR/ssh/id_rsa.mpi.pub doesn't exist, ensure exists for build
WARNING ${SSHDIR}/id_rsa.pub doesn't exist, ensure exists for build
WARNING $BASEDIR/ssh/id_rsa.mpi.pub doesn't exist, ensure exists for build
WARNING ${SSHDIR}/authorized_keys doesn't exist, ensure exists for build
WARNING $BASEDIR/default-mca-params.conf doesn't exist, ensure exists for build
WARNING ${HOME}/.openmpi/mca-params.conf doesn't exist, ensure exists for build
WARNING $BASEDIR/mpi4py_benchmarks doesn't exist, ensure exists for build
WARNING ${HOME}/mpi4py_benchmarks doesn't exist, ensure exists for build
WARNING $BASEDIR/entrypoint* doesn't exist, ensure exists for build
WARNING $ENVSDIR/ doesn't exist, ensure exists for build
WARNING $BASEDIR/default.nix doesn't exist, ensure exists for build
WARNING $ENVSDIR/ doesn't exist, ensure exists for build
As done here
These files largely overlap, but need some extra logic depending on which build is being done.
Should be executable:
ls -last /.singularity.d/runscript*
0 -rwxr-xr-x 1 root root 489 Jul 30 19:47 /.singularity.d/runscript
1 -rw-rw-r-- 1 root root 732 Jul 30 19:37 /.singularity.d/runscript-nixbase
Since the version of MPI in the container should be <= the version on the host to be able to communicate (ref), we should pin versions, by default, and a sensible version supported by most major HPC centers should be chosen. I assume this will be 3.0 or even 3.1; for our own systems it should be easy to do.
In fact, for our own systems, we should first investigate just using mpirun
from singularity to launch a singularity container, so we can avoid having to do this pairing up at all on our systems.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.