chaste / chaste-docker Goto Github PK

Docker images for Chaste

• TL;DR
• Introduction
• Getting started
  • Prerequisites
  • [Recommended] Using a pre-built image
  • [Alternative] Use the VS Code devcontainer
• Container directory structure
• Sharing data between the host and container
  • Bind mounts
  • Copying data in and out
• Developing code within the container
• Installing additional software
• Testing
• Advanced usage
  • Building your own image
  • Writing your own Dockerfile
• Citation
• Troubleshooting

1. Install Docker and allocate it at least 4GB RAM
2. docker run -it --init --rm -v chaste_data:/home/chaste chaste/release
3. This is your terminal on Chaste: chaste@301291afbedf:~$ GL HF! ;)

Docker is a lightweight virtualisation technology allowing applications with all of their dependencies to be quickly and easily run in a platform-independent manner. This project provides an image containing Chaste (and some additional scripts for convenience) which can be launched with a single command, to provide a portable, homogeneous computational environment (across several operating systems and countless hardware configurations) for the simulation of cancer, heart and soft tissue.

Docker lets you build and run a computational environment from a plaintext Dockerfile. This is analogous to compiling an executable file from source code (equivalent to using docker build to produce an image from a Dockerfile) and then executing it as a running program (akin to using docker run to run a container). The steps of this analogy are illustrated in the figure below from Nüst et al. 2020.

Docker container analogy

More generally, Docker also has an image registry which stores prebuilt images: https://hub.docker.com/. Users may upload images from their own computer (with docker push) and download those from others (e.g. with docker pull) including official dockerised applications (e.g. Python and WordPress) as well as base images (e.g. Ubuntu and Alpine) to build upon for creating your own images. The Docker architecture and wider ecosystem are illustrated here.

Some slides from a workshop introducing Docker and how to use this Chaste image can be found here.

Install Docker and configure it to have at least 4GB of RAM and as many cores as you have (more than four cores will need more RAM).

⚠️ Allocate at least 4GB of RAM to Docker or compilation will fail with strange errors!

1. If you want to get up and running with the latest release fully compiled and ready to go, after installing and configuring Docker simply run:

   docker run --init -it --rm -v chaste_data:/home/chaste chaste/release
   

   If needed, you can also specify an available tag in the image name in the form chaste/release:<tag> to pull a particular release (e.g. chaste/release:2024.1) rather than defaulting to the latest version.
2. Alternatively, if you want to use the latest development code from the develop branch, use this command to pull and run the latest chaste/develop image instead:

   docker run --init -it --rm -v chaste_data:/home/chaste chaste/develop
   

Once the container has successfully launched, you should see a command prompt similar to this:

chaste@301291afbedf:~$

This is a bash prompt within an isolated Docker container (based on ubuntu) with all the dependencies and pre-compiled code you need to start building your own Chaste projects. In here you can build and test your projects without interfering with the rest of your system.

ℹ️ To see system resource usage for your running containers, open another terminal and run docker stats.

If you don't already have a project, just use the provided script new_project.sh to create a project template in ~/projects as a starting point. Many tutorials for projects can be found here: https://chaste.cs.ox.ac.uk/trac/wiki/UserTutorials.

Once you have a project ready to build, use the script build_project.sh <TestMyProject> c (replacing <TestMyProject> with the name of your project) and you will find the output in ~/output (the c argument is only necessary when new files are created).

ℹ️ To easily share data between the Docker container and the host e.g. the output directory, a bind-mount argument can be added to the command: -v /host/path/to/output:/home/chaste/output. See the instructions on bind mounts for further details.

When you are finished with the container, simply type exit or press Ctrl+D to close it (if necessary, pressing Ctrl+C first to stop any running processes). Any changes made in /home/chaste will persist when you relaunch a container, however if the container is deleted, everything else (e.g. installed packages, changes to system files) will be reset to how it was when the image was first used.

If you use VS Code and have installed Docker, you can simply clone the Chaste code repository and open it in VS Code (installing the Remote Development extension pack if prompted to do so). Finally, when prompted by the extension, click Reopen in Container. This will seamlessly pull, run and mount the latest chaste/develop image for you.

ℹ️ Note, this will mount the locally cloned copy of the Chaste code into the container, overlaying the copy already included in the image. While the pre-compiled binaries are built against the image's internal copy of the code, they will be relatively up-to-date, so will not take long to recompile against changes you make to the locally cloned code, bringing them back into synchrony.

Further details of the devcontainer can be found here.

For more advanced use cases, see Building your own image below.

Once launched, the container will start in the chaste user's home directory at /home/chaste with the following structure:

.
|-- build
|-- projects -> /home/chaste/src/projects
|-- scripts
|-- src
`-- output

These folders contain the following types of data:

• build: precompiled Chaste binaries and libraries
• projects: a symlink to /home/chaste/src/projects for user projects
• scripts: convenience scripts for creating, building and testing projects
• src: the Chaste source code
• output: the output folder for the project testing framework (set with $CHASTE_TEST_OUTPUT)

Corresponding environment variables are also set as follows:

• CHASTE_DIR="/home/chaste"
• CHASTE_BUILD_DIR="${CHASTE_DIR}/build"
• CHASTE_PROJECTS_DIR="${CHASTE_DIR}/src/projects"
• CHASTE_SOURCE_DIR="${CHASTE_DIR}/src"
• CHASTE_TEST_OUTPUT="${CHASTE_DIR}/output"

ℹ️ If building your own image, the CHASTE_DIR path can be changed at buildtime with a build argument e.g. --build-arg CHASTE_DIR=/path/to/alternative which will then set the other directories relative to that path.

Any changes made in the home folder (/home/chaste) will persist between restarting containers as it is designated as a VOLUME. Additionally, specific folders may be mounted over any of these subfolders, for example, to gain access to the test outputs for visualising in ParaView or for mounting a different version of the Chaste source code. In general, data should be left in a (named) volume, as file I/O performance will be best that way. However, bind mounting host directories can be convenient e.g. for access to output files and so is explained next.

⚠️ Docker containers are ephemeral by design and no changes will be saved after exiting (except to files in volumes or folders bind-mounted from the host). The contents of the container's home directory (including the Chaste source code and binaries) are stored in a Docker VOLUME and so will persist between container instances. However if you reset Docker, all volumes and their contained data will be lost, so be sure to regularly push your projects to a remote git repository!

This image is set up to store the Chaste source code, compiled libraries and scripts in a Docker volume as this is the recommended mechanism for data persistence and yields the best File I/O performance across multiple platforms.

One drawback of this type of mount is that the contents are more difficult to access from the host. However, to gain direct access to e.g. the output of the container from the host, or share datasets on the host with the container, a bind mount can be used (even overlaying a directory within the volume if needed).

For further details and illustrations of the Docker mount options see the storage documentation.

Any host directory (specified with an absolute path e.g. /path/to/output) may be mounted in the container e.g. the output directory. Alternatively, navigate to the folder on the host which contains these directories e.g. C:\Users\$USERNAME\chaste (Windows) or ~/chaste (Linux/macOS) and use $(pwd)/output instead as shown below. In the following examples, the image name (final argument) is assumed to be chaste/release rather than e.g. chaste/develop or chaste/release:2024.1 for simplicity.

docker run -it --init --rm -v chaste_data:/home/chaste -v "${PWD}"/output:/home/chaste/output chaste/release

On macOS and Windows (but not Linux), reading and writing files in bind mounts from the host have a greater overhead than for files in Docker volumes. This may slow down simulations where there is a lot of File I/O in those folders (e.g. output), so bind mounts should be used sparingly in such scenarios. A faster alternative would be to leave the files in a volume and use docker cp to copy them out at the end of the simulation (or copy modified files back in).

For example, use the following commands to copy the whole src folder, where the container has been labelled chaste e.g. with a command beginning: docker run --name chaste ...:

docker cp chaste:/home/chaste/src .  # copy out
# Make changes to the source files here
docker cp src/. chaste:/home/chaste/src  # copy in

We recommend using VS Code with the "Remote Development" extension which allows the files within a container to be directly accessed, edited and searched as if they were on the host system while preserving the performance benefits of keeping the files within the volume.

ℹ️ These steps relate to the currently recommended pre-built image method. If you are using the new devcontainer instructions, these steps are done automatically.

1. Start the container from a terminal with the command given
2. In VS Code select "Remote-Containers: Attach to Running Container..."
3. Choose the chaste-docker container (which will have a random name unless you launch it by adding --name <name> to the run command)
4. Open the folder /home/chaste with VS Code's built-in file browser and you will be able to access the files and directories described above.

ℹ️ For small edits to the code from the terminal, nano is installed in the image for convenience, along with git for pushing the changes.

If you want to use a package which is not installed within the image, you can install it with the command:

sudo apt-get update && sudo apt-get install <PackageName>

Replacing <PackageName> as appropriate. Enter the password: chaste when prompted to do so.

Note that packages installed this way will not persist after the container is deleted (because the relevant files are not stored in /home/chaste). This can be avoided by omitting the --rm flag from the docker run command and using docker start <container_name> to relaunch a previously used container. If there is a package you think would be a particularly useful permanent addition to the Docker image, then email your suggestion to me or submit a pull request.

To check Chaste compiled correctly you may wish to run the continuous test pack from the CHASTE_BUILD_DIR directory:

ctest -j$(nproc) -L Continuous

The script test.sh (in /home/chaste/scripts) is provided in the users's path for convenience.

The following test can be run separately to quickly check the build environment and installed dependencies available to chaste:

ctest --verbose -R TestChasteBuildInfo$

For more information on testing see: https://chaste.cs.ox.ac.uk/trac/wiki/ChasteGuides/CmakeBuildGuide.

If you're a more advanced developer and want to build your own image with a particular code branch, make sure you have Docker up and running then read on! In these examples, we tag the image chaste:custom for illustration but you are encouraged to give it a more descriptive name.

1. Build the Chaste image:

   1. From the latest commit on Chaste's GitHub develop branch:

      docker build -t chaste:custom --build-arg GIT_TAG=develop https://github.com/chaste/chaste-docker.git
      

   2. Alternatively a specific branch or tag may be specified through the argument --build-arg GIT_TAG=<branch|tag> (with the same tag appended onto the docker image name for clarity) e.g.:

      docker build -t chaste:custom --build-arg GIT_TAG=2024.1 https://github.com/chaste/chaste-docker.git
      

   3. Finally, if you want a bare container ready for you to clone and compile your own Chaste code, pull a base image with docker pull chaste/base (specifying an available Ubuntu distribution if desired e.g. chaste/base:focal) Alternatively, build a fresh image by running the following command (omitting the --build-arg GIT_TAG=<branch|tag> argument above, or explicitly passing --build-arg GIT_TAG=-, which will skip compiling Chaste within the image):

      docker build -t chaste:custom https://github.com/chaste/chaste-docker.git
      

      (When the container is running you may then edit build_chaste.sh in the scripts directory to configure the process with your own options before executing it.)

2. Launch the container:

   docker run --init -it --rm -v chaste_data:/home/chaste chaste:custom
   

   The first time will take a little longer than usual as the volume has to be populated with data. For information on accessing the contents of this volume, see the section on sharing data.

For more advanced use cases, you can also include your own software, scripts and configuration by writing your own Dockerfile. To inherit the base configuration with the necessary dependencies and configuration for Chaste already set up, begin your Dockerfile with:

FROM chaste/base

or e.g. chaste/base:focal to specify a particular base image other than the latest.

A full guide to writing a Dockerfile is beyond the scope of this project, however for more information, see the Docker documentation and reference. There is also a handy list of Ten Simple Rules to help you get started!

ℹ️ Pro tip! To write your own Dockerfiles, see Nüst et al. 2020 for best practices.

If you found this work helpful, please cite the following publication.

Cooper et al., (2020). Chaste: Cancer, Heart and Soft Tissue Environment. Journal of Open Source Software, 5(47), 1848. https://doi.org/10.21105/joss.01848

@article{Chaste_2020,
    title = {Chaste: Cancer, Heart and Soft Tissue Environment},
    journal = {Journal of Open Source Software}
    publisher = {The Open Journal},
    year = {2020},
    month = {3},
    volume = {5},
    number = {47},
    pages = {1848},
    author = {Fergus R. Cooper and Ruth E. Baker and Miguel O. Bernabeu and Rafel Bordas and Louise Bowler and Alfonso Bueno-Orovio and Helen M. Byrne and Valentina Carapella and Louie Cardone-Noott and Jonathan Cooper and Sara Dutta and Benjamin D. Evans and Alexander G. Fletcher and James A. Grogan and Wenxian Guo and Daniel G. Harvey and Maurice Hendrix and David Kay and Jochen Kursawe and Philip K. Maini and Beth McMillan and Gary R. Mirams and James M. Osborne and Pras Pathmanathan and Joe M. Pitt-Francis and Martin Robinson and Blanca Rodriguez and Raymond J. Spiteri and David J. Gavaghan},
    doi = {10.21105/joss.01848},
    url = {https://doi.org/10.21105/joss.01848},
}

Additionally, if you found the "Ten Simple Rules ..." paper helpful for general advice or writing your own Dockerfile, please consider citing that too.

Nüst D, Sochat V, Marwick B, Eglen SJ, Head T, Hirst T, and Evans, BD. (2020) Ten simple rules for writing Dockerfiles for reproducible data science. PLoS Comput Biol 16(11): e1008316. https://doi.org/10.1371/journal.pcbi.1008316

@article{TSR_Dockerfiles_2020,
    title = {Ten Simple Rules for Writing Dockerfiles for Reproducible Data Science},
    journal = {PLOS Computational Biology},
    publisher = {Public Library of Science},
    year = {2020},
    month = {11},
    volume = {16},
    number = {11},
    pages = {1--24},
    author = {Daniel N{\"u}st and Vanessa Sochat and Ben Marwick and Stephen J. Eglen and Tim Head and Tony Hirst and Benjamin D. Evans},
    doi = {10.1371/journal.pcbi.1008316},
    url = {https://doi.org/10.1371/journal.pcbi.1008316},
}

• Firstly, make sure you have given Docker at least 4GB RAM, especially if you are compiling Chaste from source.

• If you get a message beginning: Unexpected end of /proc/mounts line ..., this can be safely ignored!

• If you ran a container before and explicitly gave it a name (e.g. using --name chaste as an argument to docker run) but it now refuses to launch with an error message like below, it's because you need to remove the existing (stopped) container before one can be recreated with the same name.

  docker: Error response from daemon: Conflict. The container name "/chaste" is already in use by container "1711bce2674e399b6084c6d452857377f6ed4dd8ee3aa19460de00fac7b86bc7". You have to remove (or rename) that container to be able to reuse that name.
  

  To remove the container, simply run the following command then rerun the docker run ... command to launch the container (N.B. This will not delete the data stored in the chaste_data volume but other changes made within the container will be lost e.g. installed software):

  docker rm chaste
  

  N.B. You can find out the names of existing containers (and their status) with the command: docker ps -a.

• If building the image from scratch, occasionally problems can occur if a dependency fails to download and install correctly. If such an issue occurs, try resetting your Docker environment (i.e. remove all containers, images and their intermediate layers) with the following command:

  docker system prune -a
  

  This will give you a clean slate from which to restart the building process described above.

• If you have deleted or otherwise corrupted the persistent data in the chaste_data volume, the command can be used with the --volumes flag. ⚠️ Warning! ⚠️ this will completely reset any changes to data in the image home directory along with any other Docker images on your system (except where other host folders have been bind-mounted). Commit and push any changes made to the Chaste source code or projects and save any important test outputs before running the command with this flag. If you are unsure, do not use this flag - instead list the volumes on your system with docker volume ls and then use the following command to delete a specific volume once you are happy that no important data remains within it:

  docker volume rm <volume_name>
  

  For more information on cleaning up Docker, see this tutorial.

• For more general troubleshooting, opening a terminal and running docker events then launching the container in another terminal will provide logging information of the events happening behind the scenes.