As of March 4, 2018, this project is no longer actively maintained. The Docker for Mac app
has made significant improvements in terms of mounted volume performance and file watching, so docker-osx-dev is no longer necessary.
Docker and Boot2Docker are awesome for running containers on OS X, but if you try to use them to do iterative development by mounting a source folder from OS X into your Docker container, you will run into two major problems:
- Mounted volumes on VirtualBox use vboxsf, which is extremely slow, so compilation and startup times for code in mounted folders is 10-20x slower.
- File watching is broken since vboxsf does not trigger the inotify file watching mechanism. The only workaround is to enable polling, which is much slower to pick up changes and eats up a lot of resources.
I tried many different solutions (see Alternatives) that didn't work until I finally stumbled across one that does: rsync. With rsync, build and compilation performance in mounted folders is on par with native OS X performance and standard file watching mechanisms work properly too. However, setting it up correctly is a painful process that involves many steps, so to make life easier, I've packaged this process up in this docker-osx-dev project.
For more info, check out the blog post A productive development environment with Docker on OS X.
Beta. A number of developers are successfully using and contributing to docker-osx-dev. It still has some rough edges, but it works well, and makes the docker experience on OS X much better. Give it a try, share your feedback, and submit some pull requests!
Note: this project is inherently a temporary workaround. I hope that in the future, someone will build a better alternative to vboxsf for mounting source code from OS X, and thereby make this entire project obsolete. Until that day comes, I will continue to use the docker-osx-dev scripts to keep myself productive.
Prerequisite: HomeBrew must be installed.
The docker-osx-dev script has an install command that can setup your entire
Docker development environment on OS X, including installing Docker and
Boot2Docker:
curl -o /usr/local/bin/docker-osx-dev https://raw.githubusercontent.com/brikis98/docker-osx-dev/master/src/docker-osx-dev
chmod +x /usr/local/bin/docker-osx-dev
docker-osx-dev installFour notes about the install command:
- It is idempotent, so if you have some of the dependencies installed already, it will not overwrite them.
- When the install completes, it prints out instructions for one
sourcecommand you have to run to pick up important environment variables in your current shell, so make sure not to skip that step! - Once the install completes, you can use the
docker-osx-devscript to sync files, as described in the next section. - It assumes the user you want to use for development can also run homebrew
(eg. write to
/usr/local). If it doesn't, you need to split the installation in 2 parts: one run asadmin(the name of the user who can run homebrew), and one as yourself:
su admin
docker-osx-dev install --only-dependencies
exit
docker-osx-dev install --skip-dependenciesThe install command will install, configure, and run Boot2Docker on your
system, so the only thing left to do is to run the docker-osx-dev script and
tell it what folders to sync. If you run it with no arguments, it will sync the
current folder to the Boot2Docker VM:
> cd /foo/bar
> docker-osx-dev
[INFO] Performing initial sync of paths: /foo/bar
[INFO] Watching: /foo/bar
Alternatively, you can use the -s flag to specify what folders to sync
(run docker-osx-dev -h to see all supported options):
> docker-osx-dev -s /foo/bar
[INFO] Performing initial sync of paths: /foo/bar
[INFO] Watching: /foo/bar
Now, in a separate tab, you can run a Docker container and mount the current
folder in it using the -v parameter. For example, here is how you can fire up
the tiny Alpine Linux image
and get a Linux console in seconds:
> cd /foo/bar
> docker run -v $(pwd):/src -it --rm gliderlabs/alpine:3.1 sh
/ # cd /src
/ # echo "I'm in a $(uname) container and my OS X files are being synced to $(pwd)!"
I'm in a Linux container and my OS X files are being synced to /src!
As you make changes to the files in the /foo/bar folder on OS X, using the
text editors, IDEs, and tools you're used to, they will be automatically
synced to the /src folder in the Docker image. Moreover, file watchers should
work normally in the Docker container for any framework that supports hot
reload (e.g. Grunt, SBT, Jekyll) without any need for polling, so you should be
able to follow a "make a change and refresh the page" development model.
If you are using Docker Compose,
docker-osx-dev will automatically sync any folders marked as
volumes in docker-compose.yml.
For example, let's say you had the following docker-compose.yml file:
web:
image: training/webapp
volumes:
- /foo:/src
ports:
- "5000:5000"
db:
image: postgres First, run docker-osx-dev:
> docker-osx-dev
[INFO] Using sync paths from Docker Compose file at docker-compose.yml
[INFO] Performing initial sync of paths: /foo
[INFO] Watching: /foo
Notice how it automatically found /foo in the docker-compose.yml file.
Now you can start your Docker containers:
docker-compose upThis will fire up a Postgres
database and the training
webapp (a simple "Hello,
World" Python app), mount the /foo folder into /src in the webapp container,
and expose port 5000. You can now test this webapp by going to:
http://dockerhost:5000
When you install docker-osx-dev, it adds an entry to your /etc/hosts file so
that http://dockerhost works as a URL for testing your Docker containers.
docker-machine support is experimental. You can use it as the way it is used for
boot2docker, but run docker-machine env before. So as an example, run as:
> docker-machine create --driver virtualbox <machine-name>
> eval "$(docker-machine env <machine-name>)"
> docker-osx-dev install
> cd /foo/bar
> docker-osx-dev
[INFO] Performing initial sync of paths: /foo/bar
[INFO] Watching: /foo/bar
In this case, docker-osx-dev will use the machine defined in the DOCKER_MACHINE_NAME env var,
defined by docker-machine env. Alternatively, use the --machine-name <machine-name> argument.
Note: when running docker-osx-dev for boot2docker, please make sure the env var DOCKER_MACHINE_NAME
is not defined.
The install command installs all the software you need:
- Docker
- Boot2Docker
- Docker Compose
- VirtualBox
- fswatch
- The
docker-osx-devscript which you can use to start/stop file syncing
The install command also:
- Adds the Docker environment variables to your environment file (e.g.
~/.bash_profile) so it is available at startup. - Adds an entry to
/etc/hostsso thathttp://dockerhostworks as a valid URL for your docker container for easy testing.
Instead of using VirtualBox shared folders and vboxsf, docker-osx-dev keeps
files in sync by using fswatch to
watch for changes and rsync to quickly
sync the files to the Boot2Docker VM. By default, the current source folder
(i.e. the one you're in when you run docker-osx-dev) is synced. If you use
docker-compose, docker-osx-dev will sync any folders marked as
volumes. Run docker-osx-dev -h
to see all the other options supported.
File syncing is currently one way only. That is, changes you make on OS X will be visible very quickly in the Docker container. However, changes in the Docker container will not be propagated back to OS X. This isn't a problem for most development scenarios, but time permitting, I'll be looking into using Unison to support two-way sync. The biggest limitation at the moment is getting a build of Unison that will run on the Boot2Docker VM.
Contributions are very welcome via pull request. This project is in a very early alpha stage and it needs a lot of work. Take a look at the issues for known bugs and enhancements, especially the ones marked with the help wanted tag.
To run the local version of the code, just clone the repo and run your local
copy of docker-osx-dev:
> git clone https://github.com/brikis98/docker-osx-dev.git
> cd docker-osx-dev
> ./src/docker-osx-dev
To run the unit tests, install bats
(brew install bats) and run the corresponding files in the test folder:
> ./test/docker-osx-dev.bats
✓ index_of doesn't find match in empty array
✓ index_of finds match in 1 item array
✓ index_of doesn't find match in 1 item array
✓ index_of finds match in 3 item array
[...]
51 tests, 0 failures
I started to create integration tests for this project in
test/integration-test.sh, but I hit a wall. The point of the integration test
would be to run Boot2Docker in a VM, but most CI providers (e.g. TravisCI and
CircleCI) already run your build in their own VM, so this would require running
a VM-in-a-VM. As described in #7,
I can't find any way to make this work. If anyone has any ideas, please take a
look!
Below are some of the other solutions I tried to make Docker productive on OS X (I even created a StackOverflow Discussion to find out what other people were doing.) With most of them, file syncing was still too slow to be usable, but they were useful to me to learn more about the Docker ecosystem, and perhaps they will be useful for you if docker-osx-dev doesn't work out:
- boot2docker-vagrant: Docker, Vagrant, and the ability to choose between NFS, Samba, rsync, and vboxsf for file syncing. A lot of the work in this project inspired docker-osx-dev.
- dinghy: Docker + Vagrant + NFS. I found NFS was 2-3x slower than running builds locally, which was much faster than the 10-20x slowness of vboxsf, but still too slow to be usable.
- docker-unison: Docker + Unison. The Unison File Synchronizer should be almost as fast as rsync, but I ran into strange connection errors when I tried to use it with Docker.
- Polling in Jekyll and Polling in SBT/Play. Some of the file syncing solutions, such as vboxsf and NFS, don't work correctly with file watchers that rely on inotify, so these are a couple examples of how to switch from file watching to polling. Unfortunately, this eats up a fair amount of resources and responds to file changes slower, especially as the project gets larger.
- Hodor. Uses the Unison File Synchronizer to sync files. I have not had a chance to try this project out yet.
- docker-machine-nfs: Activates NFS for an existing machine.
This code is released under the MIT License. See LICENSE.txt.
- 06/05/15: merged the
setup.shanddocker-osx-devscripts together since they share a lot of the same code and bash scripts don't have any easy ways to define modules, download dependencies, etc. - 05/25/15: Second version released. Removes Vagrant dependency and uses just
rsync + Boot2Docker. If you had installed the first version, you should
delete your
Vagrantfile, delete the old version of/usr/local/bin/docker-osx-dev, and re-run thesetup.shscript. - 05/19/15: Initial version released. Uses Vagrant + rsync + Boot2Docker.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent