brikis98 / docker-osx-dev Goto Github PK

Feature request: I think riddance from Brew would be a good move, many (old-fashioned) devs still don't believe in extending on top of the OS X utils and are using MacPorts instead. For reliability reasons.

Running latest version:

$ docker-osx-dev
2015-06-30 11:17:45 [INFO] Using default sync paths: .
2015-06-30 11:17:45 [INFO] Complete list of paths to sync: /Users/saul/projects/swoon
2015-06-30 11:17:45 [INFO] Using default exclude paths: .git
2015-06-30 11:17:45 [INFO] Complete list of paths to exclude: .git
2015-06-30 11:17:45 [INFO] Starting docker-osx-dev file syncing
2015-06-30 11:17:45 [INFO] Installing rsync in the Boot2Docker image
wget: not an http or ftp url: https://raw.githubusercontent.com/brikis98/docker-osx-dev/master/lib/rsync

I already use boot2docker, and if I run through the install, init, start steps for docker-osx-dev, it doesn't work.

(boot2docker is stopped)

➜  docker ps
FATA[0000] Get http:///var/run/docker.sock/v1.18/containers/json: dial unix /var/run/docker.sock: no such file or directory. Are you trying to connect to a TLS-enabled daemon without TLS?
➜   docker-compose up
Couldn't connect to Docker daemon - you might need to run `boot2docker up`.
➜  env | grep DOCKER
➜  

The project VM shows up in VirtualBox, but I can't do anything with docker.

Question: how do you create a HomeBrew recipe that has brew-cask dependencies?

would be cool to have a http-proxy inside the VM just like https://github.com/codekitchen/dinghy#dns installing the container isnt an issue, the local dnsmasq stuff is a bit more tricky. would love to see this feature :)

docker-osx-dev starts fine but:

$ docker ps
FATA[0000] An error occurred trying to connect: Get https://192.168.59.103:2376/v1.18/containers/json: x509: certificate is valid for 127.0.0.1, 10.0.2.15, not 192.168.59.103
$ docker version
Client version: 1.6.2
Client API version: 1.18
Go version (client): go1.4.2
Git commit (client): 7c8fca2
OS/Arch (client): darwin/amd64
FATA[0000] Get http:///var/run/docker.sock/v1.18/version: dial unix /var/run/docker.sock: no such file or directory. Are you trying to connect to a TLS-enabled daemon without TLS?
$ boot2docker status
running
$ boot2docker version
Boot2Docker-cli version: v1.6.2
Git commit: cb2c3bc
$ boot2docker ssh
                        ##         .
                  ## ## ##        ==
               ## ## ## ## ##    ===
           /"""""""""""""""""\___/ ===
      ~~~ {~~ ~~~~ ~~~ ~~~~ ~~~ ~ /  ===- ~~~
           \______ o           __/
             \    \         __/
              \____\_______/
 _                 _   ____     _            _
| |__   ___   ___ | |_|___ \ __| | ___   ___| | _____ _ __
| '_ \ / _ \ / _ \| __| __) / _` |/ _ \ / __| |/ / _ \ '__|
| |_) | (_) | (_) | |_ / __/ (_| | (_) | (__|   <  __/ |
|_.__/ \___/ \___/ \__|_____\__,_|\___/ \___|_|\_\___|_|
Boot2Docker version 1.7.0, build master : 7960f90 - Thu Jun 18 18:31:45 UTC 2015
Docker version 1.7.0, build 0baf609
docker@boot2docker:~$

Hi, how would you feel about a PR adding support for docker-machine? A few simple hacks made it work for me, and I'm pretty sure accommodating both options would be possible. If it's something you would like to see, I could try to come up with a reasonable PR.

This does not repro using the standard boot2docker config.

Repro:

1. Running containers via docker-compose. Local src folder is mounted into webapp container.
2. Nodemon is run via NODE_ENV=development nodemon ./bin/www
3. Change a file monitored by Nodemon. I tried both Sublime 2 and RubyMine and had the same result.

Results:

webapp_1 | 20 May 05:09:22 - [nodemon] restarting due to changes...
webapp_1 | 20 May 05:09:22 - [nodemon] starting `node ./bin/www`
webapp_1 | 20 May 05:09:23 - [nodemon] restarting due to changes...
webapp_1 | 20 May 05:09:23 - [nodemon] starting `node ./bin/www`
webapp_1 | 20 May 05:09:24 - [nodemon] restarting due to changes...
webapp_1 | 20 May 05:09:24 - [nodemon] starting `node ./bin/www`
webapp_1 | 20 May 05:09:25 - [nodemon] restarting due to changes...
webapp_1 | 20 May 05:09:25 - [nodemon] starting `node ./bin/www`
webapp_1 | 20 May 05:09:26 - [nodemon] restarting due to changes...
webapp_1 | 20 May 05:09:26 - [nodemon] starting `node ./bin/www`
webapp_1 | 20 May 05:09:27 - [nodemon] restarting due to changes...
webapp_1 | 20 May 05:09:27 - [nodemon] starting `node ./bin/www`
webapp_1 | 20 May 05:09:28 - [nodemon] restarting due to changes...
webapp_1 | 20 May 05:09:28 - [nodemon] starting `node ./bin/www`

Notice the timestamps. It's restarting multiple times for a single change.

When I run docker-osx-dev in a folder where there is a docker-compose.yml, it failed to resolve path using the tilde, such as ~/app/data.

It tries to resolve /Users/foobar/~/app/data instead of /Users/foobar/app/data.

Here is the error I got:

2015-06-08 11:21:56 [INFO] rsync: link_stat "/Users/foobar/app/~/var/www" failed: No such file or directory (2)
2015-06-08 11:21:56 [INFO] rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-45/rsync/main.c(992) [sender=2.6.9]

I noticed the hard way, while testing the case-sensitivity issue, that the tool handles symlinked folders somewhat unexpectedly.
For testing purposes, I put my work folder under /tmp/... without noticing that /tmp is symlinked to /private /tmp.
fswatch or rsync "silently" translates the paths to the full system path (/private/tmp/...) and syncs these to /private/tmp/... inside the boot2docker. This is as such not a problem, except that if I am locally at /tmp, then:
docker run -v $(pwd):/data ....
fails to find the files as /tmp is not a symlink to /private/tmp inside the boot2docker.

I don't know just yet what behaviour I would prefer, so maybe just a warning in the docs for now?

How does one uninstall docker-osx-dev? I tried removing the script, however any host to docker mounts no longer work as they did before docker-osx-dev.

Right now, everything is being pulled from master, and nothing is versioned, so troubleshooting will get very difficult. The setup.sh and docker-osx-dev scripts should be versioned and released using GitHub's releases support.

After running docker-osx-dev init then docker-osx-dev start I got the following error:

Vagrantfile:66:in `[]': no implicit conversion of Symbol into Integer (TypeError)

It seems that in the generated Vagrantfile DEFAULT_FOLDERS_TO_SYNC should be an array containing a hash:

DEFAULT_FOLDERS_TO_SYNC = [{:src => VAGRANT_ROOT, :dest => VAGRANT_ROOT}]

instead of just the hash:

DEFAULT_FOLDERS_TO_SYNC = {:src => VAGRANT_ROOT, :dest => VAGRANT_ROOT}

Hi,

your project looks awesome so far! I really like it.

Just one problem I ran into:

I have some docker images which I've build for myself. I use these images as a base for nearly every project I have. With docker-osx-dev I have the problem, that a docker image is available globally due to the fact that the VM is different for every folder and I need to build these images every time I start a new project.

Wouldn't it be better to share a VM for every docker-osx-dev project so that downloaded/custom build Images are available everywhere? You can't start multiple ones anyways. At least not without changing the Port in your Vagrantfile.

Greetings

The setup.sh script is idempotent and tries to avoid overwriting software that's already installed, but it doesn't check any version numbers. It should verify that the user has a reasonable version of all dependencies installed, as well as the docker-osx-dev scripts themselves so that the script can be used to update to a new version.

Greets,

We were trying to use this to deploy development environment in our enterprise

But we find out that without using the command "docker-osx-dev start-sync" when I change a file it does not sync.

When starting the project we use this process :

docker-osx-dev start : output
[INFO] Starting Docker and Vagrant
Bringing machine 'boot2docker' up with 'virtualbox' provider...
==> boot2docker: Clearing any previously set forwarded ports...
==> boot2docker: Fixed port collision for 22 => 2222. Now on port 2200.
==> boot2docker: Clearing any previously set network interfaces...
==> boot2docker: Preparing network interfaces based on configuration...
boot2docker: Adapter 1: nat
boot2docker: Adapter 2: hostonly
==> boot2docker: Forwarding ports...
boot2docker: 2375 => 2375 (adapter 1)
boot2docker: 2376 => 2376 (adapter 1)
boot2docker: 22 => 2200 (adapter 1)
==> boot2docker: Running 'pre-boot' VM customizations...
==> boot2docker: Booting VM...
==> boot2docker: Waiting for machine to boot. This may take a few minutes...
boot2docker: SSH address: 127.0.0.1:2200
boot2docker: SSH username: docker
boot2docker: SSH auth method: private key
boot2docker: Warning: Connection timeout. Retrying...
==> boot2docker: Machine booted and ready!
==> boot2docker: Checking for guest additions in VM...
==> boot2docker: Configuring and enabling network interfaces...
==> boot2docker: Rsyncing folder: /Users/hamza/projet/project/typo3conf/ => /Users/hamza/projet/project/typo3conf
==> boot2docker: - Exclude: [".vagrant/", "fileadmin/", "!fileadmin/project/", "!fileadmin/templates/", "typo3conf/localconf.php", "typo3conf/ENABLE_INSTALL_TOOL", "typo3conf/temp_CACHED__", "typo3conf/l10n", "typo3conf/.log", "typo3_src", "typo3", "INSTALL.txt", "phpinfo.php", "README.txt", "RELEASE_NOTES.txt", "..DS_Store", ".__", ".htaccess", "dump/", "purge/", "uploads/", "/nbproject/private/", ".DS_Store", "typo3temp", "dockerdata"]
==> boot2docker: Rsyncing folder: /Users/hamza/projet/project/uploads/ => /Users/hamza/projet/project/uploads
==> boot2docker: - Exclude: [".vagrant/", "fileadmin/", "!fileadmin/project/", "!fileadmin/templates/", "typo3conf/localconf.php", "typo3conf/ENABLE_INSTALL_TOOL", "typo3conf/temp_CACHED__", "typo3conf/l10n", "typo3conf/.log", "typo3_src", "typo3", "INSTALL.txt", "phpinfo.php", "README.txt", "RELEASE_NOTES.txt", "..DS_Store", ".", ".htaccess", "dump/", "purge/", "uploads/", "/nbproject/private/", ".DS_Store", "typo3temp", "dockerdata"]
==> boot2docker: Rsyncing folder: /Users/hamza/projet/project/fileadmin/ => /Users/hamza/projet/project/fileadmin
==> boot2docker: - Exclude: [".vagrant/", "fileadmin/_", "!fileadmin/project/", "!fileadmin/templates/", "typo3conf/localconf.php", "typo3conf/ENABLE_INSTALL_TOOL", "typo3conf/temp_CACHED", "typo3conf/l10n", "typo3conf/.log", "typo3_src", "typo3", "INSTALL.txt", "phpinfo.php", "README.txt", "RELEASE_NOTES.txt", "..DS_Store", ".", ".htaccess", "dump/", "purge/", "uploads/", "/nbproject/private/", ".DS_Store", "typo3temp", "dockerdata"]
==> boot2docker: Rsyncing folder: /Users/hamza/projet/project/typo3temp/ => /Users/hamza/projet/project/typo3temp
==> boot2docker: - Exclude: [".vagrant/", "fileadmin/_", "!fileadmin/poject/", "!fileadmin/templates/", "typo3conf/localconf.php", "typo3conf/ENABLE_INSTALL_TOOL", "typo3conf/temp_CACHED", "typo3conf/l10n", "typo3conf/.log", "typo3_src", "typo3", "INSTALL.txt", "phpinfo.php", "README.txt", "RELEASE_NOTES.txt", "..DS_Store", ".", ".htaccess", "dump/", "purge/", "uploads/", "/nbproject/private/", ".DS_Store", "typo3temp", "dockerdata"]
==> boot2docker: Rsyncing folder: /Users/hamza/projet/project/dockerdata/mysql/ => /Users/hamza/projet/project/dockerdata/mysql
==> boot2docker: - Exclude: [".vagrant/", "fileadmin/_", "!fileadmin//", "!fileadmin/templates/", "typo3conf/localconf.php", "typo3conf/ENABLE_INSTALL_TOOL", "typo3conf/temp_CACHED", "typo3conf/l10n", "typo3conf/.log", "typo3_src", "typo3", "INSTALL.txt", "phpinfo.php", "README.txt", "RELEASE_NOTES.txt", "..DS_Store", ".__", ".htaccess", "dump/", "purge/", "uploads/", "/nbproject/private/", ".DS_Store", "typo3temp", "dockerdata"]
==> boot2docker: Mounting shared folders...
boot2docker: /vagrant => /Users/hamza/projet/project
==> boot2docker: Machine already provisioned. Run vagrant provision or use the --provision
==> boot2docker: to force provisioning. Provisioners marked to run always will still run.
[INFO] Starting file syncing

docker-compose up :
Recreating project_db_1...
Recreating project_web_1...
Attaching to project_db_1, project_web_1
db_1 | 150526 6:54:57 [Note] Plugin 'FEDERATED' is disabled.
db_1 | 150526 6:54:57 InnoDB: Initializing buffer pool, size = 8.0M
db_1 | 150526 6:54:57 InnoDB: Completed initialization of buffer pool
db_1 | InnoDB: The log sequence number in ibdata files does not match
db_1 | InnoDB: the log sequence number in the ib_logfiles!
db_1 | 150526 6:54:57 InnoDB: Database was not shut down normally!
db_1 | InnoDB: Starting crash recovery.
db_1 | InnoDB: Reading tablespace information from the .ibd files...
db_1 | InnoDB: Restoring possible half-written data pages from the doublewrite
db_1 | InnoDB: buffer...
db_1 | 150526 6:54:57 InnoDB: Started; log sequence number 0 859185403
db_1 | 150526 6:54:57 [Note] Event Scheduler: Loaded 0 events
db_1 | 150526 6:54:57 [Note] mysqld: ready for connections.
db_1 | Version: '5.1.73' socket: '/tmp/mysql.sock' port: 3306 MySQL Community Server (GPL)

At this point everything is going except for the syncing.

I'm on a mac mini OSX 10.10.1

Docker version:

docker -v
Docker version 1.6.0, build 4749651

vagrant -v
Vagrant 1.7.2

Hi,

I really appreciate your take to make docker and osx development a lot easier for all of us.
I tried using docker-osx-dev and everything worked well but file changes I made on local files are not reflected in the docker container...
I am wondering how can I investigate what I'm doing wrong. Are there are restrictions to how I can share folders?

This is the relevant part of my compose.yml:
rails:
build: .
environment:
RAILS_ENV: development
command: "bundle exec rails s"
working_dir: /var/www/rails_app/
expose:
- "3000"
volumes:
- .:/var/www/rails_app/

I was expecting that changes I make on my local files will be reflected on /var/www/90min on the docker container but they are not...

Any help would be appreciated :)

Not sure is an issue with the scripts, but I found my self with this issue and nothing gets synced:

Ferrans-MacBook-Pro-2:association-service ferran-picsolve$ docker-osx-dev
2015-07-02 12:01:56 [INFO] Using default sync paths: .
2015-07-02 12:01:56 [INFO] Complete list of paths to sync: /Users/ferran-picsolve/Documents/Picsolve/projects/association-service
2015-07-02 12:01:56 [INFO] Using default exclude paths: .git
2015-07-02 12:01:56 [INFO] Complete list of paths to exclude: .git
2015-07-02 12:01:56 [INFO] Starting docker-osx-dev file syncing
2015-07-02 12:01:57 [INFO] Installing rsync in the Boot2Docker image
2015-07-02 12:01:57 [INFO] Performing initial sync of paths: /Users/ferran-picsolve/Documents/Picsolve/projects/association-service
2015-07-02 12:01:57 [INFO] ssh: Could not resolve hostname dockerhost: nodename nor servname provided, or not known
2015-07-02 12:01:57 [INFO] rsync: connection unexpectedly closed (0 bytes received so far) [sender]
2015-07-02 12:01:57 [INFO] rsync error: unexplained error (code 255) at /SourceCache/rsync/rsync-45/rsync/io.c(453) [sender=2.6.9]
2015-07-02 12:01:57 [INFO] Initial sync done
2015-07-02 12:01:57 [INFO] Watching: /Users/ferran-picsolve/Documents/Picsolve/projects/association-service

╰─○ bash <(curl -s https://raw.githubusercontent.com/brikis98/docker-osx-dev/master/setup.sh)
[INFO] Updating HomeBrew
Already up-to-date.
[WARN] Cask is already installed by HomeBrew, skipping
[WARN] Found command VBoxManage, assuming VirtualBox is already installed and skipping
[WARN] Boot2Docker is already installed by HomeBrew, skipping
[WARN] Found command docker-compose, assuming Docker Compose is already installed and skipping
[WARN] fswatch is already installed by HomeBrew, skipping
[WARN] GNU core utilities is already installed by HomeBrew, skipping
[INFO] Installing rsync in the Boot2Docker image
[INFO] Adding /usr/local/bin/docker-osx-dev
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 15257  100 15257    0     0  12443      0  0:00:01  0:00:01 --:--:-- 12444
[INFO] Adding dockerhost entry to /etc/hosts so you can use http://dockerhost URLs for testing
[INSTRUCTIONS] Modifying /etc/hosts requires sudo privileges, please enter your password.
Password:
[INFO] Adding DOCKER_HOST to /Users/noah/.zshrc

<< indefinite hang. zsh is maxing out CPU. >>

I made sure to clear /etc/hosts and ~/.zshrc of any old Docker stuff.

I can successfully run shellinit:

─○ boot2docker shellinit
Writing /Users/noah/.boot2docker/certs/boot2docker-vm/ca.pem
Writing /Users/noah/.boot2docker/certs/boot2docker-vm/cert.pem
Writing /Users/noah/.boot2docker/certs/boot2docker-vm/key.pem
    export DOCKER_CERT_PATH=/Users/noah/.boot2docker/certs/boot2docker-vm
    export DOCKER_TLS_VERIFY=1
    export DOCKER_HOST=tcp://192.168.59.103:2376

I tried running (cd src/webapp; docker-osx-dev -i .dockerignore) so that I wouldn't have to rsync massive folders like node_modules. You can see my .dockerignore at https://github.com/league-wins-pool/league-wins-pool/blob/develop/src/webapp/.dockerignore

The problem is once my containers run, the node_modules folder is not present (presumably because it's in my .dockerignore) and Node starts complaining about missing modules.

I suppose the ideal situation is one in which node_modules is mapped into the container, but then not monitored for changes. Or use the node_modules folder that is built into the container instead of removing it when I map my local source folder. I hope that makes sense.

What do you suggest?

╰─± docker-osx-dev -l DEBUG
[DEBUG] Using default sync paths: .
[INFO] Using excludes from ignore file .dockerignore: .git .tmp .sass-cache bower_components dist node_modules .dockerignore
[INFO] Performing initial sync of paths: /Users/noah/Projects/league-wins-pool/src/webapp
[DEBUG] Creating parent directories in Docker VM: sudo mkdir -p /Users/noah/Projects/league-wins-pool/src && sudo chown -R docker /Users/noah/Projects/league-wins-pool/src
[DEBUG] rsync --archive --verbose --delete --omit-dir-times --inplace --whole-file --exclude .git --exclude .tmp --exclude .sass-cache --exclude bower_components --exclude dist --exclude node_modules --exclude .dockerignore --rsh="ssh -i /Users/noah/.ssh/id_boot2docker -o StrictHostKeyChecking=no" /Users/noah/Projects/league-wins-pool/src/webapp docker@dockerhost:/Users/noah/Projects/league-wins-pool/src
[INFO] Warning: Permanently added 'dockerhost,192.168.59.104' (RSA) to the list of known hosts.
[INFO] building file list ... done
[INFO]
[INFO] sent 2631 bytes  received 20 bytes  5302.00 bytes/sec
[INFO] total size is 347403  speedup is 131.05
[INFO] Initial sync done
[INFO] Watching: /Users/noah/Projects/league-wins-pool/src/webapp
[DEBUG] fswatch -0 --exclude .git --exclude .tmp --exclude .sass-cache --exclude bower_components --exclude dist --exclude node_modules --exclude .dockerignore /Users/noah/Projects/league-wins-pool/src/webapp

The initial sync sync just the first folder when running the docker-osx-dev commend :
[INFO] Performing initial sync of paths: /Users/hamza/typo3conf /Users/hamza//uploads /Users/hamza/fileadmin

[DEBUG] /Users/hamza/typo3conf /Users/hamza/uploads /Users/fileadmin :
The log_debug i added with log_debug "$path" (in sync Line 227)

[DEBUG] rsync --archive --verbose --delete --omit-dir-times --inplace --whole-file --exclude .git --rsh="ssh -i /Users/hamza/.ssh/id_boot2docker -o StrictHostKeyChecking=no" /Users/hamza/typo3conf docker@dockerhost:/Users/hamza/projet/uccife62/uccife

output sync

[INFO] Initial sync done

[INFO] Watching: ....

(I will try to correct it)

Unison is a file synchronizer that allows two-way sync. It should be nearly as fast as rsync and perhaps even support file watchers (e.g. inotify) just as well.

There are two difficulties:

1. Unison must be installed on client and server, and it must be at the exact same version on both. It's possible to install it in a Docker image and mount that as a volume everywhere (e.g. see docker-unison), but I think it would be nicer to find a way to install Unison on the Boot2Docker image itself. Boot2Docker does have a package manager called tce (see docs, packages), but unison is not one of the supported packages. That means we need to either a) find a pre-built unison executable that runs on generic Linux x86_64 or b) build it from source on the Boot2Docker image as part of the install.
2. Unison is, in general, more complicated to configure and use than rsync. Going from one-way to two-way sync introduces complexities with what happens when the unison server DB gets stale, as well as different sets of excludes on each side of the sync.

Here is the project https://github.com/gansbrest/hodor

Defined volumes in Docker Compose

  volumes:
    - .:/docker
    - ~/.ssh:/home/deploy/.ssh

Resulting behaviour by docker-osx-dev:

2015-06-17 18:20:53 [INFO] Performing initial sync of paths: /Users/ain/projects/od /Users/ain/projects/od/~/.ssh
2015-06-17 18:20:53 [INFO] key_read: ...
2015-06-17 18:20:53 [INFO] failed
2015-06-17 18:20:55 [INFO] Watching: /Users/ain/projects/od /Users/ain/projects/od/~/.ssh

Maintain a log of the rsync stdout so the user can inspect it

Running integration tests in CircleCI or TravisCI will fail. This is because the integration tests need to run Boot2Docker in a VM, but in a CI environment, the tests are already running in a VM, and nested VMs are not supported in most environments. See comments below for more details.

Instead of downloading the latest in master. Otherwise, the file being tested is out of sync with the commit. Should probably add a -f flag to docker-osx-dev init.

Re-run ./setup.sh during testing.

I applied #36 today and tried to run docker-osx-dev again, but got stuck with:

2015-06-04 15:12:48 [INFO] Using sync paths from Docker Compose file at docker-compose.yml: . ~/.ssh
2015-06-04 15:12:48 [INFO] Paths to sync: /Users/ain/projects/app /Users/ain/projects/app/~/.ssh
2015-06-04 15:12:48 [INFO] Using default exclude paths: .git
2015-06-04 15:12:48 [INFO] Paths to exclude: .git
2015-06-04 15:12:48 [INFO] Installing rsync in the Boot2Docker image
2015-06-04 15:12:48 [INFO] Performing initial sync of paths: /Users/ain/projects/app /Users/ain/projects/sennheiser/~/.ssh
2015-06-04 15:12:49 [INFO] Syncing app/frontend/log/development.log: <f.stp......
2015-06-04 15:12:49 [INFO] rsync: [sender] write error: Broken pipe (32)
2015-06-04 15:12:49 [INFO] rsync: write failed on "/Users/ain/projects/app/frontend/log/development.log": No space left on device (28)

docker@boot2docker:~$ df -h
Filesystem                Size      Used Available Use% Mounted on
tmpfs                     1.8G      1.8G         0 100% /
tmpfs                  1002.1M    584.0K   1001.5M   0% /dev/shm
/dev/sda1                18.2G      9.9G      7.3G  58% /mnt/sda1
cgroup                 1002.1M         0   1002.1M   0% /sys/fs/cgroup
/dev/sda1                18.2G      9.9G      7.3G  58% /mnt/sda1/var/lib/docker/aufs
/dev/loop0               28.0K     28.0K         0 100% /mnt/sda1/tmp/tcloop/popt
/dev/loop1              180.0K    180.0K         0 100% /mnt/sda1/tmp/tcloop/rsync
none                     18.2G      9.9G      7.3G  58% /mnt/sda1/var/lib/docker/aufs/mnt/1e839836a117350eeec30f7d0d3022e308c73b79acc2e2d6173f2be52dca58ad
none                     18.2G      9.9G      7.3G  58% /mnt/sda1/var/lib/docker/aufs/mnt/6ce47e2330ae920aaffe6e70ef019360cdecbfa8cf44593f6f1f247313efa2d5
none                     18.2G      9.9G      7.3G  58% /mnt/sda1/var/lib/docker/aufs/mnt/62d6747418cc4580bae58191c8815670cdebc0beb8a6ca62205b1002f3c12761

Any ideas on how to prevent this?

As of today, setup.sh for me does this:

curl https://raw.githubusercontent.com/brikis98/docker-osx-dev/master/setup.sh | bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 13690  100 13690    0     0   6491      0  0:00:02  0:00:02 --:--:--  6491
[INFO] Updating HomeBrew
Updated Homebrew from e9ae5916 to 03e4c2a5.
==> Updated Formulae
stty: stdin isn't a terminal
monit
[WARN] Cask is already installed by HomeBrew, skipping
[WARN] Found command VBoxManage, assuming VirtualBox is already installed and skipping
[WARN] Boot2Docker is already installed by HomeBrew, skipping
[WARN] Found command docker-compose, assuming Docker Compose is already installed and skipping
[WARN] fswatch is already installed by HomeBrew, skipping
[WARN] GNU core utilities is already installed by HomeBrew, skipping
[INFO] Initializing Boot2Docker VM
Latest release for github.com/boot2docker/boot2docker is v1.6.2
Downloading boot2docker ISO image...
Success: downloaded https://github.com/boot2docker/boot2docker/releases/download/v1.6.2/boot2docker.iso
    to /Users/ilau/.boot2docker/boot2docker.iso
Generating public/private rsa key pair.
Your identification has been saved in /Users/ilau/.ssh/id_boot2docker.
Your public key has been saved in /Users/ilau/.ssh/id_boot2docker.pub.
The key fingerprint is:
48:48:00:ce:f1:e6:61:e1:6c:cf:f9:1c:f7:e8:2b:fe [email protected]
The key's randomart image is:
+--[ RSA 2048]----+
|.o.o.            |
|o =...           |
| o O. .          |
|  = +...         |
|   . +..S.       |
|      o o o      |
|       o . .     |
|       ..        |
|      ..oE.      |
+-----------------+
[INFO] Starting Boot2Docker VM
Waiting for VM and Docker daemon to start...
.........................ooooooooooooooooooooooo
Started.
Writing /Users/ilau/.boot2docker/certs/boot2docker-vm/ca.pem
Writing /Users/ilau/.boot2docker/certs/boot2docker-vm/cert.pem
Writing /Users/ilau/.boot2docker/certs/boot2docker-vm/key.pem
Your environment variables are already set correctly.

[INFO] Installing rsync in the Boot2Docker image
Downloading: popt.tcz
Connecting to repo.tinycorelinux.net (89.22.99.37:80)
popt.tcz              12% |****                           |  3724   0:00:06 ETApopt.tcz: OK
popt.tcz             100% |*******************************| 28672   0:00:00 ETA
Downloading: rsync.tcz
Connecting to repo.tinycorelinux.net (89.22.99.37:80)
rsync.tcz             19% |*****                          | 35115   0:00:04 ETArsync.tcz: OK
rsync.tcz            100% |*******************************|   180k  0:00:00 ETA
$ echo $?
0

However, at this point ssh'ing into boot2docker reveals

docker@boot2docker:~$ which rsync
/usr/local/bin/rsync

so apparently the install succeeded. Also, running setup.sh a second time usually goes through to the very end, although just now I had an incident where it just quit immediately after checking if rsync is already installed.

Currently, utility functions such as logging are copy and pasted between setup.sh and the docker-osx-dev script. The code needs to be more DRY, which means extracting that functionality into a single location. This may involve adding a build step to generate the final scripts.

Hey Ho,

first to say: Thanks a bunch for making docker development on osx possible.
Now it is fast and makes a lot of fun! Without rsync it was not useable...

I glanced through the code a bit. Is it really necessary to remove the VirtualBox Share through restarting the box? Wouldn't it be enough to umount the share (e.g.: boot2docker ssh sudo umount /Users)? After that the rsync would be on the virtual hard disk of the machine and should not slow down the file sync anymore.

Should I prepare a Pull Request or did i omit a fact?

Greetings, @it_supertramp

Script seems to be running until it tries to install rsync in boot2docker

2015-07-01 16:14:01 [INFO] Using default sync paths: .
2015-07-01 16:14:01 [INFO] Complete list of paths to sync: /Users/ferran-picsolve/Documents/Picsolve/projects/association-service
2015-07-01 16:14:01 [INFO] Using default exclude paths: .git
2015-07-01 16:14:01 [INFO] Complete list of paths to exclude: .git
2015-07-01 16:14:01 [INFO] Starting docker-osx-dev file syncing
/usr/local/bin/docker-osx-dev: line 286: boot2docker: command not found
2015-07-01 16:14:01 [INFO] Initializing Boot2Docker VM
/usr/local/bin/docker-osx-dev: line 372: boot2docker: command not found
Ferrans-MacBook-Pro-2:association-service ferran-picsolve$ docker-osx-dev install
2015-07-01 16:14:06 [INFO] Starting install of docker-osx-dev
2015-07-01 16:14:06 [INFO] Updating HomeBrew
Already up-to-date.
2015-07-01 16:14:15 [WARN] Cask is already installed by HomeBrew, skipping
2015-07-01 16:14:15 [INFO] Installing Boot2Docker
==> Installing boot2docker dependency: docker
==> Downloading https://homebrew.bintray.com/bottles/docker-1.7.0.yosemite.bottle.tar.gz
Already downloaded: /Library/Caches/Homebrew/docker-1.7.0.yosemite.bottle.tar.gz
==> Pouring docker-1.7.0.yosemite.bottle.tar.gz
==> Caveats

Bash completion has been installed to:
  /usr/local/etc/bash_completion.d

zsh completion has been installed to:
  /usr/local/share/zsh/site-functions
==> Summary
🍺  /usr/local/Cellar/docker/1.7.0: 9 files, 7.4M
==> Installing boot2docker
==> Downloading https://homebrew.bintray.com/bottles/boot2docker-1.7.0.yosemite.bottle.tar.gz
Already downloaded: /Library/Caches/Homebrew/boot2docker-1.7.0.yosemite.bottle.tar.gz
==> Pouring boot2docker-1.7.0.yosemite.bottle.tar.gz
==> Caveats

To have launchd start boot2docker at login:
    ln -sfv /usr/local/opt/boot2docker/*.plist ~/Library/LaunchAgents
Then to load boot2docker now:
    launchctl load ~/Library/LaunchAgents/homebrew.mxcl.boot2docker.plist
==> Summary
🍺  /usr/local/Cellar/boot2docker/1.7.0: 3 files, 7.3M
2015-07-01 16:14:17 [WARN] Found command docker-compose, assuming Docker Compose is already installed and skipping
2015-07-01 16:14:17 [WARN] fswatch is already installed by HomeBrew, skipping
2015-07-01 16:14:17 [WARN] GNU core utilities is already installed by HomeBrew, skipping
2015-07-01 16:14:18 [INFO] Installing rsync in the Boot2Docker image
wget: not an http or ftp url: https://raw.githubusercontent.com/brikis98/docker-osx-dev/master/lib/rsync
error in run: exit status 1

Hi, rsync is working perfectly and I can confirm that changes made on the local volume are being reflected in the docker container's source code by using docker exec and entering the container.

However, I'm using nodemon to watch for changes and it isn't triggering a server restart even though the files being watched have changed?

Any help would be much appreciated, thanks!

Hi there,

We have been notified of a bug since the last version you released,

In order to get docker-osx-dev working we have to do the following :

boot2docker start --vbox-share=disable
boot2docker ssh
tce (then s, rsync i)

Then it works like a charm.

The setup.sh script should see what environment variables are set using the env command, executed in a new shell (e.g. "${SHELL}" -i -c "env | grep \"^${var}=\""), instead of using grep on the .bashrc style files. See #18 for more info.

$ docker-osx-dev start

[INFO] Starting Docker and Vagrant
Bringing machine 'boot2docker' up with 'virtualbox' provider...
==> boot2docker: Importing base box 'blinkreaction/boot2docker'...
==> boot2docker: Matching MAC address for NAT networking...
==> boot2docker: Setting the name of the VM: nodeschool-hk_boot2docker
==> boot2docker: Clearing any previously set network interfaces...
==> boot2docker: Preparing network interfaces based on configuration...
    boot2docker: Adapter 1: nat
    boot2docker: Adapter 2: hostonly
==> boot2docker: Forwarding ports...
    boot2docker: 2375 => 2375 (adapter 1)
    boot2docker: 2376 => 2376 (adapter 1)
    boot2docker: 22 => 2222 (adapter 1)
==> boot2docker: Running 'pre-boot' VM customizations...
==> boot2docker: Booting VM...
==> boot2docker: Waiting for machine to boot. This may take a few minutes...
    boot2docker: SSH address: 127.0.0.1:2222
    boot2docker: SSH username: docker
    boot2docker: SSH auth method: private key
    boot2docker: Warning: Connection timeout. Retrying...
==> boot2docker: Machine booted and ready!
==> boot2docker: Checking for guest additions in VM...
==> boot2docker: Configuring and enabling network interfaces...
==> boot2docker: Rsyncing folder: /Users/abtris/Sites/nodeschool-hk/ => /Users/abtris/Sites/nodeschool-hk
==> boot2docker:   - Exclude: [".vagrant/", "npm-debug.log", ".grunt", ".git", ".sass-cache/", ".DS_Store", ".AppleDouble", ".LSOverride", ".cordova", "node_modules", ".vagrant"]
==> boot2docker: Mounting shared folders...
    boot2docker: /vagrant => /Users/abtris/Sites/nodeschool-hk
==> boot2docker: Running provisioner: shell...
    boot2docker: Running: inline script

$ docker build .

Sending build context to Docker daemon
FATA[0000] An error occurred trying to connect: Post https://localhost:2375/v1.18/build?cpusetcpus=&cpushares=0&dockerfile=Dockerfile&memory=0&memswap=0&rm=1&t=: tls: oversized record received with length 20527

$ echo $DOCKER_HOST

tcp://localhost:2375

Idea how solve this? With normal boot2docker, I don't have any this type of problems.

Rsync won't install on the boot2docker 1.7 image, so this won't work on it (boot2docker/boot2docker#936)

A current workaround is to manually tell boot2docker to use the earlier image.

docker-osx-dev install
# wait for it to install some stuff and then fail
boot2docker delete
rm -r ~/.boot2docker
boot2docker init --iso-url="https://github.com/boot2docker/boot2docker/releases/download/v1.6.2/boot2docker.iso" -v
docker-osx-dev install

Even though it says it is ignoring the .git directory, it still gives me errors when trying to sync that directory

$ docker-osx-dev
2015-06-25 08:35:01 [INFO] Using sync paths from Docker Compose file at docker-compose.yml: .
2015-06-25 08:35:01 [INFO] Complete list of paths to sync: /Users/saul/projects/django-report-builder
2015-06-25 08:35:01 [INFO] Using default exclude paths: .git
2015-06-25 08:35:01 [INFO] Complete list of paths to exclude: .git
2015-06-25 08:35:01 [INFO] Starting docker-osx-dev file syncing
2015-06-25 08:35:02 [INFO] Installing rsync in the Boot2Docker image
2015-06-25 08:35:02 [INFO] Performing initial sync of paths: /Users/saul/projects/django-report-builder
2015-06-25 08:35:02 [INFO] Warning: Permanently added 'dockerhost,192.168.59.103' (RSA) to the list of known hosts.
2015-06-25 08:35:02 [INFO] Initial sync done
2015-06-25 08:35:02 [INFO] Watching: /Users/saul/projects/django-report-builder
2015-06-25 08:35:08 [INFO] rsync: link_stat "/Users/saul/projects/django-report-builder/.git/index.lock" failed: No such file or directory (2)

The script appends to /etc/hosts incorrectly, without a line break:

192.168.59.103 uk-en.sr-frontend.dev
192.168.59.103 de-en.sr-frontend.dev
192.168.59.103 de-de.sr-frontend.dev192.168.59.103 dockerhost

Although integration tests are currently blocked due to #7, it should still be possible to add unit tests for most of the shell scripts.

I am trying to ssh into the vm in order to add the --insecure-registry flag to the docker daemon. We us a private registry behind our firewall.

I tried running ssh localdocker which prompts me to add localdocker to knownhosts, but then it requires a password (which I don't know).

Am I going about it the wrong way?

Nice work! It's about almost the same speed to deploy Tomcat on a local directory.
But I met a problem when I was installing docker-osx-dev. My boot2docker-vm was running at that time and I didn't notice it. Below is my message I got:

==> Installing fig dependency: docker
==> Downloading https://homebrew.bintray.com/bottles/docker-1.6.2.yosemite.bottl

################################################################## 100.0%

==> Pouring docker-1.6.2.yosemite.bottle.tar.gz
==> Caveats
Bash completion has been installed to:
/usr/local/etc/bash_completion.d

zsh completion has been installed to:
/usr/local/share/zsh/site-functions
Error: The brew link step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink bin/docker
Target /usr/local/bin/docker
already exists. You may want to remove it:
rm '/usr/local/bin/docker'

To force the link and overwrite all conflicting files:
brew link --overwrite docker

To list all files that would be deleted:
brew link --overwrite --dry-run docker

Possible conflicting files are:
/usr/local/bin/docker
==> Summary
🍺 /usr/local/Cellar/docker/1.6.2: 9 files, 7.3M
==> Installing fig dependency: boot2docker
==> Downloading https://homebrew.bintray.com/bottles/boot2docker-1.6.2.yosemite.

################################################################## 100.0%

==> Pouring boot2docker-1.6.2.yosemite.bottle.tar.gz
==> Caveats
To have launchd start boot2docker at login:
ln -sfv /usr/local/opt/boot2docker/*.plist ~/Library/LaunchAgents
Then to load boot2docker now:
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.boot2docker.plist
Error: The brew link step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink bin/boot2docker
Target /usr/local/bin/boot2docker
already exists. You may want to remove it:
rm '/usr/local/bin/boot2docker'

To force the link and overwrite all conflicting files:
brew link --overwrite boot2docker

To list all files that would be deleted:
brew link --overwrite --dry-run boot2docker

Possible conflicting files are:
/usr/local/bin/boot2docker
==> Summary
🍺 /usr/local/Cellar/boot2docker/1.6.2: 3 files, 7.3M
==> Installing fig
==> Downloading https://homebrew.bintray.com/bottles/fig-1.2.0.yosemite.bottle.t

################################################################## 100.0%

==> Pouring fig-1.2.0.yosemite.bottle.tar.gz
==> Caveats
Bash completion has been installed to:
/usr/local/etc/bash_completion.d
==> Summary
🍺 /usr/local/Cellar/fig/1.2.0: 212 files, 2.7M

It's my first time to use this script so I guess it was installed.
I used the command docker-osx-dev within the folder I want to share. I got

2015-06-08 09:40:50 [INFO] Using default sync paths: .
/usr/local/bin/docker-osx-dev: line 842: greadlink: command not found

Obviously, some dependencies are not installed properly. I think it must be related to the error message before. So I re-installed docker-osx-dev. All problem are solved.

So my opinion is, shouldn't we warn people to shut down boot2docker-vm before installing docker-osx-dev in README.md?

i am using zsh and have added rsync plugin in .zshrc file.
rsync can work sometime ,but sometimes errors are as follows:

➜ /Users/wanghaisheng/workspace/docker-images/mobileapi git:(master) ✗>docker-osx-dev 
Log level: INFO
[INFO] Using default sync paths: .
[INFO] Paths to sync: /Users/wanghaisheng/workspace/docker-images/mobileapi
[INFO] Using default exclude paths: .git
[INFO] Paths to exclude: .git
[INFO] Performing initial sync of paths: /Users/wanghaisheng/workspace/docker-images/mobileapi
[INFO] sh: rsync: not found
[INFO] rsync: connection unexpectedly closed (0 bytes received so far) [sender]
[INFO] rsync error: remote command not found (code 127) at /SourceCache/rsync/rsync-45/rsync/io.c(453) [sender=2.6.9]
[INFO] Initial sync done
[INFO] Watching: /Users/wanghaisheng/workspace/docker-images/mobileapi

when running docker-osx-dev in a directory with docker compose, i'm getting this output:

2015-07-01 22:54:52 [INFO] Using sync paths from Docker Compose file at docker-compose.yml: ./src
2015-07-01 22:54:52 [INFO] Complete list of paths to sync: /Users/markshust/Sites/src
2015-07-01 22:54:52 [INFO] Using default exclude paths: .git
2015-07-01 22:54:52 [INFO] Complete list of paths to exclude: .git
2015-07-01 22:54:52 [INFO] Starting docker-osx-dev file syncing
2015-07-01 22:54:53 [INFO] Installing rsync in the Boot2Docker image
2015-07-01 22:54:53 [INFO] Performing initial sync of paths: /Users/markshust/Sites/src
2015-07-01 22:54:53 [INFO] sh: rsync: not found
2015-07-01 22:54:53 [INFO] rsync: connection unexpectedly closed (0 bytes received so far) [sender]
2015-07-01 22:54:53 [INFO] rsync error: remote command not found (code 127) at /SourceCache/rsync/rsync-45/rsync/io.c(453) [sender=2.6.9]
2015-07-01 22:54:53 [INFO] Initial sync done
2015-07-01 22:54:53 [INFO] Watching: /Users/markshust/Sites/src

It appears rsync isn't installed on the boot2docker instance.

If I do boot2docker ssh "which rsync" it returns:
/usr/local/bin/rsync

However, if I do boot2docker ssh "rsync" it returns:
sh: rsync: not found
error in run: exit status 127

Nothing is syncing when running docker-compose up -d

Hey. Stuck at the initial stage of running docker-osx-dev. The command fails at the busybox stage. It also shows greadlink command not being found. Given below is the output

Adnans-MacBook-Pro:docker-test kiriappeee$ docker-osx-dev -s /Users/kiriappeee/Programming/common/learning/languages/node/

2015-06-13 22:41:59 [INFO] Using sync paths from command line args: /Users/kiriappeee/Programming/common/learning/languages/node/
/usr/local/bin/docker-osx-dev: line 842: greadlink: command not found
2015-06-13 22:41:59 [INFO] Complete list of paths to sync: 
2015-06-13 22:41:59 [INFO] Using default exclude paths: .git
2015-06-13 22:41:59 [INFO] Complete list of paths to exclude: .git
2015-06-13 22:41:59 [INFO] Starting docker-osx-dev file syncing
2015-06-13 22:41:59 [INFO] Starting Boot2Docker VM
Waiting for VM and Docker daemon to start...
..............oooo
Started.
Writing /Users/kiriappeee/.boot2docker/certs/boot2docker-vm/ca.pem
Writing /Users/kiriappeee/.boot2docker/certs/boot2docker-vm/cert.pem
Writing /Users/kiriappeee/.boot2docker/certs/boot2docker-vm/key.pem
Your environment variables are already set correctly.

2015-06-13 22:42:15 [INFO] Installing rsync in the Boot2Docker image
Downloading: popt.tcz
Connecting to repo.tinycorelinux.net (89.22.99.37:80)
popt.tcz             100% |*******************************| 28672   0:00:00 ETA
popt.tcz: OK
Downloading: rsync.tcz
Connecting to repo.tinycorelinux.net (89.22.99.37:80)
rsync.tcz            100% |*******************************|   180k  0:00:00 ETA
rsync.tcz: OK
2015-06-13 22:42:23 [INFO] Performing initial sync of paths: 
BusyBox v1.22.1 (2014-04-04 15:16:09 UTC) multi-call binary.

Usage: mkdir [OPTIONS] DIRECTORY...

Create DIRECTORY

    -m MODE Mode
    -p  No error if exists; make parent directories as needed

error in run: exit status 1

I create a container with -v /host/path:/container/path to mount a directory.

Changes to /container/path in my container are not synced back to /host/path on my host machine.

However, changes to /host/path are synced to /container/path.