wojas / docker-mac-network Goto Github PK

Access your Docker for Mac's internal networks from your macOS host machine

License: MIT License

Shell 100.00%

docker-mac-network's Introduction

This solution allows you to access your Docker for Mac's internal networks directly from your macOS host machine, using OpenVPN.

Quickstart

To access your Docker networks:

Install Tunnelblick (open source macOS OpenVPN client that sits in your menubar)
Run docker-compose up. The first time it will take up to a minute to startup, because it needs to generate keys.
Doubleclick the generated docker-for-mac.ovpn in Finder (or run open docker-for-mac.ovpn in your terminal) to add it to Tunnelblick. You will find it in the current folder.
In Tunnelblick, connect to your new docker-for-mac profile.

You will now be able to access the internal Docker networks from macOS.

Implementation notes

The Compose configuration consists of two services, both based on the tiny Alpine Linux distribution.

openvpn

The OpenVPN image used is kylemanna/openvpn.

Both server-side and client-side configuration are automatically generated by helpers/run.sh, which calls the helper scripts in original image and adjust configuration for accessing the Docker for Mac networks only.

This service runs on TCP port 1194 using host networking, which means it has access to all Docker networks in the host VM.

Only the 172.16.0.0/20 private network is routed to Docker for Mac by the generated config. No DNS servers are set on the host.

The OpenVPN configuration (/etc/openvpn/*) is stored locally in ./config/ so that it survives Docker filesystem resets and allows for easy customization.

proxy

Since containers running in host networking mode in Docker for Mac cannot bind ports to make them accessible from macOS, we need an extra TCP proxy. This image uses socat to forward port 13194 to the OpenVPN container.

Tips

Add restart: always to both services in docker-compose.yml to have them automaticaly restart.
To route extra subnets, add extra route statements in your docker-for-mac.ovpn
To setup static IP addresses for containers, check the app_net examples in the Compose file reference
To regenerate all files, remove config/* and docker-for-mac.ovpn

docker-mac-network's People

Contributors

Stargazers

Watchers

docker-mac-network's Issues

Split DNS proxy to docker embedded dns

Is this possible? Trying to find a way to set this up

Missing docker-for-mac.ovpn

Hello I am not sure where docker-for-mac.ovpn file is generated.
Is there any step/configuration I am missing?

Can't access containers

Hello! Let me know anything else I need to provide.

I attempted to set this up so that I could access machines via docker-compose. I set it up as specified except I changed it from 172.16 to 172.18 as that's what docker inspect reported it as being on.

I only modified the .ovpn. However I cannot access 172.18.0.3 with a simple ping or the actual service. Is there a reason why this is happening?

Thanks!

Make Docker internal DNS name resolution service available to OSX through the VPN

Using 4km3/dnsmasq it's possible to run a resolver that passes DNS queries through to Docker's internal name resolution service. This works for container names, as well as network alias names.

It would be really handy if the docker-mac-network ovpn server could push the DNS proxy container's IP as a resolver when the VPN is connected, allowing users to access their docker containers by name / hostname / alias, rather than by IP address.

I've got this working in a demo project but it requires manual editing of the .ovpn file before importing into tunnelblick.

Still can't connect

Hi, thanks for the helpers.

I'm trying to connect to kafka using this docker-compose.yml in this url, the docker-compose.yml I'm using resides inexamples/kafka-single-node in that repository.

I installed your solution, composed upped it, then, followed your steps, added the vpn config to my tunnelblick. It's OK. Then, when I try to connect to the kafka container, it didn't response.

# the kafka container uses this port
$ telnet localhost 29092
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused
telnet: Unable to connect to remote host

I also tried to expose the port in docker-compose.yml file, still no luck.

This is the docker-compose.yml I'm using to launch kafka and zookeper:

---
version: '2'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    network_mode: host
    environment:
      ZOOKEEPER_CLIENT_PORT: 32181
      ZOOKEEPER_TICK_TIME: 2000
    extra_hosts:
      - "moby:127.0.0.1"

  kafka:
    image: confluentinc/cp-kafka:latest
    network_mode: host
    ports:
      - "29092:29092"
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: localhost:32181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:29092
    extra_hosts:
      - "moby:127.0.0.1"

VERIFY ERROR

2019-07-14 21:16:53 VERIFY ERROR: depth=1, error=certificate signature failure: /CN=localhost
2019-07-14 21:16:53 TLS_ERROR: BIO read tls_read_plaintext error: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
2019-07-14 21:16:53 TLS Error: TLS object -> incoming plaintext read error
2019-07-14 21:16:53 TLS Error: TLS handshake failed

How to access nginx through browser

$ docker run -d --name nginx -p80:80 nginx
$ curl 172.16.0.2
this is successful

but when I open 172.16.0.2 through chrome, it doesn't work

another example is that I can ping Kafka IP successful when I build a Kafka clusters Outside docker
but I can't connect with Kafka clusters Outside docker

Hosts with non bound ports are not accessible through VPN

First, this is a great project! Wonderful approach to get around the problem that we have on OSX currently.

My problem is that I want to be able to do a docker-compose scale against one of my services. This service needs to have a static port and a different IP for each. I can't map the port to OSX because there would then be conflicts. I cannot make this a random port because the app must know what port and IP it's serving on so that it can announce it and be reached.

VPN is a perfect solution for this however, when I try to access any of the hosts that do not have mapped ports, I cannot reach them.

I tracked the issue down to the fact that there are firewall rules on the docker VM that only allow traffic to hosts that have mapped ports and drop all other traffic.

I assume that these rules were created by docker.

I was able to fix the issue by adding a few rules from within the OpenVPN container (against the docker host). I don't' want to have to do this manually or programmatically for every service so I'd like to automatically open up the interface on start of the VPN.

My plan is to add a rule or rules to the OpenVPN container startup that will allow this traffic. I'm not sure if docker will try to overwrite these changes however and am wondering if there is a better way.

Any thoughts?

Can't access containers in the internal Docker bridge network

I've tried running the current version but I can't connect to my other running docker containers through the internal Docker IPs.

After tracking down the issue, it seems that starting the openvpn container in host network mode is the culprit. I've forked the repo and made some changes so that openvpn runs in bridge network mode (and also removed the proxy container since it's not really needed) and now it works!

If you are interested I can make a pull request?

Can't connect to containers

I had this working successfully for quite a while, just following the simple guide from the readme.

For some reason though, after reinstalling macOS, I can't get it to work again.

I've followed the steps, connect to the VPN, but I simply can't connect to any service using their IP's.

Can someone please help me out? Let me know what more information I could provide.

Issue with socat and ncurses

Please see that this does not work with docker 17.06 ownards . On docker 17.09 I get curses & socat errors but on 17.06 socat errors.

┌─(/Volumes/Data/backend/development/projects//git/docker-mac-network)──────────────────────────────────────────────────────────────────────────────(arung@IN4454-01:s000)─┐
└─(09:21:19 on master)──> docker-compose up 1 ↵ ──(Fri,Oct13)─┘
Building proxy
Step 1/3 : FROM alpine
---> 76da55c8019d
Step 2/3 : RUN apk add --no-cache socat
---> Running in 5074461b4afa
fetch http://dl-cdn.alpinelinux.org/alpine/v3.6/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.6/community/x86_64/APKINDEX.tar.gz
(1/5) Installing ncurses-terminfo-base (6.0_p20170930-r0)
(2/5) Installing ncurses-terminfo (6.0_p20170930-r0)
(3/5) Installing ncurses-libs (6.0_p20170930-r0)
(4/5) Installing readline (6.3.008-r5)
(5/5) Installing socat (1.7.3.2-r1)
ERROR: socat-1.7.3.2-r1: Protocol error
Executing busybox-1.26.2-r5.trigger
1 errors; 12 MiB in 15 packages
ERROR: Service 'proxy' failed to build: The command '/bin/sh -c apk add --no-cache socat' returned a non-zero code: 1

openvpn:latest image doesn't work with client config

Since OpenVPN 2.4, the server has apparently gotten pickier about got compression configuration and the generated client config causes errors like this for me (Viscosity, OS X):

openvpn_1  | Mon Oct 15 02:59:03 2018 host/172.24.0.2:55030 Bad compression stub (swap) decompression header byte: 42
openvpn_1  | Mon Oct 15 02:59:13 2018 host/172.24.0.2:55030 Bad compression stub (swap) decompression header byte: 42
openvpn_1  | Mon Oct 15 02:59:23 2018 host/172.24.0.2:55030 Bad compression stub (swap) decompression header byte: 42
openvpn_1  | Mon Oct 15 02:59:33 2018 host/172.24.0.2:55030 Bad compression stub (swap) decompression header byte: 42
openvpn_1  | Mon Oct 15 02:59:43 2018 host/172.24.0.2:55030 Bad compression stub (swap) decompression header byte: 42
openvpn_1  | Mon Oct 15 02:59:53 2018 host/172.24.0.2:55030 Bad compression stub (swap) decompression header byte: 42

Pinning the image to kylemanna/openvpn:2.3 in docker-compose.yml resolved this issue. I tried my hand at fixing the generated client / server config, but I am a real novice with OpenVPN and didn't get anywhere.

iptables v1.8.4 (legacy): can't initialize iptables table `nat': iptables who? (do you need to insmod?)

Hi, I want to ask why I get a "docker-mac-network_openvpn_1 exited with code 3" when I tried the docker-compose up. Every steps seems correct but it keeps looping at this iptables. Any suggestion will be appreciated. The .ovpn created by this does not work.