Giter VIP home page Giter VIP logo

nginx_proxy's Introduction

Nginx forward proxy

Nginx is a very fast HTTP and reverse proxy server. Usually, Nginx is used to serve and cache static assets or as proxy or load balancer for incoming traffic to application servers. In this repository, it is used as forward proxy.

Use Case

Assume you have a network where you want to control outgoing traffic. You either want to:

  • Deny all outgoing calls by default and only allow HTTP(S) calls to allowlisted URLs.
  • Allow all outgoing calls by default and only block HTTP(S) calls to denylisted URLs.

The Docker daemon can be configured that way that it routes all traffic through an proxy. This proxy can be an Nginx which is configured as forwarding proxying.

ngx_http_proxy_connect_module

Nginx can be configured for forwarding proxying. Unfortunately, that doesn't work very well with HTTPS connections. As soon the user is calling a URL via https, Nginx will throw errors. There is a StackOverflow issue to that topic. Luckily there is a solution for that problem. The ngx_http_proxy_connect_module is solving this issue. If Nginx is compiled with that module, the proxying will work with SSL connections as well.

Docker Build

The Dockerfile in this repository is assembling an Nginx with the ngx_http_proxy_connect_module and an nginx.conf file which blocks all outgoing traffic by default, but allows access to some allowlisted domains like google.com. The Docker image can be built like this:

docker build -t reiz/nginx_proxy:0.0.3 . 

Or simply download it from Docker Hub with:

docker pull reiz/nginx_proxy:0.0.3

Allowlist certain domains

This repository contains two nginx configuration files. The nginx_allowlist.conf file is built for the use case that you want to deny all outgoing traffic by default and only allow some allowlisted domains. In the first server section domains can be allowlisted by simply adding a server_name * line for each allowlisted domain. Here an example:

    # Allowlist Google
    server {
        listen       8888;
        server_name  google.com;
        server_name  *.google.com;
        server_name  google.de;
        server_name  www.google.de;
        proxy_connect;
        proxy_max_temp_file_size 0;
        resolver 8.8.8.8;
        location / {
           proxy_pass http://$http_host;
           proxy_set_header Host $http_host;
        }
    }

Regex can be used to describe a domain. This *.google.com for example is allowlisting all subdomains of google.com. In the above example, google.com and all subdomains of it are allowlisted. Beside that google.de and www.google.de are allowlisted. Subdomains of google.de are not allowlisted. The proxy would allow outgoing calls to this domains:

This domains are blocked with the above configuration:

  • mail.google.de
  • api.google.de

By starting the Docker container the file can be mounted into the running container.

docker run -d -p 8888:8888 -v ${PWD}/nginx_allowlist.conf:/usr/local/nginx/conf/nginx.conf reiz/nginx_proxy:0.0.3 

Now the Docker container is running with the mounted configuration.

Denylist certain domains

This repository contains two nginx configuration files. The nginx_denylist.conf file is built for the use case that you want to allow all outgoing traffic by default and only block traffic to some domains. In the first server section domains can be denylisted by simply adding a server_name * line for each denylisted domain. Here an example:

    server {
        listen       8888;
        server_name  google.com;
        server_name  *.google.com;
        return 404;
    }

In the example above all pages would be accessible, but google.com and all subdomains of it would be blocked. Regex can be used here in the same way as in the allowlist example. By starting the Docker container the file can be mounted into the running container.

docker run -d -p 8888:8888 -v ${PWD}/nginx_denylist.conf:/usr/local/nginx/conf/nginx.conf reiz/nginx_proxy:0.0.3

Now the Docker container is running with the mounted configuration.

Testing

You can test your configuration by pointing your Browser to the Nginx proxy in the running Docker container. If you run the Docker container on your localhost, then you can point your Browser to localhost:8888. Here is an example how it looks like in Firefox:

Firefox Proxy Settings

Configuring Docker and Kubernetes with a Proxy

Assuming you have a cluster of Docker machines (Kubernetes cluster) and you would like to route all outgoing traffic to your proxy. That can be achieved by setting some global ENV variables on each Docker machine.

For RedHat/CentOS version 6:

cat <<EOF | sudo tee -a /etc/sysconfig/docker
export http_proxy="http://myproxy.example.com:8888"
export https_proxy="https://myproxy.example.com:8888"
export no_proxy=<REGISTRY_IP>
EOF
 
sudo service docker restart

For RedHat/CentOS version 7, remove export:

cat <<EOF | sudo tee -a /etc/sysconfig/docker
http_proxy="http://myproxy.example.com:8888"
https_proxy="https://myproxy.example.com:8888"
no_proxy=<REGISTRY_IP>
EOF
 
sudo sed -i '/\[Service\]/a EnvironmentFile=/etc/sysconfig/docker' /usr/lib/systemd/system/docker.service
sudo systemctl daemon-reload
sudo service docker restart

For Ubuntu 14.04:

cat <<EOF | sudo tee -a /etc/default/docker
export http_proxy="http://myproxy.example.com:8888"
export https_proxy="https://myproxy.example.com:8888"
export no_proxy=<REGISTRY_IP>
EOF
 
sudo restart docker

For Kubernetes it works the same way. The http_proxy ENV has to be set before the K8S processes are starting. Minikube can be started with proxy params directly. Here an example:

https_proxy=http://<PROXY_SERVER>:80 minikube start --docker-env HTTP_PROXY=http://<PROXY_SERVER>:80 --docker-env HTTPS_PROXY=http://<PROXY_SERVER>:80 --docker-env NO_PROXY=192.168.99.0/24

Alternatively the Proxy can be set by Container start as well:

docker run -e "http_proxy=http://myproxy.example.com:8888" \
           -e "https_proxy=https://myproxy.example.com:8888" \
           -d liveperson\app run.sh

Version Matrix

This tables show which Docker tag contains which Ubuntu & Nginx version:

Docker tag Ubuntu version Nginx version Published at Comment
reiz/nginx_proxy:0.0.1 16.04 1.10.3
reiz/nginx_proxy:0.0.2 18.04 1.14.0
reiz/nginx_proxy:0.0.3 20.04 1.18.0
reiz/nginx_proxy:0.0.4 20.04 1.18.0 2021-04-25 With libssl-dev
reiz/nginx_proxy:0.0.5 20.04 1.18.0 2021-09-17 Reduced image size from 471MB to 147MB

License

This repository is open source under the MIT license. The full license text is available in the LICENSE.md file.

nginx_proxy's People

Contributors

antonio-ramadas avatar behoyh avatar dependabot[bot] avatar maxlemieux avatar nathan-b avatar pratikbin avatar reiz avatar tmcheung avatar toversus avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

nginx_proxy's Issues

will it support nginx 1.17.*

thanks for writing in about nginx forward proxy here.
i am using nginx 1.17., i have tried 1.16 connect patch but it's not working.
any thoughts on this?

Fix docker-compose

Docker-compose references old non-inclusive nginx conf filename.
Fix in #24

Edit: also fixed Dockerfile reference

Proxy forward to other Proxy

Hi there Robert,
This isn't an issue as such alas I couldn't figure out another way to contact you. I apologise if this it the incorrect forum but would truly appreciate any help/guidance you could provide as you seem to really know what's going on wrt HTTPS traffic going through an nginx server as a forwarding proxy.

I'm trying out your forwarding proxy but I don't appear to be able to get through it with https?
Ideally I was looking to see if there was a way I could use your docker image as a form of traffic redirect? I'd have a bunch of regexs that would then redirect the traffic to an internal proxy and another set that would redirect it to a charles proxy running locally from where I could then redirect to the external proxy in work.

In the case below, any url with samdev goes to an internal proxy and
anything with 'google' goes to my charles proxy and out through our external proxy.

i'd build up these regex's like a PAC file once I have the redirects working.

You probably recognise the nginx.conf below, I got it from
https://github.com/kawanet/nginx-forward-proxy/blob/master/etc/nginx.conf

One last thing, I tried in vain to redirect the access/error logs to /dev/stdout and &/dev/stderr respectively but alas when nothing appeared in either using the docker command

docker logs -f reizman 2>dev/null
docker logs -f reizman 1>dev/null

user www-data;
worker_processes auto;
daemon off; # Don't run Nginx as daemon, as we run it in Docker we need a foreground process.
events { }

http {
  # google's DNS server
  resolver 8.8.8.8;
  resolver_timeout 5s;

  access_log /dev/stdout;
  error_log /dev/stderr;
  proxy_temp_path ./temp;
 
  server {
    # proxy server port
    listen 8888;

    location / {
      # internal http proxy
      set $internal_host "127.0.0.1";
      set $internal_port "8889";

      # external proxy
      set $external_host "127.0.0.1";
      set $external_port "8889";

      # proxy (default)
      set $proxy_host "$http_host";
      set $url "$scheme://$http_host$request_uri";

      # Internal traffic
      if ($host ~* "^samdev$") {
        set $proxy_host "$1";
        set $url "http://$internal_host:$internal_port$request_uri";
      }

      # External Traffic for anything google
      if ($http_host ~* "^google$") {
        set $proxy_host "$1:$2";
        set $url "http://$external_host:$external_port$request_uri";
      }

      proxy_redirect off;
      proxy_set_header Host $proxy_host;
      proxy_set_header X-Forwarded-Host $http_host;
      proxy_pass "$url";
    }
  }
}

Cache Settings for Particular domain

Hii Thanks for great one,

We are using reiz nginx forward proxy for our non internet system to pull updates from cloud server, we want enable cache for to serve multiple system since our bandwidth high.

Can you help us to enable the cache settings.

https proxy setting

the proxy is working for me but when setting the https_proxy it should point to the proxy using http not https!

correctly:
export https_proxy=http://proxyhost:8888

not working !!!!
export https_proxy=https://proxyhost:8888

at least from my experience it worked like that

How to use this with youtube-dl

getting this 400 when set proxy with youtube-dl

$ youtube-dl --proxy http://nginx_proxy:8888 -icwf best 'https://www.youtube.com/watch?v=Wk1oClYJE58' 
[youtube] Wk1oClYJE58: Downloading webpage
WARNING: Unable to download webpage: <urlopen error Tunnel connection failed: 400 Bad Request>
[youtube] Wk1oClYJE58: Downloading API JSON
ERROR: Unable to download API page: <urlopen error Tunnel connection failed: 400 Bad Request> (caused by URLError(OSError('Tunnel connection failed: 400 Bad Request')))

getting this in /var/log/nginx_errors.log

2021/08/31 17:53:23 [error] 7#7: *156 proxy_connect: connection error while connecting to upstream, client: 172.17.0.1, server: ~.+, request: "CONNECT firebaseinstallations.googleapis.com:443 HTTP/1.1", host: "firebaseinstallations.googleapis.com:443"
2021/08/31 17:53:23 [crit] 7#7: *167 connect() to [2607:f8b0:4003:c05::5f]:443 failed (99: Cannot assign requested address) while connecting to upstream, client: 172.17.0.1, server: ~.+, request: "CONNECT firebaseinstallations.googleapis.com:443 HTTP/1.1", host: "firebaseinstallations.googleapis.com:443"

I'm running this proxy with gluetun VPN, I'm sure that VPN is running properly.

version: "3.7"
services:
  gluetun:
    image: qmcgaw/gluetun
    container_name: gluetun
    restart: always
    cap_add:
      - NET_ADMIN
    network_mode: bridge
    ports:
      - 8888:8888 # Built-in HTTP control server
    volumes:
      - ./vpn:/gluetun
    environment:
      - VPNSP=nordvpn
      - OPENVPN_USER=
      - OPENVPN_PASSWORD=
      - TZ=Asia/Kolkata
  nginx:
    network_mode: "container:gluetun"
    restart: always
    image: reiz/nginx_proxy
    container_name: forward
    volumes:
      - ./nginx.conf:/usr/local/nginx/conf/nginx.conf
    environment:
      - TZ=Asia/Kolkata

Is there a way to configure one nginx-proxy forward requests to another nginx-proxy

Briefly cascading nginx-proxys.

This is the original example:

# stand alone nginx proxy
        location / {
           proxy_pass http://$http_host;
           proxy_set_header Host $http_host;
        }

And This is the cascading example:

# cascading nginx proxys, client -> A proxy -> B proxy -> target server

# A proxy, forwarding requests to B proxy
        location / {
           proxy_pass http://{B proxy endpoint};
           proxy_set_header Host $http_host;
        }

# B proxy, same as the example
        location / {
           proxy_pass http://$http_host;
           proxy_set_header Host $http_host;
        }

I tried with this, but it shows that B proxy is bypassed. It seems that the proxy_pass configuration is ignored.

manifest for reiz/nginx_proxy:0.0.1 not found

I follow the document from readme file. But got the following errors

# docker run reiz/nginx_proxy:0.0.1
Unable to find image 'reiz/nginx_proxy:0.0.1' locally
docker: Error response from daemon: manifest for reiz/nginx_proxy:0.0.1 not found.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.