vadimeisenbergibm / envoy-generic-forward-proxy Goto Github PK

https://github.com/vadimeisenbergibm/envoy-generic-forward-proxy/blob/master/envoy_forward_proxy/envoy_config.json

Is this config implements forward proxy similar nto squid proxy?

Example, I have 10 clients which needs to access Internet via envoy proxy. Is it possible? This can be done in squid.

In this config, what are those 8081 and 8082 ? I'd this some external proxy server?

This config looks like just a port translation

@vadimeisenbergibm

@alejandropal @dvillanustre Sorry for the delayed response, I was very busy this week. Please note that orig_dst cluster can only work in the same pod/container. In your case you cannot use it.

I would recommend you the following - create a regular cluster with www.google.com as a target. Check that it works.

If you need envoy to access arbitrary sites, you can use the solution I described in the README of this repo: https://github.com/vadimeisenbergibm/envoy-generic-forward-proxy#envoy-as-a-generic-forward-proxy-to-other-pods. You would need to deploy an Nginx proxy in addition to Envoy, and use the Nginx's address as the destination of your Envoy's cluster.

@vadimeisenbergibm - Good rep.., attach the docker-compose file , also looks like there is an issue with
https://github.com/vadimeisenbergibm/envoy-generic-forward-proxy/tree/master/envoy_forward_proxy/envoy_config.json file.
help me in correcting it.
FYI, i am trying set-up envoy as front-proxy in docker.

PFB docker-compose.yml

version: '2'
services:
envoy:
build:
context: .
dockerfile: Dockerfile
ports:
- "80:80"
- "443:443"
- "8001:8001"
expose:
- "80"
- "443"
- "8001"

Logs:

Building envoy
Step 1/8 : FROM /envoy-dev:latest
latest: Pulling from envoy-dev
34667c7e4631: Already exists
d18d76a881a4: Already exists
119c7358fbfc: Already exists
2aaf13f3eff0: Already exists
1750bd4d707f: Pull complete
6dbf34881af8: Pull complete
597e8e616785: Pull complete
59b3f5a6b6e2: Pull complete
978686368e74: Pull complete
Digest: sha256:954a3a1b4865600ca2bf869b2a80f948430e008870872c10b0267fb19e18932a
Status: Downloaded newer image for :latest
---> 32d7dd074ce0
Step 2/8 : ENV HOME /root
---> Running in 97127650ef1d
Removing intermediate container 97127650ef1d
---> e64cbfb49113
Step 3/8 : WORKDIR $HOME
Removing intermediate container aa9af3c46d93
---> 34e05d10fd28
Step 4/8 : COPY envoy_config.json $HOME
---> 8f1d665d61d6
Step 5/8 : EXPOSE 10000
---> Running in 5de2b2bfe7b2
Removing intermediate container 5de2b2bfe7b2
---> 3630f4a0ea32
Step 6/8 : EXPOSE 80
---> Running in 5e7d241323ce
Removing intermediate container 5e7d241323ce
---> 038dcd95de8a
Step 7/8 : EXPOSE 443
---> Running in 3b9eea0ef36a
Removing intermediate container 3b9eea0ef36a
---> 571ad62d1dbc
Step 8/8 : CMD envoy -c envoy_config.json
---> Running in c1a5d97df97d
Removing intermediate container c1a5d97df97d
---> 6dae250a9168

Successfully built 6dae250a9168
Successfully tagged envoy_envoy:latest
WARNING: Image for service envoy was built because it did not already exist. To rebuild this image you must use docker-compose build or docker-compose up --build.
Creating envoy_envoy_1 ... done
Attaching to envoy_envoy_1
envoy_1 | [2019-07-02 09:53:18.776][6][info][main] [source/server/server.cc:205] initializing epoch 0 (hot restart version=10.200.16384.127.options=capacity=16384, num_slots=8209 hash=228984379728933
363 size=2654312)
envoy_1 | [2019-07-02 09:53:18.776][6][info][main] [source/server/server.cc:207] statically linked extensions:
envoy_1 | [2019-07-02 09:53:18.776][6][info][main] [source/server/server.cc:209] access_loggers: envoy.file_access_log,envoy.http_grpc_access_log
envoy_1 | [2019-07-02 09:53:18.776][6][info][main] [source/server/server.cc:212] filters.http: envoy.buffer,envoy.cors,envoy.ext_authz,envoy.fault,envoy.filters.http.grpc_http1_reverse_bridge,envoy
.filters.http.header_to_metadata,envoy.filters.http.jwt_authn,envoy.filters.http.rbac,envoy.filters.http.tap,envoy.grpc_http1_bridge,envoy.grpc_json_transcoder,envoy.grpc_web,envoy.gzip,envoy.health_c
heck,envoy.http_dynamo_filter,envoy.ip_tagging,envoy.lua,envoy.rate_limit,envoy.router,envoy.squash
envoy_1 | [2019-07-02 09:53:18.776][6][info][main] [source/server/server.cc:215] filters.listener: envoy.listener.original_dst,envoy.listener.original_src,envoy.listener.proxy_protocol,envoy.listen
er.tls_inspector
envoy_1 | [2019-07-02 09:53:18.776][6][info][main] [source/server/server.cc:218] filters.network: envoy.client_ssl_auth,envoy.echo,envoy.ext_authz,envoy.filters.network.dubbo_proxy,envoy.filters.ne
twork.mysql_proxy,envoy.filters.network.rbac,envoy.filters.network.sni_cluster,envoy.filters.network.thrift_proxy,envoy.filters.network.zookeeper_proxy,envoy.http_connection_manager,envoy.mongo_proxy,
envoy.ratelimit,envoy.redis_proxy,envoy.tcp_proxy
envoy_1 | [2019-07-02 09:53:18.776][6][info][main] [source/server/server.cc:220] stat_sinks: envoy.dog_statsd,envoy.metrics_service,envoy.stat_sinks.hystrix,envoy.statsd
envoy_1 | [2019-07-02 09:53:18.776][6][info][main] [source/server/server.cc:222] tracers: envoy.dynamic.ot,envoy.lightstep,envoy.tracers.datadog,envoy.zipkin
envoy_1 | [2019-07-02 09:53:18.776][6][info][main] [source/server/server.cc:225] transport_sockets.downstream: envoy.transport_sockets.alts,envoy.transport_sockets.tap,raw_buffer,tls
envoy_1 | [2019-07-02 09:53:18.776][6][info][main] [source/server/server.cc:228] transport_sockets.upstream: envoy.transport_sockets.alts,envoy.transport_sockets.tap,raw_buffer,tls
envoy_1 | [2019-07-02 09:53:18.776][6][info][main] [source/server/server.cc:234] buffer implementation: old (libevent)
envoy_1 | [2019-07-02 09:53:18.777][6][critical][main] [source/server/server.cc:90] error initializing configuration 'envoy_config.json': Unable to parse JSON as proto (INVALID_ARGUMENT:(cluster_mana
ger) clusters: Cannot find field.): {
envoy_1 | "listeners": [
envoy_1 | {
envoy_1 | "address": "tcp://0.0.0.0:80",
envoy_1 | "filters": [
envoy_1 | {
envoy_1 | "type": "read",
envoy_1 | "name": "http_connection_manager",
envoy_1 | "config": {
envoy_1 | "codec_type": "auto",
envoy_1 | "stat_prefix": "forward_http",
envoy_1 | "http1_settings": {
envoy_1 | "allow_absolute_url": true

Originally posted by @ravikumarkgit in #1 (comment)

Looks like there is an issue in the config file. I'm trying to use Envoy as forward proxy but if I use the config it says 'invalid option cluster_manager'

hello! I'm trying to implement an outbound transparent proxy with envoy, running in different containers.

I'm using this configuration:

  "listeners": [
    {
      "address": "tcp://0.0.0.0:80",
      "bind_to_port": false,
      "filters": [
        {
          "type": "read",
          "name": "http_connection_manager",
          "config": {
            "access_log": [
                {
                "path": "/tmp/envoy.log"
                }
            ],
            "codec_type": "auto",
            "stat_prefix": "forward_http",
            "route_config": {
              "virtual_hosts": [
                {
                  "name": "default_http",
                  "domains": ["*"],
                  "routes": [
                    {
                      "timeout_ms": 0,
                      "prefix": "/",
                      "cluster": "outbound_forward_proxy_http"
                    }
                  ]
                }
              ]
            },
            "filters": [
              {
                "type": "decoder",
                "name": "router",
                "config": {}
              }
            ]
          }
        }
      ]
    },
    {
        "address": "tcp://0.0.0.0:15001",
        "filters": [],
        "bind_to_port": true,
        "use_original_dst": true
    }
  ],
  "admin": {
    "access_log_path": "/tmp/access_log",
    "address": "tcp://0.0.0.0:8001"
  },
  "cluster_manager": {
      "clusters": [
          {
              "name": "outbound_forward_proxy_http",
              "connect_timeout_ms": 2500,
              "type": "original_dst",
              "lb_type": "original_dst_lb"
          }
    ]
  }
}

This is an docker-compose example yaml:

version: '2'
services:
  envoy:
    build:
      context: ./envoy
      dockerfile: Dockerfile-envoy
    cap_add:
      - NET_ADMIN
    ports:
      - "80:80"
    expose:
      - "80"
  application:
    build:
      context: ./api
      dockerfile: Dockerfile-api
    cap_add:
      - NET_ADMIN

I use IPTABLES in order to redirect all the traffic outgoing from application to envoy. If I use locally (the http requester application and envoy are in the same container) it works like a charm.

In the other hand, if I run envoy in a different container, I receive this message from envoy when I redirect the application requests:

upstream connect error or disconnect/reset before headers. The request arrives to envoy ( I can see in the access_log.

My IPTABLES rules in envoy:

iptables -t nat -N ISTIO_REDIRECT
iptables -t nat -A ISTIO_REDIRECT -p tcp -j REDIRECT --to-port $ENVOY_PORT
iptables -t nat -A PREROUTING -j ISTIO_REDIRECT

iptables -t nat -N ISTIO_OUTPUT
iptables -t nat -A OUTPUT -p tcp -j ISTIO_OUTPUT
iptables -t nat -A ISTIO_OUTPUT -m owner --uid-owner ${ENVOY_UID} -j RETURN
iptables -t nat -A ISTIO_OUTPUT -j ISTIO_REDIRECT.

In application:
iptables -t nat -I OUTPUT -p tcp --dport 80 -j DNAT --to-destination $ENVOY_IP:80

Could you give me an advice how can I configure envoy as a transparent proxy running in another container?

Thanks in advance!