First of all thank you for the amazing project. Great work !

I have noticed that most of the requests are processed very more slowly than others (some times they are very slow > 1 minute). As I understand the speed of request processing depends on the number of tors that I'm using. But in other hand more tors result in better security (traffic obfuscation). What is the best ratio for speed/security (speed/tors number) ? Also in my understanding the tor circus is using local computer resources ( CPU and RAM ) to obfuscate the traffic. Does this mean that if I have more computing power the requests will be processed faster ?

Is it safe to expose ports 4444 and 5566 to the internet? Will proxying work without listening to 0.0.0.0?

Hey dude,

I know normally you need to edit torrc for changing exit nodes to set countrys.

how would you do it on this?

would i fork it and then add a new torrc file or would i change something in the ruby file to choose the country exit nodes?

Im getting this error at 4/20

I wonder, if this is possible? like i wanted to use fresh IP every 1-3 hours but with limit of 5 tors. Until this much hours, IP may get blacklist

Perhaps I'm reading this wrong but doing lsof -a -p <pid> shows me a bunch of TCP filedescriptors my node program uses, while routing through rotating-proxy. These all read as follows:

TCP 192.168.178.16:<var>->sandiego.ar01.host28.scalematrix.net:http (ESTABLISHED)

Shouldn't sandiego.ar01.host28.scalematrix.net always be changing, or isn't this the exit node?

Anyway to protect 5566 with a password?

ubuntu is quite a heavy image. Is it possible to use alpine as a base image?

Hello i have multiple backends i will map by domain name http mapping works https not, how to modify the haproxy.cfg file?

Sending build context to Docker daemon 115.2kB
Step 1/20 : FROM ubuntu:14.04
---> df043b4f0cf1
Step 2/20 : MAINTAINER Matthias Kadenbach [email protected]
---> Running in 85ebc99db8d0
Removing intermediate container 85ebc99db8d0
---> 13f27e6c1ec0
Step 3/20 : RUN echo 'deb http://deb.torproject.org/torproject.org trusty main' | tee /etc/apt/sources.list.d/torproject.list
---> Running in 24a07eb5da94
deb http://deb.torproject.org/torproject.org trusty main
Removing intermediate container 24a07eb5da94
---> 3f854f07a962
Step 4/20 : RUN gpg --keyserver keys.gnupg.net --recv 886DDD89
---> Running in 481e57515175
gpg: directory /root/.gnupg' created gpg: new configuration file /root/.gnupg/gpg.conf' created
gpg: WARNING: options in /root/.gnupg/gpg.conf' are not yet active during this run gpg: keyring /root/.gnupg/secring.gpg' created
gpg: keyring `/root/.gnupg/pubring.gpg' created
gpg: requesting key 886DDD89 from hkp server keys.gnupg.net
gpg: packet(13) too large
gpg: read_block: read error: invalid packet
gpg: Total number processed: 0
gpg: no valid OpenPGP data found.
The command '/bin/sh -c gpg --keyserver keys.gnupg.net --recv 886DDD89' returned a non-zero code: 2

Hello,

I have to use a proxy to connect tor network, can not use this docker image directly .
I know how to set proxy to tor brower, but how do I set a proxy in ubuntu docker ?

I create a container use the command
docker run -d -p 5566:5566 -p 9051:9051 --env tors=100 mattes/rotating-proxy

and use the python lib 'stem'
do something like that

from stem import Signal
from stem.control import Controlle

with Controller.from_port(port = 9051) as controller:
    controller.authenticate()
    controller.signal(Signal.NEWNYM)

then I got a error
stem.connection.AuthenticationFailure: socket connection failed ()

can anyone tell me how I can do ?

Hi, code need to be updated to use version 3 onion services?

Hi. It seems there is some problem with running your project in Windows. I've pulled it in Docker, started it as the instruction says, but tor doesn`t make outgoing connections. I can see in HAProxy stats that requests come to it, but then nothing happens. In Linux the projects with the same settings work in proper way. What can it be and can it be fixed? (And thanks for your work, by the way :) )

How can I use it with Scrapy, Any sample code pls

running as the pulled container, i'm getting a consistent 503 error and i'm not sure why. i've repulled multitple times and all the usual. the logs look find to me. Looking for suggestions on troubleshooting this.

server6:~ # curl --proxy torprox.docker:5566 http://www.google.ca

503 Service Unavailable

As I'm not fluent in ruby ^^ I wonder if it's possible to run this image with a custom list of proxies if I don't need Tor.
Does someone know if an existing fork has been made this way ?

Been trying to deploy it to Heroku, having some issue with the setup.
Anyone tried deploying it on Heroku?

Stats are showing traffic in/out see http://prntscr.com/eyfw9q (on both attached/detached) however it doesn't return response (it seems) to haproxy.

I get a response (assume after timeout) of

<html><body><h1>502 Bad Gateway</h1>
The server returned an invalid or incomplete response.
</body></html>

This is exactly what i'm after and looks like an excellent dockerfile just seems to need patching upto date or im being stupid?

Running following command (multiple times) from host

curl --proxy 127.0.0.1:5566 http://echoip.com

results in...

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html><head>
<title>Proxy error: 504 Connect to echoip.com:80 failed: SOCKS error: host unreachable.</title>
</head><body>
<h1>504 Connect to echoip.com:80 failed: SOCKS error: host unreachable</h1>
<p>The following error occurred while trying to access <strong>http://echoip.com/</strong>:<br><br>
<strong>504 Connect to echoip.com:80 failed: SOCKS error: host unreachable</strong></p>
<hr>Generated Fri, 14 Feb 2020 15:54:33 UTC by Polipo on <em>0a12f1e025a0:20016</em>.
</body></html>

top

Tasks:  55 total,   1 running,  54 sleeping,   0 stopped,   0 zombie
%Cpu(s):  2.6 us,  2.6 sy,  0.0 ni, 92.1 id,  2.6 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:   8132020 total,  4542468 used,  3589552 free,    36124 buffers
KiB Swap:  1048572 total,        0 used,  1048572 free.  2553736 cached Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
  437 root      20   0   53828  31072   6248 S   5.3  0.4   0:01.61 tor
    1 root      20   0    4448    676    600 S   0.0  0.0   0:00.01 sh
    6 root      20   0 1089804  17188   6444 S   0.0  0.2   0:01.42 ruby
   32 root      20   0    7136   1580   1416 S   0.0  0.0   0:00.00 polipo
   49 root      20   0    7136   1688   1528 S   0.0  0.0   0:00.00 polipo
   60 root      20   0    7136   1408   1248 S   0.0  0.0   0:00.00 polipo
   64 root      20   0   53888  31156   6300 S   0.0  0.4   0:01.44 tor
   68 root      20   0   53804  31316   6452 S   0.0  0.4   0:01.30 tor
   82 root      20   0    7136   1636   1472 S   0.0  0.0   0:00.00 polipo
   89 root      20   0    7136   1612   1448 S   0.0  0.0   0:00.00 polipo
  101 root      20   0   53828  31112   6240 S   0.0  0.4   0:01.42 tor
  109 root      20   0    7136   1544   1380 S   0.0  0.0   0:00.00 polipo
  125 root      20   0    7136   1416   1248 S   0.0  0.0   0:00.00 polipo
  126 root      20   0   53948  31116   6248 S   0.0  0.4   0:01.49 tor
  139 root      20   0   53724  31100   6272 S   0.0  0.4   0:01.45 tor
  149 root      20   0    7136   1420   1252 S   0.0  0.0   0:00.00 polipo
  159 root      20   0    7136   1544   1380 S   0.0  0.0   0:00.00 polipo
  160 root      20   0   54168  31484   6404 S   0.0  0.4   0:01.44 tor
  176 root      20   0   53692  30900   6064 S   0.0  0.4   0:01.45 tor
  183 root      20   0    7136   1544   1380 S   0.0  0.0   0:00.00 polipo
  192 root      20   0   53840  31148   6260 S   0.0  0.4   0:01.43 tor
  199 root      20   0    7136   1688   1528 S   0.0  0.0   0:00.00 polipo
  213 root      20   0    7136   1692   1528 S   0.0  0.0   0:00.00 polipo
  222 root      20   0   53792  31020   6180 S   0.0  0.4   0:01.47 tor
  237 root      20   0    7136   1604   1444 S   0.0  0.0   0:00.00 polipo
  248 root      20   0    7136   1580   1416 S   0.0  0.0   0:00.00 polipo
  256 root      20   0    7136   1408   1244 S   0.0  0.0   0:00.00 polipo
  263 root      20   0   53872  31184   6352 S   0.0  0.4   0:01.53 tor
  267 root      20   0   54264  31492   6352 S   0.0  0.4   0:01.52 tor
  275 root      20   0    7136   1608   1444 S   0.0  0.0   0:00.00 polipo
  290 root      20   0    7136   1468   1304 S   0.0  0.0   0:00.00 polipo
  306 root      20   0   53740  31156   6296 S   0.0  0.4   0:01.50 tor
  307 root      20   0   53696  31036   6144 S   0.0  0.4   0:01.48 tor
  314 root      20   0    7136   1636   1472 S   0.0  0.0   0:00.00 polipo
  318 root      20   0   54012  31116   6268 S   0.0  0.4   0:01.45 tor
  330 root      20   0    7136   1412   1248 S   0.0  0.0   0:00.00 polipo
  339 root      20   0   53812  30944   6112 S   0.0  0.4   0:01.34 tor
  350 root      20   0    7136   1616   1456 S   0.0  0.0   0:00.00 polipo
  363 root      20   0   53740  30860   6000 S   0.0  0.4   0:01.52 tor
  367 root      20   0    7136   1580   1416 S   0.0  0.0   0:00.00 polipo
  369 root      20   0   53964  31144   6264 S   0.0  0.4   0:01.53 tor
  378 root      20   0    7136   1612   1444 S   0.0  0.0   0:00.00 polipo
  380 root      20   0   53916  31316   6436 S   0.0  0.4   0:01.51 tor
  399 root      20   0    7136   1696   1528 S   0.0  0.0   0:00.00 polipo
  400 root      20   0   53768  31144   6260 S   0.0  0.4   0:01.55 tor
  414 root      20   0    7136   1616   1456 S   0.0  0.0   0:00.00 polipo
  427 root      20   0   53728  31332   6480 S   0.0  0.4   0:01.48 tor
  428 root      20   0    7136   1412   1248 S   0.0  0.0   0:00.00 polipo
  431 root      20   0   53924  31152   6272 S   0.0  0.4   0:01.50 tor
  432 root      20   0   11788    840    320 S   0.0  0.0   0:00.05 haproxy
  434 root      20   0   54148  31212   6388 S   0.0  0.4   0:01.31 tor
  438 root      20   0   54184  31468   6388 S   0.0  0.4   0:01.51 tor
  440 root      20   0   53860  31160   6324 S   0.0  0.4   0:01.44 tor
 3673 root      20   0   18168   3176   2876 S   0.0  0.0   0:00.01 bash
 3687 root      20   0   19876   2536   2092 R   0.0  0.0   0:00.01 top

any suggestions?

Haproxy stats

I noticed a pattern:

1. start container

$ docker run -d -p 5566:5566 -p 4444:4444 --env tors=25 mattes/rotating-proxy
a82c7efd44798f5d0686e372663a3302a2df429b810e8302a2c69435efb57e07

2. send request

$ curl --proxy localhost:5566 http://www.example.com -I
HTTP/1.0 502 Bad Gateway
Cache-Control: no-cache
Connection: close
Content-Type: text/html

3. send request after the first one returned 502:

$ curl --proxy localhost:5566 http://www.example.com -I
HTTP/1.1 200 OK
Content-Length: 606
ETag: "359670651"
Date: Sun, 26 Mar 2017 08:56:23 GMT
Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
Expires: Sun, 02 Apr 2017 08:56:23 GMT
Content-Encoding: gzip
Accept-Ranges: bytes
Content-Type: text/html
Server: ECS (iad/182A)
X-Cache: HIT
Age: 1
Connection: keep-alive

Any idea why's the first time response 502 ?

Tnx,
Luka

When I run this container interactively everything works great. When I run it detached connections sometimes work and mostly doesn't. Nginx responds there was a problem so I assume the problem is related to one of the items in the chain of processes (nginx -> haproxy -> polipo -> tor) is having some issues running in non interactive mode.

Running the suggested test from the readme doesnt work, because echoip.com doesnt exist
curl --proxy 127.0.0.1:5566 http://echoip.com
I would suggest to run:
curl --proxy 127.0.0.1:5566 http://ip.jsontest.com

Hello,

request details on hover in the stats section are not displayed. Any solution?

monit has some issues with correct uid/gid permissions. especially when running scrappy. needs to be checked.

• Privoxy can remove ads
• Polipo uses caching
  Interested in understanding why that choice was made.

For some weird reason i built a new rotating-proxy today and when using the same node.js code to do a request that i used on a previous build I'm getting the following error:

"SOCKS authentication failed. Unexpected number of bytes received."

Running it on an ubuntu image on Digital Ocean.

worth mentioning i had done exactly the same 1 year ago and had been running it as proxy to my node.js app for over an year and that never happened before.

is it possible that something has changed on ubuntu on the image that now requires additional configuration?

Wasn't able to get the HAProxy to accept only requests that contain user:password and use HTTPS.
By any chance anyone knows how to set it up?

Can you provide instructions on how to setup docker?

Do I install docker on the server (ubuntu) and then run these commands?

https://wiki.archlinux.org/index.php/Tor#TorDNS
The link above might be useful.

Hi,
I installed the mattes/rotating-proxy and it working just fine form the host,
But how can I use the proxy inside another container ? what is the IP address I have to use

Hi I tried to run a Raspberry rotatting-proxy, I got a problem with "the requested image platform (linux / amd64)" I used --platform linux/amd64 to start them, no problem appeared but the proxy does not work. Anyone know if it can be done or it just doesn't work? Thanks for the reply.

I have 2 docker containers running using
docker run -d -p 5566:5566 -p 4444:4444 --env tors=25 mattes/rotating-proxy
docker run -d -p 5567:5566 -p 4445:4444 --env tors=25 mattes/rotating-proxy

One on port 5566 and one on 5567 but they do not run concurrently, either one runs or the other.
I looked into the container with command docker exec -it <containerID> /bin/bash and found there are 3 haproxy configurations

1. /etc/haproxy/haproxy.cfg – doesn’t define frontend or backend definitions
   
2. /usr/local/etc/haproxy.cfg.erb – defines frontend and but does not specify binding port
3. /usr/local/etc/haproxy.cfg – defines frontend binding port 5566
   I want to be able to define the ports to run on inside the docker container and I believe I should set the binding port in file /usr/local/etc/haproxy.cfg is this correct?

I also want to be able to stop start haproxy service, I believe there is a executable to run this /etc/init.d/haproxy How do I stop start the haproxy service?

Spec:
Server: Ubuntu 18.04 (running as VM)

Hi.

I set the tor to fork 50 processes so I can have 50 open ports.
The problem, the docker log file takes more than 80% of the server space! the total space is 917G, and the docker files are around 700 GB!
WAW. ps the docker has only once running container. which is rotating-proxy
even cleaning Docker dos not help. the only way to reclaim the wasting space is by uninstalling the docker and deleting the docker folder altogether.

Please advice

How can restart haproxy and tor while using this setup?

hello to all!
i have installed rotating-proxy from many months an works.

yesterday there are an update of docker and now i'm at version 2.1.0.0 (36873) channel edge.
after this update rotating-proxy not work! :-(

you can help me please?

I get curl: (56) Recv failure: Connection reset by peer or curl: (56) Proxy CONNECT aborted errors when I increase the number of tors to be greater than 25.

What are the required components on Linux

e.g ruby god tor

and the gems