mattes / rotating-proxy Goto Github PK
View Code? Open in Web Editor NEWRotating TOR proxy with Docker
License: The Unlicense
Rotating TOR proxy with Docker
License: The Unlicense
Is it safe to expose ports 4444 and 5566 to the internet? Will proxying work without listening to 0.0.0.0?
running as the pulled container, i'm getting a consistent 503 error and i'm not sure why. i've repulled multitple times and all the usual. the logs look find to me. Looking for suggestions on troubleshooting this.
server6:~ # curl --proxy torprox.docker:5566 http://www.google.ca
I have 2 docker containers running using
docker run -d -p 5566:5566 -p 4444:4444 --env tors=25 mattes/rotating-proxy
docker run -d -p 5567:5566 -p 4445:4444 --env tors=25 mattes/rotating-proxy
One on port 5566 and one on 5567 but they do not run concurrently, either one runs or the other.
I looked into the container with command docker exec -it <containerID> /bin/bash
and found there are 3 haproxy configurations
I also want to be able to stop start haproxy service, I believe there is a executable to run this /etc/init.d/haproxy How do I stop start the haproxy service?
Spec:
Server: Ubuntu 18.04 (running as VM)
Perhaps I'm reading this wrong but doing lsof -a -p <pid>
shows me a bunch of TCP filedescriptors my node program uses, while routing through rotating-proxy
. These all read as follows:
TCP 192.168.178.16:<var>->sandiego.ar01.host28.scalematrix.net:http (ESTABLISHED)
Shouldn't sandiego.ar01.host28.scalematrix.net
always be changing, or isn't this the exit node?
ubuntu is quite a heavy image. Is it possible to use alpine as a base image?
Hey dude,
I know normally you need to edit torrc for changing exit nodes to set countrys.
how would you do it on this?
would i fork it and then add a new torrc file or would i change something in the ruby file to choose the country exit nodes?
Wasn't able to get the HAProxy to accept only requests that contain user:password and use HTTPS.
By any chance anyone knows how to set it up?
I get curl: (56) Recv failure: Connection reset by peer
or curl: (56) Proxy CONNECT aborted
errors when I increase the number of tors to be greater than 25.
What are the required components on Linux
e.g ruby god tor
and the gems
Hi.
I set the tor to fork 50 processes so I can have 50 open ports.
The problem, the docker log file takes more than 80% of the server space! the total space is 917G, and the docker files are around 700 GB!
WAW. ps the docker has only once running container. which is rotating-proxy
even cleaning Docker dos not help. the only way to reclaim the wasting space is by uninstalling the docker and deleting the docker folder altogether.
Please advice
I wonder, if this is possible? like i wanted to use fresh IP every 1-3 hours but with limit of 5 tors. Until this much hours, IP may get blacklist
hello to all!
i have installed rotating-proxy from many months an works.
yesterday there are an update of docker and now i'm at version 2.1.0.0 (36873) channel edge.
after this update rotating-proxy not work! :-(
you can help me please?
For some weird reason i built a new rotating-proxy today and when using the same node.js code to do a request that i used on a previous build I'm getting the following error:
"SOCKS authentication failed. Unexpected number of bytes received."
Running it on an ubuntu image on Digital Ocean.
worth mentioning i had done exactly the same 1 year ago and had been running it as proxy to my node.js app for over an year and that never happened before.
is it possible that something has changed on ubuntu on the image that now requires additional configuration?
Sending build context to Docker daemon 115.2kB
Step 1/20 : FROM ubuntu:14.04
---> df043b4f0cf1
Step 2/20 : MAINTAINER Matthias Kadenbach [email protected]
---> Running in 85ebc99db8d0
Removing intermediate container 85ebc99db8d0
---> 13f27e6c1ec0
Step 3/20 : RUN echo 'deb http://deb.torproject.org/torproject.org trusty main' | tee /etc/apt/sources.list.d/torproject.list
---> Running in 24a07eb5da94
deb http://deb.torproject.org/torproject.org trusty main
Removing intermediate container 24a07eb5da94
---> 3f854f07a962
Step 4/20 : RUN gpg --keyserver keys.gnupg.net --recv 886DDD89
---> Running in 481e57515175
gpg: directory /root/.gnupg' created gpg: new configuration file
/root/.gnupg/gpg.conf' created
gpg: WARNING: options in /root/.gnupg/gpg.conf' are not yet active during this run gpg: keyring
/root/.gnupg/secring.gpg' created
gpg: keyring `/root/.gnupg/pubring.gpg' created
gpg: requesting key 886DDD89 from hkp server keys.gnupg.net
gpg: packet(13) too large
gpg: read_block: read error: invalid packet
gpg: Total number processed: 0
gpg: no valid OpenPGP data found.
The command '/bin/sh -c gpg --keyserver keys.gnupg.net --recv 886DDD89' returned a non-zero code: 2
When I run this container interactively everything works great. When I run it detached connections sometimes work and mostly doesn't. Nginx responds there was a problem so I assume the problem is related to one of the items in the chain of processes (nginx -> haproxy -> polipo -> tor) is having some issues running in non interactive mode.
I noticed a pattern:
$ docker run -d -p 5566:5566 -p 4444:4444 --env tors=25 mattes/rotating-proxy
a82c7efd44798f5d0686e372663a3302a2df429b810e8302a2c69435efb57e07
$ curl --proxy localhost:5566 http://www.example.com -I
HTTP/1.0 502 Bad Gateway
Cache-Control: no-cache
Connection: close
Content-Type: text/html
$ curl --proxy localhost:5566 http://www.example.com -I
HTTP/1.1 200 OK
Content-Length: 606
ETag: "359670651"
Date: Sun, 26 Mar 2017 08:56:23 GMT
Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
Expires: Sun, 02 Apr 2017 08:56:23 GMT
Content-Encoding: gzip
Accept-Ranges: bytes
Content-Type: text/html
Server: ECS (iad/182A)
X-Cache: HIT
Age: 1
Connection: keep-alive
Any idea why's the first time response 502 ?
Tnx,
Luka
Anyway to protect 5566 with a password?
Hi I tried to run a Raspberry rotatting-proxy, I got a problem with "the requested image platform (linux / amd64)" I used --platform linux/amd64
to start them, no problem appeared but the proxy does not work. Anyone know if it can be done or it just doesn't work? Thanks for the reply.
https://wiki.archlinux.org/index.php/Tor#TorDNS
The link above might be useful.
How can restart haproxy and tor while using this setup?
Hello,
I have to use a proxy to connect tor network, can not use this docker image directly .
I know how to set proxy to tor brower, but how do I set a proxy in ubuntu docker ?
Hi,
I installed the mattes/rotating-proxy and it working just fine form the host,
But how can I use the proxy inside another container ? what is the IP address I have to use
Been trying to deploy it to Heroku, having some issue with the setup.
Anyone tried deploying it on Heroku?
Running the suggested test from the readme doesnt work, because echoip.com doesnt exist
curl --proxy 127.0.0.1:5566 http://echoip.com
I would suggest to run:
curl --proxy 127.0.0.1:5566 http://ip.jsontest.com
Can you provide instructions on how to setup docker?
Do I install docker on the server (ubuntu) and then run these commands?
Stats are showing traffic in/out see http://prntscr.com/eyfw9q (on both attached/detached) however it doesn't return response (it seems) to haproxy.
I get a response (assume after timeout) of
<html><body><h1>502 Bad Gateway</h1>
The server returned an invalid or incomplete response.
</body></html>
This is exactly what i'm after and looks like an excellent dockerfile just seems to need patching upto date or im being stupid?
$ curl --proxy localhost:5566 http://httpbin.org/ip
curl: (56) Recv failure: Connection reset by peer
I create a container use the command
docker run -d -p 5566:5566 -p 9051:9051 --env tors=100 mattes/rotating-proxy
and use the python lib 'stem'
do something like that
from stem import Signal
from stem.control import Controlle
with Controller.from_port(port = 9051) as controller:
controller.authenticate()
controller.signal(Signal.NEWNYM)
then I got a error
stem.connection.AuthenticationFailure: socket connection failed ()
can anyone tell me how I can do ?
Running following command (multiple times) from host
curl --proxy 127.0.0.1:5566 http://echoip.com
results in...
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html><head>
<title>Proxy error: 504 Connect to echoip.com:80 failed: SOCKS error: host unreachable.</title>
</head><body>
<h1>504 Connect to echoip.com:80 failed: SOCKS error: host unreachable</h1>
<p>The following error occurred while trying to access <strong>http://echoip.com/</strong>:<br><br>
<strong>504 Connect to echoip.com:80 failed: SOCKS error: host unreachable</strong></p>
<hr>Generated Fri, 14 Feb 2020 15:54:33 UTC by Polipo on <em>0a12f1e025a0:20016</em>.
</body></html>
Tasks: 55 total, 1 running, 54 sleeping, 0 stopped, 0 zombie
%Cpu(s): 2.6 us, 2.6 sy, 0.0 ni, 92.1 id, 2.6 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem: 8132020 total, 4542468 used, 3589552 free, 36124 buffers
KiB Swap: 1048572 total, 0 used, 1048572 free. 2553736 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
437 root 20 0 53828 31072 6248 S 5.3 0.4 0:01.61 tor
1 root 20 0 4448 676 600 S 0.0 0.0 0:00.01 sh
6 root 20 0 1089804 17188 6444 S 0.0 0.2 0:01.42 ruby
32 root 20 0 7136 1580 1416 S 0.0 0.0 0:00.00 polipo
49 root 20 0 7136 1688 1528 S 0.0 0.0 0:00.00 polipo
60 root 20 0 7136 1408 1248 S 0.0 0.0 0:00.00 polipo
64 root 20 0 53888 31156 6300 S 0.0 0.4 0:01.44 tor
68 root 20 0 53804 31316 6452 S 0.0 0.4 0:01.30 tor
82 root 20 0 7136 1636 1472 S 0.0 0.0 0:00.00 polipo
89 root 20 0 7136 1612 1448 S 0.0 0.0 0:00.00 polipo
101 root 20 0 53828 31112 6240 S 0.0 0.4 0:01.42 tor
109 root 20 0 7136 1544 1380 S 0.0 0.0 0:00.00 polipo
125 root 20 0 7136 1416 1248 S 0.0 0.0 0:00.00 polipo
126 root 20 0 53948 31116 6248 S 0.0 0.4 0:01.49 tor
139 root 20 0 53724 31100 6272 S 0.0 0.4 0:01.45 tor
149 root 20 0 7136 1420 1252 S 0.0 0.0 0:00.00 polipo
159 root 20 0 7136 1544 1380 S 0.0 0.0 0:00.00 polipo
160 root 20 0 54168 31484 6404 S 0.0 0.4 0:01.44 tor
176 root 20 0 53692 30900 6064 S 0.0 0.4 0:01.45 tor
183 root 20 0 7136 1544 1380 S 0.0 0.0 0:00.00 polipo
192 root 20 0 53840 31148 6260 S 0.0 0.4 0:01.43 tor
199 root 20 0 7136 1688 1528 S 0.0 0.0 0:00.00 polipo
213 root 20 0 7136 1692 1528 S 0.0 0.0 0:00.00 polipo
222 root 20 0 53792 31020 6180 S 0.0 0.4 0:01.47 tor
237 root 20 0 7136 1604 1444 S 0.0 0.0 0:00.00 polipo
248 root 20 0 7136 1580 1416 S 0.0 0.0 0:00.00 polipo
256 root 20 0 7136 1408 1244 S 0.0 0.0 0:00.00 polipo
263 root 20 0 53872 31184 6352 S 0.0 0.4 0:01.53 tor
267 root 20 0 54264 31492 6352 S 0.0 0.4 0:01.52 tor
275 root 20 0 7136 1608 1444 S 0.0 0.0 0:00.00 polipo
290 root 20 0 7136 1468 1304 S 0.0 0.0 0:00.00 polipo
306 root 20 0 53740 31156 6296 S 0.0 0.4 0:01.50 tor
307 root 20 0 53696 31036 6144 S 0.0 0.4 0:01.48 tor
314 root 20 0 7136 1636 1472 S 0.0 0.0 0:00.00 polipo
318 root 20 0 54012 31116 6268 S 0.0 0.4 0:01.45 tor
330 root 20 0 7136 1412 1248 S 0.0 0.0 0:00.00 polipo
339 root 20 0 53812 30944 6112 S 0.0 0.4 0:01.34 tor
350 root 20 0 7136 1616 1456 S 0.0 0.0 0:00.00 polipo
363 root 20 0 53740 30860 6000 S 0.0 0.4 0:01.52 tor
367 root 20 0 7136 1580 1416 S 0.0 0.0 0:00.00 polipo
369 root 20 0 53964 31144 6264 S 0.0 0.4 0:01.53 tor
378 root 20 0 7136 1612 1444 S 0.0 0.0 0:00.00 polipo
380 root 20 0 53916 31316 6436 S 0.0 0.4 0:01.51 tor
399 root 20 0 7136 1696 1528 S 0.0 0.0 0:00.00 polipo
400 root 20 0 53768 31144 6260 S 0.0 0.4 0:01.55 tor
414 root 20 0 7136 1616 1456 S 0.0 0.0 0:00.00 polipo
427 root 20 0 53728 31332 6480 S 0.0 0.4 0:01.48 tor
428 root 20 0 7136 1412 1248 S 0.0 0.0 0:00.00 polipo
431 root 20 0 53924 31152 6272 S 0.0 0.4 0:01.50 tor
432 root 20 0 11788 840 320 S 0.0 0.0 0:00.05 haproxy
434 root 20 0 54148 31212 6388 S 0.0 0.4 0:01.31 tor
438 root 20 0 54184 31468 6388 S 0.0 0.4 0:01.51 tor
440 root 20 0 53860 31160 6324 S 0.0 0.4 0:01.44 tor
3673 root 20 0 18168 3176 2876 S 0.0 0.0 0:00.01 bash
3687 root 20 0 19876 2536 2092 R 0.0 0.0 0:00.01 top
any suggestions?
monit has some issues with correct uid/gid permissions. especially when running scrappy. needs to be checked.
Hi, code need to be updated to use version 3 onion services?
As I'm not fluent in ruby ^^ I wonder if it's possible to run this image with a custom list of proxies if I don't need Tor.
Does someone know if an existing fork has been made this way ?
Hello i have multiple backends i will map by domain name http mapping works https not, how to modify the haproxy.cfg file?
Hi. It seems there is some problem with running your project in Windows. I've pulled it in Docker, started it as the instruction says, but tor doesn`t make outgoing connections. I can see in HAProxy stats that requests come to it, but then nothing happens. In Linux the projects with the same settings work in proper way. What can it be and can it be fixed? (And thanks for your work, by the way :) )
First of all thank you for the amazing project. Great work !
I have noticed that most of the requests are processed very more slowly than others (some times they are very slow > 1 minute). As I understand the speed of request processing depends on the number of tors that I'm using. But in other hand more tors result in better security (traffic obfuscation). What is the best ratio for speed/security (speed/tors number) ? Also in my understanding the tor circus is using local computer resources ( CPU and RAM ) to obfuscate the traffic. Does this mean that if I have more computing power the requests will be processed faster ?
How can I use it with Scrapy, Any sample code pls
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.