nfriedly / node-unblocker Goto Github PK

require('http').globalAgent.maxSockets = 64;

https://github.com/substack/hyperquest#rant

Greetings,

I am testing latest git.Sometime using a search engine google.com your module redirect me to wrong url, (http://localhost/url?sa=t&rct=j&q=&esrc=s&source=web&cd=3&ved=0CDwQFjACahUKEwjsv-r014LHAhUBhSwKHbq7AYo&url=%2Fproxy%2Fhttp%3A%2F%2Fwww.apple.com%2Fru%2Fstevejobs%2F&ei=1wG6Vaz4HoGKsgG694bQCA&usg=AFQjCNELu0sMKoBA1ZZwj5D57TXfU17AyA&bvm=bv.99028883,d.bGg instead of allaboutstevejobs.com/) to reproduce the problem you need to go to the google, search Steave Jobs and click on link "all about Steve Jobs.com"

The following HTTPS url loads without problems:

https://www.optimcore.com/proxy/http://example.com/

However if the final trailing slash is omitted the request is redirected to HTTP

https://www.optimcore.com/proxy/http://example.com -> http://www.optimcore.com/proxy/http://example.com/

Redirects like these cause Mixed Content errors on HTTPS pages such the below error when requesting:

https://www.optimcore.com/proxy/http://www.bbc.co.uk/

Mixed Content: The page at 'https://www.optimcore.com/proxy/http://www.bbc.co.uk/' was loaded over HTTPS, but requested an insecure script 'http://www.optimcore.com/proxy/https://fig.bbc.co.uk/frameworks/fig/1/fig.js'. This request has been blocked; the content must be served over HTTPS.

I recently encountered an error:
`
events.js:160
throw er; // Unhandled 'error' event
^

Error: unexpected end of file
at Zlib.__dirname.Zlib.mode._handle.onerror (zlib.js:370:17)
`
Please help me in fixing this!!

Hi @hellais, @emilhem, and @tfMen,

I received a request for an alternate-licensed version of node-unblocker for inclusion in a project that's not compatible with the GPL. (node-unblocker is currently GPL-licensed, which applies to your contributions as well.) Are you guys willing to dual-license your contributions as GPL and MIT?

i edit the config
using my hostname without port
using my ip host

but when i run it ,that app break

[root@test node-unblocker]# node debug app.js
< debugger listening on port 5858
connecting... ok
break in app.js:13
11
12 // native imports
13 var http = require('http'),
14 https = require('https'),
15 url = require('url'),
debug>

I tried this app for loading Google pages in iframe.But it dont show anything ..... iframe is just blank.What might be causing this error and what could be the fix?

Google is having all kinds of fun on my spare copy of the internet. Tell it to stop.

You may already know, the webapp doesn't work anymore

Heroku | No such app
There is no app configured at that hostname.
Perhaps the app owner has renamed it, or you mistyped the URL.

Cheers,

http://2.testhellp.appspot.com/html/whoami?only=ip

Hey again,
this Website also breaks the proxy in the way, that links point to the wrong location:

http://node-unblocker.herokuapp.com/proxy/http://www.bytebybytesolutions.com/BBB/Blog/Blog.html

Every Blogpost is linking to a link like this

http://node-unblocker.herokuapp.com/proxy/http://www.bytebybytesolutions.com/BBB/Blog/http://bytebybytesolutions.com/BBB/Blog/Entries/2013/4/17_Tutorial_-_Reusable_Game_Center_Singleton.html

Thanks for the grat service.

Hello !

I fallowed steps from "installation on your system" and after npm start
I'm seeing fallowing error:

MBP-Mateusz:node-unblocker mateusz$ npm start

> [email protected] start /Users/mateusz/Desktop/node-unblocker
> ./node_modules/gatling/gatling.js --threads 2 ./app.js

module.js:338
    throw err;
          ^
Error: Cannot find module '../build/Debug/iconv.node'
    at Function.Module._resolveFilename (module.js:336:15)
    at Function.Module._load (module.js:278:25)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at Object.<anonymous> (/Users/mateusz/Desktop/node-unblocker/node_modules/iconv/lib/iconv.js:27:14)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)

npm ERR! Darwin 14.3.0
npm ERR! argv "node" "/usr/local/bin/npm" "start"
npm ERR! node v0.12.0
npm ERR! npm  v2.5.1
npm ERR! code ELIFECYCLE
npm ERR! [email protected] start: `./node_modules/gatling/gatling.js --threads 2 ./app.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start script './node_modules/gatling/gatling.js --threads 2 ./app.js'.
npm ERR! This is most likely a problem with the node-unblocker package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     ./node_modules/gatling/gatling.js --threads 2 ./app.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls node-unblocker
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/mateusz/Desktop/node-unblocker/npm-debug.log

any thoughts ?

for example: http://nodeunblocker.com/proxy/http://asdf.com/ works but http://nodeunblocker.com/proxy/http://asdf.com (no trailing slash) doesn't show the image.

I will admit, I definitely agree that the fact that URLs do not need to be altered aside from changing the site root is wonderful! While I would absolutely love to be able to use node-unblocker like this, it is unfortunately detected by the web filtering in use at my workplace. When I use the "encode URLs" feature offered by Glype, I have no issues, but it will be blocked just the same if I don't use that feature.

If there is any way to make it so that the URLs are able to be more random, or a way to exclude the indicators that I am accessing another domain name (like the http:// and .com parts in the URL), that would be absolutely lovely!

Your install instructions seem to be missing a step

npm start

npm ERR! [email protected] start: `gatling ./app.js`
(...)
npm ERR! Tell the Author that this fails on your system:
npm ERR!     gatling ./app.js

Installing gatling fixes the issue, but I thought I'd mention it.

Hi! great project.

I'm having an issue where the proxy works fine with the url, but fails when placed behind a reverse proxy.
I get 'Oops, there was a problem!'

my nginx config is very basic:

server {
    listen 80;
    server_name proxy.site.com;
    keepalive_timeout 15m;

    location / {
            proxy_pass http://192.168.1.157:8080/;

    }

    location /robots.txt {
    return 200 "User-agent: *\nDisallow: /";
    }

}

I get in the terminal log:

error TypeError: Cannot read property 'split' of null
    at registeredDomain (/root/node-unblocker/proxy/node_modules/unblocker/node_modules/tld/lib/tld.js:80:23)
    at Object.exports.registered (/root/node-unblocker/proxy/node_modules/unblocker/node_modules/tld/lib/tld.js:120:12)
    at rewriteCookiesAndLinks (/root/node-unblocker/proxy/node_modules/unblocker/lib/cookies.js:97:27)
    at /root/node-unblocker/proxy/node_modules/unblocker/lib/proxy.js:66:13
    at arraySome (/root/node-unblocker/proxy/node_modules/unblocker/node_modules/lodash/index.js:1493:13)
    at Function.some (/root/node-unblocker/proxy/node_modules/unblocker/node_modules/lodash/index.js:7051:14)
    at proxyResponse (/root/node-unblocker/proxy/node_modules/unblocker/lib/proxy.js:65:43)
    at ClientRequest.<anonymous> (/root/node-unblocker/proxy/node_modules/unblocker/lib/proxy.js:42:17)
    at ClientRequest.g (events.js:260:16)
    at emitOne (events.js:77:13)

How can we add a mini url form at the top of every result page , so that user dont have to go back to the home page to open a new website?

Hey,
This Website doesn't work on the heroku version at least. haven't had the time to setup one myself.

http://node-unblocker.herokuapp.com/proxy/http://us.battle.net/sc2/en/forum/topic/11040734962 

Maybe You can look into that?

When going to domains with internationalized domainnames such as http://www.vårdverktyget.se/ it causes unblocker to redirect to http://www.v/årdverktyget.se/ instead making unblocker trying to connect to the domain www.v instead of www.xn--vrdverktyget-tcb.se.

I might look into this at a later time unless you get to it first.

Look into Punycode: https://en.wikipedia.org/wiki/Punycode

Possible module: https://www.npmjs.com/package/punycode

The proxy is crashing on some sites. An example of a URL that causes the proxy to crash is: http://www.optimcore.com/

Here is the full error message:

Air:simple matthew$ npm start

> [email protected] start /Users/matthew/Sites/node-unblocker-master/examples/simple
> node server.js

/Users/matthew/Sites/node-unblocker-master/examples/simple/node_modules/tld/lib/tld.js:80
    var parts = domain.split('.'),
                      ^

TypeError: Cannot read property 'split' of null
    at registeredDomain (/Users/matthew/Sites/node-unblocker-master/examples/simple/node_modules/tld/lib/tld.js:80:23)
    at Object.exports.registered (/Users/matthew/Sites/node-unblocker-master/examples/simple/node_modules/tld/lib/tld.js:120:12)
    at rewriteCookiesAndLinks (/Users/matthew/Sites/node-unblocker-master/examples/simple/node_modules/unblocker/lib/cookies.js:97:27)
    at /Users/matthew/Sites/node-unblocker-master/examples/simple/node_modules/unblocker/lib/proxy.js:66:13
    at arraySome (/Users/matthew/Sites/node-unblocker-master/examples/simple/node_modules/lodash/index.js:1493:13)
    at Function.some (/Users/matthew/Sites/node-unblocker-master/examples/simple/node_modules/lodash/index.js:7051:14)
    at proxyResponse (/Users/matthew/Sites/node-unblocker-master/examples/simple/node_modules/unblocker/lib/proxy.js:65:43)
    at ClientRequest.<anonymous> (/Users/matthew/Sites/node-unblocker-master/examples/simple/node_modules/unblocker/lib/proxy.js:42:17)
    at ClientRequest.g (events.js:260:16)
    at emitOne (events.js:77:13)

npm ERR! Darwin 15.2.0
npm ERR! argv "/opt/local/bin/node" "/opt/local/bin/npm" "start"
npm ERR! node v4.2.3
npm ERR! npm  v3.5.2
npm ERR! code ELIFECYCLE
npm ERR! [email protected] start: `node server.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start script 'node server.js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the simple-unblocker-server package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node server.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs simple-unblocker-server
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls simple-unblocker-server
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/matthew/Sites/node-unblocker-master/examples/simple/npm-debug.log

and here is the contents of the log file:

0 info it worked if it ends with ok
1 verbose cli [ '/opt/local/bin/node', '/opt/local/bin/npm', 'start' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prestart', 'start', 'poststart' ]
5 info lifecycle [email protected]~prestart: [email protected]
6 silly lifecycle [email protected]~prestart: no script for prestart, continuing
7 info lifecycle [email protected]~start: [email protected]
8 verbose lifecycle [email protected]~start: unsafe-perm in lifecycle true
9 verbose lifecycle [email protected]~start: PATH: /opt/local/lib/node_modules/npm/bin/node-gyp-bin:/Users/matthew/Sites/node-unblocker-master/examples/simple/node_modules/.bin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
10 verbose lifecycle [email protected]~start: CWD: /Users/matthew/Sites/node-unblocker-master/examples/simple
11 silly lifecycle [email protected]~start: Args: [ '-c', 'node server.js' ]
12 silly lifecycle [email protected]~start: Returned: code: 1  signal: null
13 info lifecycle [email protected]~start: Failed to exec start script
14 verbose stack Error: [email protected] start: `node server.js`
14 verbose stack Exit status 1
14 verbose stack     at EventEmitter.<anonymous> (/opt/local/lib/node_modules/npm/lib/utils/lifecycle.js:232:16)
14 verbose stack     at emitTwo (events.js:87:13)
14 verbose stack     at EventEmitter.emit (events.js:172:7)
14 verbose stack     at ChildProcess.<anonymous> (/opt/local/lib/node_modules/npm/lib/utils/spawn.js:24:14)
14 verbose stack     at emitTwo (events.js:87:13)
14 verbose stack     at ChildProcess.emit (events.js:172:7)
14 verbose stack     at maybeClose (internal/child_process.js:818:16)
14 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)
15 verbose pkgid [email protected]
16 verbose cwd /Users/matthew/Sites/node-unblocker-master/examples/simple
17 error Darwin 15.2.0
18 error argv "/opt/local/bin/node" "/opt/local/bin/npm" "start"
19 error node v4.2.3
20 error npm  v3.5.2
21 error code ELIFECYCLE
22 error [email protected] start: `node server.js`
22 error Exit status 1
23 error Failed at the [email protected] start script 'node server.js'.
23 error Make sure you have the latest version of node.js and npm installed.
23 error If you do, this is most likely a problem with the simple-unblocker-server package,
23 error not with npm itself.
23 error Tell the author that this fails on your system:
23 error     node server.js
23 error You can get information on how to open an issue for this project with:
23 error     npm bugs simple-unblocker-server
23 error Or if that isn't available, you can get their info via:
23 error     npm owner ls simple-unblocker-server
23 error There is likely additional logging output above.
24 verbose exit [ 1, true ]

0 info it worked if it ends with ok
1 verbose cli [ 'C:\Program Files (x86)\nodejs\node.exe',
1 verbose cli 'C:\Program Files (x86)\nodejs\node_modules\npm\bin\npm-cli.js',
1 verbose cli 'start' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prestart', 'start', 'poststart' ]
5 info prestart [email protected]
6 info start [email protected]
7 verbose unsafe-perm in lifecycle true
8 info [email protected] Failed to exec start script
9 verbose stack Error: [email protected] start: ./node_modules/gatling/gatling.js ./app.js
9 verbose stack Exit status 1
9 verbose stack at EventEmitter. (C:\Program Files (x86)\nodejs\node_modules\npm\lib\utils\lifecycle.js:217:16)
9 verbose stack at emitTwo (events.js:87:13)
9 verbose stack at EventEmitter.emit (events.js:172:7)
9 verbose stack at ChildProcess. (C:\Program Files (x86)\nodejs\node_modules\npm\lib\utils\spawn.js:24:14)
9 verbose stack at emitTwo (events.js:87:13)
9 verbose stack at ChildProcess.emit (events.js:172:7)
9 verbose stack at maybeClose (internal/child_process.js:827:16)
9 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)
10 verbose pkgid [email protected]
11 verbose cwd C:\Users\Administrator\Desktop\node-unblocker\examples\nodeunblocker.com
12 error Windows_NT 6.2.9200
13 error argv "C:\Program Files (x86)\nodejs\node.exe" "C:\Program Files (x86)\nodejs\node_modules\npm\bin\npm-cli.js" "start"
14 error node v4.4.5
15 error npm v2.15.5
16 error code ELIFECYCLE
17 error [email protected] start: ./node_modules/gatling/gatling.js ./app.js
17 error Exit status 1
18 error Failed at the [email protected] start script './node_modules/gatling/gatling.js ./app.js'.
18 error This is most likely a problem with the nodeunblocker.com package,
18 error not with npm itself.
18 error Tell the author that this fails on your system:
18 error ./node_modules/gatling/gatling.js ./app.js
18 error You can get information on how to open an issue for this project with:
18 error npm bugs nodeunblocker.com
18 error Or if that isn't available, you can get their info via:
18 error
18 error npm owner ls nodeunblocker.com
18 error There is likely additional logging output above.
19 verbose exit [ 1, true ]

For example, https://node-unblocker.herokuapp.com/proxy/http://nfriedly.com/ looks very broken while http://node-unblocker.herokuapp.com/proxy/http://nfriedly.com/ looks just fine.

So there is this technology that more and more sites are starting to use that breaks the way that unblocker handles paths. (https://github.com/KidkArolis/location-bar)

The technology enables JavaScript to change the complete path (not including domains) for the client without reloading the browser.

I noticed this problem when trying to make DynaMeds page work in my project Proxive.

Other examples of this technology in use:
Discourse: http://try.discourse.org/
LinkedIn Pulse: https://www.linkedin.com/pulse/how-those-totally-useless-meetings-can-make-break-your-jack-welch
Ember.js uses this technology in its routers.

To fix this it would require a complete rewrite of unblocker so it instead uses the domain for the domain of the page visited like this:

http://google.com.example.com/path?query=query#hash instead of the current http://example.com/proxy/http://google.com/path?query=query#hash

The biggest issue I see is how to detect wheter the site is https or not. Maybe a solution would be prepending the domain with something like h-t-t-p-s so that the domain would look like h-t-t-p-s.google.com.example.com.

You can close this issue if you feel that this will not/cannot be part of this project. Maybe we can cooperate on a separate project/branch that enables the above? (I'm planning on working on this very soon since my job needs to stay on top of the issue so please respond as soon as possible if you're interested in cooperating.)

http://node-unblocker.herokuapp.com/proxy/https://plus.google.com/up/start/?continue=https://plus.google.com/&type=st&gpcaz=5d1f5b58

and then you end up at

http://node-unblocker.herokuapp.com/proxy/https://plus.google.com/http://node-unblocker.herokuapp.com/proxy/https://node-unblocker.herokuapp.com/proxy/https://accounts.google.com/ServiceLogin?service=oz&continue=https://plus.google.com/?gpcaz%3D5d1f5b58&hl=en-US

It could be google just screwing with me, but it's worth checking.

So, for example, http://asdf.com/?asdf sends you to http://nodeunblocker.com/proxy/http://asdf.com%3Fasdf when it should actually send you to http://nodeunblocker.com/proxy/http://asdf.com/?asdf

Hi,

I'm trying to put node-unblocker behind a reverse proxy for intranet access:

I need to have under a subdirectory, and have modified the code to point to /foo/bar as it's root and it is working perfectly on it's own. But when I put nginx infront of node-unblocker I get stuck in a redirect loop:

curling node-unblocker directly results in the expected output:

curl -i http://172.16.6.168:800/foo/bar/proxy/http://example.com/
**expected output**

while the proxy returns:

curl -i http://172.16.6.168/foo/bar/proxy/http://example.com
HTTP/1.1 307 Temporary Redirect
Date: Wed, 25 Mar 2015 13:28:05 GMT
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: unblocker.sid=s%3AOWn3F7Kx4eGCwtoTxgeui6ui.w0vbLLD3wwWYHCXUQly85wTf0PhwEj9S89ccLrhrQCo; Path=/
location: http://172.16.6.168/foo/bar/proxy/http://localhost/foo/bar/proxy

The relevant section of my nginx config is:

location /foo/bar/ {
        proxy_pass        http://X.X.X.X:XX/foo/bar/;
        proxy_http_version 1.1;
        proxy_pass_header Server;
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        }

Im sure i must be missing something pretty simple? If you had any pointers it would be greatly appreciated.

Cheers

Pete

Below is the Yandex Metrica script tag from the webpage http://www.survio.com/l-cs-dotaznik-form/

<script type="text/javascript">
(function(d, w, c) {
    (w[c] = w[c] || []).push(function() {
        try {
            w.yaCounter24267952 = new Ya.Metrika({
                id: 24267952,
                clickmap: true,
                trackLinks: true,
                accurateTrackBounce: true
            });
        } catch (e) {}
    });
    var n = d.getElementsByTagName("script")[0],
        s = d.createElement("script"),
        f = function() {
            n.parentNode.insertBefore(s, n);
        };
    s.type = "text/javascript";
    s.async = true;
    s.src = (d.location.protocol == "https:" ? "https:" : "http:") + "//mc.yandex.ru/metrika/watch.js";
    if (w.opera == "[object Opera]") {
        d.addEventListener("DOMContentLoaded", f, false);
    } else {
        f();
    }
})(document, window, "yandex_metrika_callbacks");
</script>

When this page is loaded via the unblocker proxy this script is rewritten as follows:

<script type="text/javascript">
(function(d, w, c) {
    (w[c] = w[c] || []).push(function() {
        try {
            w.yaCounter24267952 = new Ya.Metrika({
                id: 24267952,
                clickmap: true,
                trackLinks: true,
                accurateTrackBounce: true
            });
        } catch (e) {}
    });
    var n = d.getElementsByTagName("script")[0],
        s = d.createElement("script"),
        f = function() {
            n.parentNode.insertBefore(s, n);
        };
    s.type = "text/javascript";
    s.async = true;
    s.src = (d.location.protocol == "/proxy/https:" ? "/proxy/https:" : "/proxy/http:") + "/proxy/http://mc.yandex.ru/metrika/watch.js";
    if (w.opera == "[object Opera]") {
        d.addEventListener("DOMContentLoaded", f, false);
    } else {
        f();
    }
})(document, window, "yandex_metrika_callbacks");
</script>

The script then creates the following url: /proxy/http://proxy/http://mc.yandex.ru/metrika/watch.js, which is not correct and causes a 500 error.

I checked unblocker/lib/proxy.js, and this module used http and https library to send real request.
It's a little hard to integrate proxy feature.
Maybe you could think about using request library to send request?
So that the proxy feature can be easily integrated.

Articles from Baidu Baike (the largest Chinese online encyclopedia), for example, http://baike.baidu.com/view/48682.htm can be visited as-is, but if we try going through Node-Unblocker, I get back a error 503, for example as seen at http://nodeunblocker.com/proxy/http://baike.baidu.com/view/48682.htm (same issues occurs on a node-unblocker instance run on my own server from the latest git checkout).

Sorry to be back so soon again.

I've just noticed that it is possible to reach local IP addresses through the proxy. (like 192.168.1.154)

Is there a setting or some lines I can add to stop the proxy from reaching local IP addresses? I know I could create a whole vlan, but that would be overkill and interfere with the other nodejs services I have running on the same VM.

I tested this site that used IPv6. If you don't have IPv6 it returns a page with the following (only):

<META HTTP-EQUIV="Refresh" CONTENT="0;URL=http://support.bredbandskollen.se/support/solutions/articles/1000212163-vad-r-ipv6-och-hur-fungerar-bredbandskollen-ver-ipv6">

Headers:

HTTP/1.1 200 OK
Date: Mon, 11 Jul 2016 14:24:35 GMT
Server: Apache/2.2.22 (Debian)
X-Powered-By: PHP/5.4.45-0+deb7u2
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 157
Content-Type: text/html

This is the page that does this: http://ipv6.bredbandskollen.se/

The issue is that the url parsing doesn't catch this one even though the content-type is text/html...

node-unblocker starting well but when i try login any website facebook - twitter etc. website alert your browser does not support cookies please open cookies bla bla bla.

i am trying cookies on your unblocker, its works well. source code and your unblocker files are different or i lost sometings ?

Greetings,

I am testing latest git. I am unable to get it to work with any of the Google logins. Similar to report on issue #35 and just reporting here. Any ideas?

Loading the following page: http://kalkulackaenergie.com/

Through the proxy: https://www.optimcore.com/proxy/http://kalkulackaenergie.com/

Causes Mixed Content errors because image source urls within an img tag srcset attribute are loaded directly, over http. These urls should be proxyfied.

None of the google logins work, they just throw an error after attempting to login.
Twitter's password field cannot be typed into, it just remains blank when attempting to type a password.
Facebook keeps throwing redirect notifications

If you visit a site like http://perro-obediente.com/ then all the characters look correct. However, when it goes through this proxy, http://nodeunblocker.com/proxy/http://perro-obediente.com/ then we see that the characters get mangled (for example, "¿Tu perro se hace pipí dentro de la casa? " becomes "�Tu perro se hace pip� dentro de la casa?"

Greetings,

Trying to install and run from git or zip download on Linux Mint 17.1 32bit , npm install appears to run ok:

lmint@lmint-VirtualBox ~/scratch/node-unblocker-master/examples/nodeunblocker.com $ sudo npm install
npm WARN engine [email protected]: wanted: {"node":"0.12.x","npm":"2.7.x"} (current: {"node":"0.12.5","npm":"2.11.2"})
[email protected] node_modules/express
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected] ([email protected])
├── [email protected] ([email protected])
├── [email protected] ([email protected])
├── [email protected] ([email protected], [email protected])
├── [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected])
├── [email protected] ([email protected], [email protected])
└── [email protected] ([email protected], [email protected])

[email protected] node_modules/gatling
└── [email protected] ([email protected], [email protected])

[email protected] node_modules/unblocker
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected] ([email protected])
├── [email protected]
└── [email protected]

[email protected] node_modules/newrelic

However npm start does not fair so well:
lmint@lmint-VirtualBox ~/scratch/node-unblocker-master/examples/nodeunblocker.com $ sudo npm start

[email protected] start /home/lmint/scratch/node-unblocker-master/examples/nodeunblocker.com
./node_modules/gatling/gatling.js ./app.js

module.js:338
throw err;
^
Error: Cannot find module 'through'
at Function.Module._resolveFilename (module.js:336:15)
at Function.Module._load (module.js:278:25)
at Module.require (module.js:365:17)
at require (module.js:384:17)
at Object. (/home/lmint/scratch/node-unblocker-master/examples/nodeunblocker.com/app.js:16:15)
at Module._compile (module.js:460:26)
at Object.Module._extensions..js (module.js:478:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Module.require (module.js:365:17)

npm ERR! Linux 3.13.0-37-generic
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "start"
npm ERR! node v0.12.5
npm ERR! npm v2.11.2
npm ERR! code ELIFECYCLE
npm ERR! [email protected] start: ./node_modules/gatling/gatling.js ./app.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start script './node_modules/gatling/gatling.js ./app.js'.
npm ERR! This is most likely a problem with the nodeunblocker.com package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! ./node_modules/gatling/gatling.js ./app.js
npm ERR! You can get their info via:
npm ERR! npm owner ls nodeunblocker.com
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR! /home/lmint/scratch/node-unblocker-master/examples/nodeunblocker.com/npm-debug.log

Here is teh output of the log file:

lmint@lmint-VirtualBox ~/scratch/node-unblocker-master/examples/nodeunblocker.com $ cat /home/lmint/scratch/node-unblocker-master/examples/nodeunblocker.com/npm-debug.log
0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node', '/usr/local/bin/npm', 'start' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prestart', 'start', 'poststart' ]
5 info prestart [email protected]
6 info start [email protected]
7 verbose unsafe-perm in lifecycle true
8 info [email protected] Failed to exec start script
9 verbose stack Error: [email protected] start: ./node_modules/gatling/gatling.js ./app.js
9 verbose stack Exit status 1
9 verbose stack at EventEmitter. (/usr/local/lib/node_modules/npm/lib/utils/lifecycle.js:213:16)
9 verbose stack at EventEmitter.emit (events.js:110:17)
9 verbose stack at ChildProcess. (/usr/local/lib/node_modules/npm/lib/utils/spawn.js:24:14)
9 verbose stack at ChildProcess.emit (events.js:110:17)
9 verbose stack at maybeClose (child_process.js:1015:16)
9 verbose stack at Process.ChildProcess._handle.onexit (child_process.js:1087:5)
10 verbose pkgid [email protected]
11 verbose cwd /home/lmint/scratch/node-unblocker-master/examples/nodeunblocker.com
12 error Linux 3.13.0-37-generic
13 error argv "/usr/local/bin/node" "/usr/local/bin/npm" "start"
14 error node v0.12.5
15 error npm v2.11.2
16 error code ELIFECYCLE
17 error [email protected] start: ./node_modules/gatling/gatling.js ./app.js
17 error Exit status 1
18 error Failed at the [email protected] start script './node_modules/gatling/gatling.js ./app.js'.
18 error This is most likely a problem with the nodeunblocker.com package,
18 error not with npm itself.
18 error Tell the author that this fails on your system:
18 error ./node_modules/gatling/gatling.js ./app.js
18 error You can get their info via:
18 error npm owner ls nodeunblocker.com
18 error There is likely additional logging output above.
19 verbose exit [ 1, true ]

Any ideas?
Thanks.

http://node-unblocker.herokuapp.com/proxy/http://www.pocketgamer.biz/r/PG.Biz/PocketGamer.biz/feature.asp?c=56325

Hey,
I found another broken page. The images don't load. Maybe U should add a functionallity so that one can report the page directly to u via email. Or that u have a form in Which u enter your deatails and automatically get ya issues which u then can post to github.

Spruce

I've been thinking about how to make umblocker easier to embed in larger apps, and the big issue is the dependency on redis to store cookies. If we just dropped it entirely, and started passing cookies along verbatim, most things would continue to work although they would of course leak between sites.

You could set the path to /proxy/http://example.com + cookie.path and it would keep server-generated cookies from leaking, but you'd loose them when switching from http to https and vice versa & when switching subdomains. I thought about just always setting two cookies - one for http and one for https, but that seems a bit redundant and still doesn't handle the subdomain issue.

My current thinking, though is to set the cookies to /proxy/http://example.com + cookie.path but then have an extra filter that detects any links that change protocol or subdomain, and rewrite them to stay on the current domain, but add a parameter to the end that would let the proxy know to copy cookies and then redirect to the correct url.

It would also be possible to create a "meta cookie" that stored things like cookie paths and httponly flags to honor those, but I don't think it would be necessary for a first pass.

This would add a bit of complexity, but I think it might be worth it.

I'm also thinking that I want to make things a bit more modularizable - so the existing cookie code could be kept as an option, as could this new stateless mode. And anyone that wanted to embed unblocker could choose to enable one (or neither).

Nevermind.

Youtube does not play videos.

In file charsets.js i find one problem for my proxy. Into snippet:

        var charset = contentTypes.getCharset(data);

        // todo: consider just rewriting headers and meta tags to utf8 instead of reencoding things
        if (iconv.encodingExists(charset)) {
            // happy case, we know the encoding right away, so we can just return decode/recode streams
            data.charset = charset;
            data.stream = data.stream.pipe(iconv.decodeStream(charset));
            debug('decoding %s charset via iconv stream', charset);
        } else if (mayContainMeta(data.contentType)) {
            debug('decoding unknown charset via iconv html stream');
            data.charsetDecoder = new IconvHtmlStream();
            data.charsetDecoder.on('charset', function(charset) {
                // note: while the recode stream will accept content before this and just output utf-8, it shouldn't actually receive any data because the decode stream buffers until *after* this event
                data.charset = charset;
            });
            data.stream = data.stream.pipe(data.charsetDecoder);
        } else {
            debug('no charset info available, assuming utf8');
            // semi-happy case. we know the content needs parsed but have no way of knowing it's charset. Hopefully .toString() will be good enough. No recoding
            data.stream = data.stream.pipe(new PassThrough({
                encoding: 'utf8'
            }));
        }

When charset = undefined, executing snippet in else if block. But in this case, stream hang up on this place = > data.charsetDecoder.on('charset', ...).

For resolving this trouble I add new condition into if ... else ... processing:
if (charset && iconv.encodingExists(charset)) { ... }
else if(charset && mayContainMeta(data.contentType)){ ... }
else { ... }

If u have best idea for this trouble, please, add your resolve in next commit

Sorry for my english)

First off, this is a great project -- I'm really impressed at how fast it is compared to similar projects (Glype, CGIProxy)

I was just wondering if you had a fix / had thought about how to process logins. I've tried numerous sites, and none work (Google, LinkedIn, etc.).

Thanks

Will there ever be support for the "Connection: Keep-Alive" header?

I was wondering if you would change it to be more like Tor2Web, where contributers could run their own node then have it work in a round-robin to handle traffic from the actual domain.

Due to the Heroku domain-binding issue, you could give each node it's own subdomain (like node4.nodeunblocker.com) then have any requests to nodeunblocker.com redirect to a valid node.

Just a thought.

Currently http://nodeunblocker.com/proxy/http throws this error:

TypeError: Cannot read property 'length' of undefined
at getCookies (/app/server.js:334:37)
at proxy (/app/server.js:162:19)
at Object.handle (/app/server.js:89:10)
at next (/app/node_modules/connect/lib/proto.js:176:15)
at /app/node_modules/connect/lib/middleware/session.js:276:9
at /app/node_modules/connect/lib/middleware/session.js:298:9
at /app/node_modules/connect-redis/lib/connect-redis.js:85:9
at try_callback (/app/node_modules/redis/index.js:466:9)
at RedisClient.return_reply (/app/node_modules/redis/index.js:503:13)
at RedisReplyParser. (/app/node_modules/redis/index.js:259:14)

That should be caught and responded to with a redirect to something like /proxy?error=Unable to process your request, please check the URL and try again.

Currently the connection goes to a random thread which then reports only it's hit count

I created a test for this with PHP:

http://dev.ldbib.se/test/cookie_proxive_test.php
should redirect to
http://dev.ldbib.se/test/cookie_proxive_test2.php
and set a cookie named "hello" to "world"

Sourcecode to the tests:
http://dev.ldbib.se/test/cookie_proxive_test.txt
http://dev.ldbib.se/test/cookie_proxive_test2.txt

The app needs a web interface to manage the blocklist, especially for heroku instances.