getlantern / http-proxy Goto Github PK
View Code? Open in Web Editor NEWHTTP/S Proxy for Lantern
HTTP/S Proxy for Lantern
This is currently happening with what looks like every request onhttps://cloud.digitalocean.com/droplets/12291187/graphs
Update Lantern to the Latest Versions of Obfs4 as an effective transport to deploy in the event that current transports stop working
Dynamically turn on HTTP request tracing
Jun 09 14:19:41.942 ERROR server: server.go:2169 http: Accept error: read tcp 128.199.223.25:443-
Jun 09 14:20:46.956 ERROR server: server.go:2169 http: Accept error: read tcp 128.199.223.25:443->61.131.11.252:56611: i/o timeout; retrying in 10ms
Jun 09 14:21:51.973 ERROR server: server.go:2169 http: Accept error: read tcp 128.199.223.25:443-
Jun 09 14:22:56.996 ERROR server: server.go:2169 http: Accept error: read tcp 128.199.223.25:443->122.189.210.208:16086: i/o timeout; retrying in 40ms
Jun 09 14:24:02.038 ERROR server: server.go:2169 http: Accept error: read tcp 128.199.223.25:443->61.131.11.252:57558: i/o timeout; retrying in 80ms
Jun 09 14:25:07.120 ERROR server: server.go:2169 http: Accept error: read tcp 128.199.223.25:443->61.131.11.252:57560: i/o timeout; retrying in 160ms
Because Accept() is blocking, this prevents us from processing other connections.
We aren't bypassing limits for Pro users yet!
I'm not sure that this is the best place to log this, but I needed a secret repo ...
While chatting with @aranhoide, it occurred to me that we have a great network of blocking detectors, namely our Lantern clients. When Lantern clients detect that they're consistently unable to connect to a proxy at some IP address in a way that it looks blocked, we should have them report this. We could log this to Google Analytics and treat each server and/or IP as a web page URL, and geolocate the clients while we're at it. This could provide some valuable insights into blocking.
IP-Packet forwarding to tunnel the IP packets themselves to
Lantern’s proxies, which then recreate the TCP streams on the server side
We're currently on redis.v3, so 2 releases out of date. There's no urgent need for this, but probably be good to get current.
cc: @uaalto @myleshorton
@oxtoacart commented on Fri Oct 21 2016
I noticed yesterday that several of my pro proxies were unresponsive. It turns out that they were stuck in a restart loop waiting for Redis to finish "loading dataset in memory". It turns out that Redis had been restarted due to a system update, and some of my proxies restarted at a similar time. This made Lantern basically unusable for me until Redis finished loading its dataset in memory (which takes a while).
We could temporarily disable system updates for the redis server, but I don't like the fact that this means we'll miss security patches. In reality, I think we will need to take patches at some point and restart, so we need to figure out a way to make our system resilient to redis restarts.
One way to address this would be to have proxies locally store the data that they need (i.e. authorized pro users) so that even if redis is temporarily unavailable, they can continue to function.
Another option, which would also help us secure our http-proxies for use on services like Aliyun is to not even connect them with Redis at all. If we had an authentication server for pro users that issued time-limited, signed authentication tokens, our pro http-proxies could simply check the signature on the tokens to verify that the user is in fact pro. This would also have the side benefit of reducing load on redis.
We'd also have to figure out what to do about throttling, but I bet that's solvable too. One easy solution would be to just stop pretending that we're throttling since we're not actually throttling anyway :)
A completely different solution would be to switch to a disk-based database that's api-compatible with Redis and can start up in a reasonable amount of time (e.g. LedisDB). I don't love this option because it introduces a fair amount of risk, especially from a performance perspective.
The problem is on persistent HTTP connections, were HTTP requests after the first are missing values for the header. We have logic to prevent this, but it didn't support arrays of values.
So for example, you see something like this with extra trace logging:
Feb 7 16:56:12 fp-mskr1c001sea-20210202-068 http-proxy[1675]: TRACE redis: measured_reporter.go:175 throttle cohort for device blahblah country code CN platform windows version 6.3.5 and supported data caps [monthly weekly daily] is china monthly
Feb 7 16:57:35 fp-mskr1c001sea-20210202-068 http-proxy[1675]: TRACE redis: measured_reporter.go:175 throttle cohort for device blahblah country code CN platform windows version 6.3.5 and supported data caps [] is china legacy
Example:
{"extra":{"logLevel":"ERROR","version":"version (releasedate)"},"fullMessage":"ERROR http-proxy-lantern.analytics: analytics.go:123 Could not send HTTP request to GA: Post https://ssl.google-analytics.com/collect: dial tcp 173.194.126.190:443: i/o timeout\n","hostname":"fp-jp-20151019-519","instanceid":"instanceid","locationInfo":"ERROR http-proxy-lantern.analytics","message":"dial tcp 173.194.126.190:443: i/o timeout","timestamp":1454050540011}
Maybe silence these unless we want to make sure that all requests get to GA.
It looks like the locking on the http server implementation changed and we're getting blocked on hijacking:
goroutine 56 [semacquire]:
sync.runtime_notifyListWait(0xc42024e410, 0x0)
/usr/local/go/src/runtime/sema.go:297 +0x10b
sync.(*Cond).Wait(0xc42024e400)
/usr/local/go/src/sync/cond.go:57 +0x89
net/http.(*connReader).abortPendingRead(0xc42024e380)
/usr/local/go/src/net/http/server.go:686 +0xbf
net/http.(*conn).hijackLocked(0xc4201d57c0, 0x46441a8, 0xc4201d5850, 0x4580140, 0x45f88a0, 0xc42028e500)
/usr/local/go/src/net/http/server.go:292 +0x53
net/http.(*response).Hijack(0xc42020c620, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:1892 +0xfc
github.com/getlantern/http-proxy-lantern/mimic.MimicApache(0x48dd080, 0xc42020c620, 0xc420185500)
/Users/ox.to.a.cart/gocode/src/github.com/getlantern/http-proxy-lantern/mimic/apache.go:56 +0x6d
github.com/getlantern/http-proxy-lantern/tokenfilter.(*tokenFilter).Apply(0xc420222870, 0x48dd080, 0xc42020c620, 0xc420185500, 0xc42028a880, 0x0, 0x0)
/Users/ox.to.a.cart/gocode/src/github.com/getlantern/http-proxy-lantern/tokenfilter/tokenfilter.go:47 +0x5b0
github.com/getlantern/http-proxy/filters.Chain.ServeHTTP(0xc420210d80, 0x3, 0x3, 0x48dd080, 0xc42020c620, 0xc420185500)
/Users/ox.to.a.cart/gocode/src/github.com/getlantern/http-proxy/filters/filters.go:60 +0x167
github.com/getlantern/http-proxy/filters.(*Chain).ServeHTTP(0xc4201cfda0, 0x48dd080, 0xc42020c620, 0xc420185500)
<autogenerated>:4 +0x7d
github.com/getlantern/http-proxy/server.NewServer.func1(0x48dd080, 0xc42020c620, 0xc420185500)
/Users/ox.to.a.cart/gocode/src/github.com/getlantern/http-proxy/server/server.go:40 +0x111
net/http.HandlerFunc.ServeHTTP(0xc4201cfdc0, 0x48dd080, 0xc42020c620, 0xc420185500)
/usr/local/go/src/net/http/server.go:1942 +0x44
net/http.serverHandler.ServeHTTP(0xc42018f5f8, 0x48dd080, 0xc42020c620, 0xc420185500)
/usr/local/go/src/net/http/server.go:2568 +0x92
net/http.(*conn).serve(0xc4201d57c0, 0x48ddc40, 0xc42024e340)
/usr/local/go/src/net/http/server.go:1825 +0x612
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:2668 +0x2ce
(https://github.com/getlantern/lantern_aws/pull/123 depends on this.)
We would like to publish IPv6 addresses for proxies it the cloud.yaml served by the config server, because that might help with infiltration. Before we do that, we need to make sure that http-proxy(-lantern) works without problems when reached through a IPv6 address.
Feel free to close this and move it to getlantern/http-proxy if you think it belongs there instead. Or just close it if it seems super obvious that http-proxy(-lantern) is not depending on IPv4 in any way.
{"extra":{"logLevel":"ERROR","version":"version (releasedate)"},"fullMessage":"ERROR errorhandler: errorhandler.go:36 Responding with 502 due to dial tcp: lookup hcfq9zfs.vmgoxp64.netdna-cdn.com on 8.8.8.8:53: no such host: Error forwarding from 91.99.1.144:54579 to hcfq9zfs.vmgoxp64.netdna-cdn.com\n","hostname":"fp-doams3-20160330-002","instanceid":"instanceid","locationInfo":"ERROR errorhandler","message":"no such host: Error forwarding from 91.99.1.144:54579 to hcfq9zfs.vmgoxp64.netdna-cdn.com","timestamp":1463722704992}
Deploy QUIC via Cronet with built-in security via TLS 1.3, multiplexing, and faster speeds particularly
for poorly connected users
This proxy is consistently failing checks from multiple proxy checkers, and the log shows that the proxy checkers are sending a different token than the proxy expects.
May 25 20:02:14.861 DEBUG blacklist: blacklist.go:125 104.238.188.202 connected but failed to successfully send an HTTP request within 30s
May 25 20:11:48.517 DEBUG tokenfilter: tokenfilter.go:72 Mismatched token(s) iQb4HrXHloyl38W6nRnSARnkzxo98hulWCNVMLIJROWLreNs3ruKcreztH9odjqM from 104.238.188.202:35879 for request to www.google.com, mimicking apache
Eventually this results in the checker becoming blacklisted.
cc: @aranhoide
Don't give pro-only data centers access to redis at all.
This probably boils down to an issue with the git-chglog
tool we're using. I ran make dist
with git-chglog v0.9.1
and got the following:
2021-03-04
Pull Requests
Merge pull request #455 from getlantern/ox/throttle_logging
This misses a few commits and PRs which show up in git log
:
commit c99474b0258c0ca8729d016e433d2c11c2534eee (HEAD -> master, origin/master, origin/HEAD)
Author: Harry Harpham <[email protected]>
Date: Thu Mar 4 13:17:48 2021 -0700
Updated changelog for v2.6.32
CHANGELOG.md | 150 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------------------------------------------------
1 file changed, 86 insertions(+), 64 deletions(-)
commit 256dbb7701cf026cb3dd7073c26de598ef290a86 (tag: v2.6.32)
Author: hwh33 <[email protected]>
Date: Thu Mar 4 13:11:58 2021 -0700
Incorporate tlsmasq bug fix: (#456)
https://github.com/getlantern/tlsmasq/issues/9
go.mod | 2 +-
go.sum | 8 ++++----
2 files changed, 5 insertions(+), 5 deletions(-)
commit 6557a8e09bb141a9ba379c5e4e31717f79d27db3
Merge: 51cd502 4578921
Author: Myles Horton <[email protected]>
Date: Wed Mar 3 08:26:17 2021 -0500
Merge pull request #455 from getlantern/ox/throttle_logging
Added trace logging to help watch throttle cohort assignment
commit 45789218213aa4b57d8f601eb54344863a261abd
Author: Ox Cart <[email protected]>
Date: Tue Mar 2 23:19:12 2021 -0600
Added trace logging to help watch throttle cohort assignment
redis/measured_reporter.go | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
commit 51cd50205001f4b80115e9045c50482c5ba193b4
Author: Myles Horton <[email protected]>
Date: Mon Feb 15 13:21:26 2021 -0600
Log details
README.md | 7 +++++++
1 file changed, 7 insertions(+)
commit bbb13b862e06553ed4c73bd559d9567a4bbf5c06
Author: hwh33 <[email protected]>
Date: Fri Feb 12 09:02:15 2021 -0700
Update tlsmasq (#454)
go.mod | 2 +-
go.sum | 16 ++++++++++------
2 files changed, 11 insertions(+), 7 deletions(-)
commit e459a3a09d13e2cbcac7427f62e9af18c59ed2ea
Author: Ox Cart <[email protected]>
Date: Mon Feb 8 13:33:40 2021 -0600
Updated changelog for v2.6.31
CHANGELOG.md | 20 +++++++++++++++-----
1 file changed, 15 insertions(+), 5 deletions(-)
commit 1307ddde14f049911974d4e49f3400ac0d34045f (tag: v2.6.31)
Merge: 460a349 23a7bd6
Author: Percy Wegmann <[email protected]>
Date: Mon Feb 8 13:31:11 2021 -0600
Merge pull request #453 from getlantern/ox/issue452
Ignore empty array headers for measured
There is currently one later version, git-chglog v0.1.0
, but it failed to compile for me.
Discovered after working with tests.
The reason for the segfault seems to be resource exhaustion.
This probably comes as a result of an infinite loop of redirection. As a protection, we had a filter for local interfaces addresses, but since UDPgw needed that to be taken out, we are currently exposed.
The solution seems simple: reactivate loop-avoidance filter but with the exception of UDPgw port. I will test this further.
We've seen reports of people launching DDOS attacks on destination sites through the proxy. We should disallow it if we can.
This has been bothering me for a while. I thought it was due to a problem with our code, but I wrote a little test Go program that works just fine, and in fact just proxying through my Lantern client without hitting http-proxy, it's fine too.
package main
import (
"io"
"io/ioutil"
"log"
"net/http"
"time"
)
func main() {
start := time.Now()
resp, err := http.Get("http://austin.craigslist.org/")
if err != nil {
log.Fatal(err)
}
defer resp.Body.Close()
n, err := io.Copy(ioutil.Discard, resp.Body)
if err != nil {
log.Fatal(err)
}
delta := time.Now().Sub(start)
log.Print(delta)
log.Print(n)
}
go run main.go
2016/10/21 15:04:00 184.968188ms
2016/10/21 15:04:00 56259
56K in 184 ms
I then ran my test program on my proxy server and found that it was dog slow:
lantern@fp-obfs4-donyc3-20160925-001:~$ ./craigslist
2016/10/21 20:08:09 40.679969133s
2016/10/21 20:08:09 56259
56K in 40 seconds!
Curl is even slower, so slow that I stopped it at 43K
lantern@fp-obfs4-donyc3-20160925-001:~$ curl http://austin.craigslist.org/ > /dev/null
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 42842 0 42842 0 0 254 0 --:--:-- 0:02:48 --:--:-- 202
Suspecting that Craigslist is throttling known proxy servers, I spooled up another donyc3 machine and tried my test from there. It was also really slow, so perhaps they're throttling the entire datacenter.
root@craigslist:~# ./craigslist
2016/10/21 20:16:56 41.130397822s
2016/10/21 20:16:56 56259
In any case, this is not a problem with our code, so I'm just logging this for the record (in case it comes up again).
Too much time wasted every time you want to remember this.
As opposed to #90
Possible way: https://redislabs.com/blog/using-stunnel-to-secure-redis
Obfuscated QUIC that attempts to bypass attacks that block all QUIC traffic
In a production proxy, it makes the difference between about 0.4-0.5 vs 1.0+ load.
Note - the throttling limits are set high enough that no one should be getting throttled, so this load is purely from having the latent capability.
Here are some 2 minute profiles
Simplify and try to clean up the dependency structure.
We suspect that the data cap drives some attrition of free users. We should quantify this.
Updated KCP with pre-shared secret keys that enable full encryption of all handshakes and payloads, multiplexing, and forward-error-correction
For mobile users to sign up for Pro using Alipay, we have to redirect them to the browser to open Stripe Checkout. [1] Once payment information is verified by Stripe, the user is re-directed back to the app, along with the Stripe token and e-mail address. This is done using an intent filter with a custom scheme, defined in the AndroidManifest
, which tells the browser to open our app directly. Only back in the app do we submit the Pro token, device ID, and Stripe credentials to the /purchase
endpoint.
The biggest issue with this is approach is that newer versions of Chrome on Android are blocking re-directs to custom URIs completely (unless the app is registered in the Play store). I can get this to work reliably in the stock browser though. But we can't depend on this..
A possible solution would be to configure a webhook to detect whenever a user first submits their credit card or Alipay information to Stripe. We'd have to pass along the pro token and device id somehow. Stripe would respond to this event, instead of returning a token, by hitting our /purchase
endpoint. Then whenever the user re-opens the app, we would know via the /user-data
call a user is Pro and be able to display a welcome message/switch the UI to Pro mode.
[1] Issues loading Stripe Checkout in a WebView necessitate re-directing the user to the browser
I see messages in the log for fp-obfs4-ggtwnb-20160912-060 like this:
Oct 01 02:58:41.258 DEBUG devicefilter: devicefilter.go:90 Throttling connection [client_ip=119.248.246.109 device_id=QmidpI/c op=tokenfilter origin=config.getiantem.org root_op=proxy]
Note - I see this for other IPs and device IDs on this server too.
In theory, this is a pro-only server, so all connecting clients should be pro, so nothing should get throttled.
The purpose of this is to prevent people from abusing Lantern infrastructure by distributing their pro token to other people.
Looks very likely this is related to the bandwidth tracking issues we saw earlier @oxtoacart, @aranhoide, and @uaalto. This proxy, for example, has almost no traffic but the http-proxy process is swapping like crazy:
New error occurring with high frequency in StackDriver. This would likely affect throttling and bandwidth caps. Could it also cause proxies to run afoul of blocking detection?
This eliminates the perverse incentive not to upgrade in order to avoid being capped.
Unblock Lampshade in Iran with properties including pre-shared keys, multiplexing, network message coalescing, and random padding
Right now, sites like Google will misidentify traffic coming through our proxies as a bot, and will show captchas and such annoying things to force people to prove they're not bots. Similar logic is also used by content providers to prohibit the use of proxies.
We could address both issues by:
Related to getlantern/lantern#2645
This started with @aranhoide noticing that the proxy checkers were getting 502 errors. I added logging to see what causes the 502s, and there are various root causes. Below is a sampling of the errors from one server. Not sure if it's relevant, but interestingly all of the sites in question seem to be Chinese sites.
May 13 02:45:56.015 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:43564->123.58.182.253:80: use of closed network connection: Error forwarding from 14.23.61.75:42217 to notify3.ynote.youdao.com
May 13 02:45:56.084 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:23719 to hb.crm2.qq.com
May 13 02:45:58.789 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 14.18.205.103:58148 to hb.crm2.qq.com
May 13 02:45:59.470 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:43623->123.58.182.253:80: use of closed network connection: Error forwarding from 14.23.114.74:37633 to notify3.note.youdao.com
May 13 02:46:01.052 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:23719 to hb.crm2.qq.com
May 13 02:46:02.309 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:43669->123.58.182.253:80: use of closed network connection: Error forwarding from 14.18.205.103:65380 to notify3.ynote.youdao.com
May 13 02:46:05.085 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 14.18.205.103:58148 to hb.crm2.qq.com
May 13 02:46:07.029 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:23719 to hb.crm2.qq.com
May 13 02:46:07.073 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:43780->123.58.182.253:80: use of closed network connection: Error forwarding from 1.202.253.18:59705 to notify3.note.youdao.com
May 13 02:46:07.508 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:50499->123.125.65.210:80: use of closed network connection: Error forwarding from 14.23.80.167:23383 to message.tieba.baidu.com
May 13 02:46:08.021 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to dial tcp 1.1.1.1:80: getsockopt: network is unreachable: Error forwarding from 39.191.141.238:50420 to 1.1.1.1
May 13 02:46:08.744 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:43801->123.58.182.253:80: use of closed network connection: Error forwarding from 14.23.61.75:44500 to notify3.note.youdao.com
May 13 02:46:10.065 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 14.18.205.103:58148 to hb.crm2.qq.com
May 13 02:46:13.098 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:23719 to hb.crm2.qq.com
May 13 02:46:16.085 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 14.18.205.103:58148 to hb.crm2.qq.com
May 13 02:46:16.369 ERROR errorhandler: errorhandler.go:36 Responding with 504 due to dial tcp 198.11.189.98:80: i/o timeout: Error forwarding from 42.49.164.195:53687 to alimei-sub.alibaba.com
May 13 02:46:16.759 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:23719 to hb.crm2.qq.com
May 13 02:46:16.804 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to dial tcp 218.30.114.205:80: getsockopt: connection refused: Error forwarding from 1.202.222.147:30673 to online.sso.sina.com.cn
May 13 02:46:18.552 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:43913->123.58.182.253:80: use of closed network connection: Error forwarding from 1.202.169.226:51567 to notify3.ynote.youdao.com
May 13 02:46:19.116 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:26018 to hb.crm2.qq.com
May 13 02:46:20.157 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:44411->203.205.144.215:80: read: connection reset by peer: Error forwarding from 14.29.125.102:54274 to masterconn.qq.com
May 13 02:46:21.776 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 14.18.205.103:58148 to hb.crm2.qq.com
May 13 02:46:21.914 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:55707->180.149.138.108:80: use of closed network connection: Error forwarding from 1.202.15.102:59406 to 11.108.web1.im.weibo.com
May 13 02:46:22.072 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:23719 to hb.crm2.qq.com
May 13 02:46:22.822 ERROR errorhandler: errorhandler.go:36 Responding with 504 due to dial tcp 106.120.160.106:80: i/o timeout: Error forwarding from 1.202.193.65:51318 to 106.120.160.106
May 13 02:46:24.136 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:26018 to hb.crm2.qq.com
May 13 02:46:28.080 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 14.18.205.103:58148 to hb.crm2.qq.com
May 13 02:46:28.081 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:26018 to hb.crm2.qq.com
May 13 02:46:29.780 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:23719 to hb.crm2.qq.com
May 13 02:46:32.016 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:44023->123.58.182.253:80: use of closed network connection: Error forwarding from 14.18.207.75:52267 to notify3.note.youdao.com
May 13 02:46:32.273 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:44094->123.58.182.253:80: use of closed network connection: Error forwarding from 14.23.114.74:7233 to notify3.note.youdao.com
May 13 02:46:32.475 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:44098->123.58.182.253:80: use of closed network connection: Error forwarding from 14.18.236.70:11659 to notify3.note.youdao.com
May 13 02:46:33.049 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 14.18.205.103:58148 to hb.crm2.qq.com
May 13 02:46:33.074 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:26018 to hb.crm2.qq.com
May 13 02:46:33.401 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:44116->123.58.182.253:80: use of closed network connection: Error forwarding from 1.202.237.242:54824 to notify3.note.youdao.com
May 13 02:46:35.068 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:23719 to hb.crm2.qq.com
May 13 02:46:38.045 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 14.18.205.103:58148 to hb.crm2.qq.com
May 13 02:46:38.070 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:26018 to hb.crm2.qq.com
May 13 02:46:39.252 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:50912->123.125.65.210:80: use of closed network connection: Error forwarding from 14.23.80.167:24398 to message.tieba.baidu.com
May 13 02:46:39.455 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:44204->123.58.182.253:80: use of closed network connection: Error forwarding from 14.23.61.75:50814 to notify3.note.youdao.com
May 13 02:46:40.028 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:23719 to hb.crm2.qq.com
May 13 02:46:43.773 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 14.18.205.103:58148 to hb.crm2.qq.com
May 13 02:46:44.075 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:23719 to hb.crm2.qq.com
May 13 02:46:46.103 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:26018 to hb.crm2.qq.com
May 13 02:46:46.813 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:44292->123.58.182.253:80: use of closed network connection: Error forwarding from 14.18.207.75:52290 to notify3.note.youdao.com
May 13 02:46:49.079 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 14.18.205.103:58148 to hb.crm2.qq.com
May 13 02:46:49.097 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:23719 to hb.crm2.qq.com
May 13 02:46:49.369 ERROR errorhandler: errorhandler.go:36 Responding with 504 due to dial tcp 198.11.189.98:80: i/o timeout: Error forwarding from 42.49.164.195:53687 to alimei-sub.alibaba.com
May 13 02:46:50.958 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to read tcp 45.32.60.169:44383->123.58.182.253:80: use of closed network connection: Error forwarding from 14.18.207.75:61524 to notify3.note.youdao.com
May 13 02:46:51.092 ERROR errorhandler: errorhandler.go:36 Responding with 502 due to EOF: Error forwarding from 1.202.252.174:26018 to hb.crm2.qq.com
In GA code. This might be related to excessive requests. This PR could be the fix.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.