Giter VIP home page Giter VIP logo

netbare-android's Introduction

NetBare

NetBare是一款网络包拦截和注入框架,可以实现抓包、屏蔽包、改包等各种强大功能。NetBare核心是基于VPN技术,将网络包转发到本地代理服务器,再通过虚拟网关(VirtualGateway)进行拦截分发。在设计上,虚拟网关层是完全对外开放的,开发者可以自由定义虚拟网关,也可以使用NetBare内部已实现的虚拟网关进行网络包的处理。

NetBare初始化

  1. 在Application中绑定
NetBare.get().attachApplication(application, BuildConfig.DEBUG)
  1. 创建自签证书(SSL需要)
val jks = JKS(context, alias, password, commonName, organization,
        organizationalUnitName, certOrganization, certOrganizationalUnitName)
  1. 安装自签证书(SSL需要)
// 判断证书是否安装
JKS.isInstalled(context, alias)
// 安装证书
JKS.install(context, name, alias)
  1. 创建NetBare服务。由于NetBare具有危险性,所以在设计上会强制要求在通知栏提示用户服务正在运行,同时要在Manifest中配置Service。
// 继承NetBareService创建自己应用的Service
class AppService : NetBareService() {
        override fun notificationId(): Int {
            // 通知栏ID
        }
        override fun createNotification(): Notification {
            // 创建一个Notification
        }
}
<service android:name=".AppService"
            android:permission="android.permission.BIND_VPN_SERVICE">
            <intent-filter>
                <action android:name="android.net.VpnService" />
                <action android:name="com.github.megatronking.netbare.action.Start"/>
                <action android:name="com.github.megatronking.netbare.action.Stop"/>
            </intent-filter>
</service>
  1. NetBare服务的启动和停止
// 通过NetBareConfig自由配置NetBare服务并启动
NetBare.get().start(NetBareConfig)
// 停止NetBare服务
NetBare.get().stop()

NetBareConfig配置

NetBareConfig需要使用NetBareConfig.Builder进行构造,解释下以下几个重要的配置方法。

  • setMtu 最大传输单元,必要,建议大于2048。
  • setAddress 本地代理服务器IP地址,必要,建议用A类IP地址,防止冲突。
  • addRoute 设置经过VPN的目标IP包,必要,建议使用0.0.0.0,所有IP全部经过VPN。
  • dumpUid 是否dump网络包所属的uid,可选,耗电方法,建议false
  • setVirtualGatewayFactory 配置虚拟网关,可选。

NetBare框架提供了默认的NetBareConfig来快速集成:

// 创建默认的NetBareConfig,作用于所有IP协议
val config = NetBareConfig.defaultConfig()
// 为Http协议创建默认的NetBareConfig
val config = NetBareConfig.defaultHttpConfig(jks, interceptors)

NetBare虚拟网关

虚拟网关是对网络包进行拦截、解析、注入的核心,可以加载开发者自定义的拦截器,通过NetBareConfig.Builder来配置。NetBare框架提供了两个默认的虚拟网关对象。

DefaultVirtualGateway

默认虚拟网关,可以拦截到所有协议的网络包。默认虚拟网关无法直接构造,需要通过DefaultVirtualGatewayFactory来进行构造。开发者可以使用NetBareConfig.setVirtualGatewayFactory配置默认虚拟网关工厂。

// 配置自定义拦截器
val interceptors = listOf(...)
//  创建默认虚拟网关工厂
val defaultGatewayFactory = DefaultVirtualGatewayFactory(interceptors)
// 通过NetBareConfig.Builder来配置defaultGatewayFactory
...

虚拟网关拦截器,继承Interceptor。Interceptor使用工厂模式,由InterceptorFactory来构造。

class TestIntercepter : Interceptor {

    @Throws(IOException::class)
    override fun intercept(chain: RequestChain, buffer: ByteBuffer) {
        // 对请求包进行自定义处理
        ...
        // 将请求发射出去,交给下个拦截器或者发给服务器
        chain.process(buffer)
    }

    @Throws(IOException::class)
    override fun intercept(chain: ResponseChain, buffer: ByteBuffer) {
        // 对响应包进行处理
        ...
        // 将响应发射出去,交给下个拦截器或者发给客户端
        chain.process(buffer)
    }

    override fun onRequestFinished(request: Request) {
        // 请求包已全部发送完成
    }

    override fun onResponseFinished(response: Response) {
        // 响应包已全部发送完成
    }
}

HttpVirtualGateway

Http协议虚拟网关,可以拦截到所有Http协议的网络包。Http协议虚拟网关也无法直接构造,需要通过HttpVirtualGatewayFactory来进行构造。创建HttpVirtualGatewayFactory实例需要前面的JKS以及拦截器HttpInterceptor。

// 配置自定义HttpInterceptor
val interceptors = listOf(...)
//  创建Http虚拟网关工厂
val httpGatewayFactory = HttpVirtualGatewayFactory(jks, interceptors)
// 通过NetBareConfig.Builder来配置httpGatewayFactory
...

Http虚拟网关拦截器,继承HttpInterceptor。同样的HttpInterceptor也使用工厂模式,由HttpInterceptorFactory来构造。

class TestHttpIntercepter : HttpInterceptor() {

    override fun intercept(chain: HttpRequestChain, buffer: ByteBuffer) {
        // 对Http请求包进行自定义处理
        ...
        // 将Http请求发射出去,交给下个拦截器或者发给服务器
        chain.process(buffer)
    }

    override fun intercept(chain: HttpResponseChain, buffer: ByteBuffer) {
        // 对Http响应包进行自定义处理
        ...
        // 将Http响应发射出去,交给下个拦截器或者发给客户端
        chain.process(buffer)
    }

    override fun onRequestFinished(request: HttpRequest) {
        // Http请求包已全部发送完成
    }

    override fun onResponseFinished(response: HttpResponse) {
        // Http响应包已全部发送完成
    }

}

此外,NetBare框架内置了HttpIndexInterceptor等特殊拦截器方便开发者使用。

注入器Injector

NetBare提供了一套通用的injector框架来方便开发者注入请求和响应。Injector框架是基于虚拟网关的拦截器来运行的。目前只开发了针对Http协议的注入器,其它协议可以后续再扩展。

Http协议的注入是基于HttpInjectInterceptor拦截器来实现的,在配置HttpVirtualGatewayFactory的时候,需要配置此拦截器,每一个注入器对应一个拦截器实例。而所有注入器都必须实现HttpInjector接口。

class TestHttpInjector : HttpInjector {

    override fun sniffRequest(request: HttpRequest): Boolean {
        // 对request进行判定,是否需要注入。true表示需要注入。
        return false
    }

    override fun sniffResponse(response: HttpResponse): Boolean {
        // 对response进行判定,是否需要注入。true表示需要注入。
        return false
    }

    @Throws(IOException::class)
    override fun onRequestInject(header: HttpRequestHeaderPart,
                                 callback: InjectorCallback) {
        // 当sniffRequest返回true时,会走到此方法。
        // 对请求头部进行注入,包括method、url、headers都可以修改
        ...
        // 将注入完成后将新的数据发射出去
        callback.onFinished(header)
    }

    @Throws(IOException::class)
    override fun onResponseInject(header: HttpResponseHeaderPart,
                                  callback: InjectorCallback) {
        // 当sniffResponse返回true时,会走到此方法。
        // 对响应头部进行注入,包括code、message、headers都可以修改
        ...
        // 将注入完成后将新的数据发射出去
        callback.onFinished(header)
    }

    @Throws(IOException::class)
    override fun onRequestInject(request: HttpRequest, body: HttpBody,
                                 callback: InjectorCallback) {
        // 当sniffRequest返回true时,会走到此方法。
        // 对请求体进行注入,如果请求体数据较大,会多次走到此方法。
        ...
        // 将注入完成后将新的数据发射出去
        callback.onFinished(body)
    }

    @Throws(IOException::class)
    override fun onResponseInject(response: HttpResponse, body: HttpBody,
                                  callback: InjectorCallback) {
        // 当sniffResponse返回true时,会走到此方法。
        // 对响应体进行注入,如果请求体数据较大,会多次走到此方法。
        ...
        // 将注入完成后将新的数据发射出去
        callback.onFinished(body)
    }

    override fun onRequestFinished(request: HttpRequest) {
        // Http请求包已全部发送完成
    }

    override fun onResponseFinished(response: HttpResponse) {
        // Http响应包已全部发送完成
    }

将注入器绑定到拦截器,并装载到虚拟网关中:

// 配置自定义HttpInjectInterceptor
val interceptor1 = HttpInjectInterceptor.createFactory(injector1)
val interceptor2 = HttpInjectInterceptor.createFactory(injector2)
...
val interceptors = listOf(interceptor1, interceptor2 ...)
//  创建Http虚拟网关工厂
val httpGatewayFactory = HttpVirtualGatewayFactory(jks, interceptors)
// 通过NetBareConfig.Builder来配置httpGatewayFactory
...

NetBare范例

NetBare的接入步骤就有些繁琐,所以提供了一个简单的Sample工程供大家参考。sample中包含三个比较有趣的东西:

  • 拦截器1:打印所有Http请求的URL。
  • 注入器1:将百度首页的logo图片修改成自定义的图片。
  • 注入器2:将发朋友圈的定位地点修改到珠峰。

结语

NetBare框架尚未完全成熟,仍然有很多工作要做,包括ICMP、IGMP等IP协议的转发等等,后续会继续完善。

基于NetBare实现的一款抓包+注入工具,欢迎大家下载体验:https://play.google.com/store/apps/details?id=com.guoshi.httpcanary

声明:DON'T BE EVIL!NetBare只可用于学习和调试,禁止用于网络恶意攻击和钓鱼等非法途径

netbare-android's People

Contributors

cuisoap avatar djdisodo avatar joseluisametller avatar megatronking avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

netbare-android's Issues

Request access uri not changed

I hope that when I access http://192.168.0.101:5050 access to http://192.168.0.101:5050/aaaaa

But it didn't work. How can I rule out ? Please.

Thanks.

Here is my code:

    override fun sniffRequest(request: HttpRequest): Boolean {
        val shouldRequestInject = request.url().startsWith("http://192.168.0.101:5050")
        if (shouldRequestInject) {
            Log.i(TAG, "sniffRequest")
        }
        return shouldRequestInject
    }

    override fun onRequestInject(header: HttpRequestHeaderPart, callback: InjectorCallback) {
        Log.i(TAG, "test onRequestInject")
        Log.i(TAG, header.uri().toString())
        header.newBuilder()
                .method(HttpMethod.POST)
                .uri(Uri.parse("http://192.168.0.101:5050/aaaaa"))
                .build()

        callback.onFinished(header)
    }

Logs:

I/NetBare: [TCP][192.168.0.101:5050]Connect to remote server /192.168.0.101:5050
    TCP connects to: 192.168.0.101:5050
I/NetBare: [TCP][192.168.0.101:5050]Connect to remote server /192.168.0.101:5050
    [TCP][192.168.0.101:5050]Remote tunnel is connected.
    [TCP][192.168.0.101:5050]Read from proxy: 473
I/TestInjector: sniffRequest
I/TestInjector: test onRequestInject
    http://192.168.0.101:5050/api
I/NetBare: [TCP][192.168.0.101:5050]Write to remote: 473
    [TCP][192.168.0.101:5050]Remote tunnel is connected.

Android 9 deprecated functionality SHA512WITHRSA

On Android 9 devices, looking in the logcat screen, the following warning regularly appears:

2019-01-31 13:16:50.180 3029-5047/com.github.megatronking.netbare.sample E/System: ******** DEPRECATED FUNCTIONALITY ********
2019-01-31 13:16:50.180 3029-5047/com.github.megatronking.netbare.sample E/System: * The implementation of the Signature.SHA512WITHRSA algorithm from
2019-01-31 13:16:50.180 3029-5047/com.github.megatronking.netbare.sample E/System: * the BC provider is deprecated in this version of Android.
2019-01-31 13:16:50.180 3029-5047/com.github.megatronking.netbare.sample E/System: * It will be removed in a future version of Android and your
2019-01-31 13:16:50.180 3029-5047/com.github.megatronking.netbare.sample E/System: * application will no longer be able to request it. Please see
2019-01-31 13:16:50.180 3029-5047/com.github.megatronking.netbare.sample E/System: * https://android-developers.googleblog.com/2018/03/cryptography-changes-in-android-p.html
2019-01-31 13:16:50.180 3029-5047/com.github.megatronking.netbare.sample E/System: * for more details.
2019-01-31 13:16:50.194 3029-5047/com.github.megatronking.netbare.sample E/System: ******** DEPRECATED FUNCTIONALITY ********
2019-01-31 13:16:50.194 3029-5047/com.github.megatronking.netbare.sample E/System: * The implementation of the CertificateFactory.X.509 algorithm from
2019-01-31 13:16:50.194 3029-5047/com.github.megatronking.netbare.sample E/System: * the BC provider is deprecated in this version of Android.
2019-01-31 13:16:50.194 3029-5047/com.github.megatronking.netbare.sample E/System: * It will be removed in a future version of Android and your
2019-01-31 13:16:50.194 3029-5047/com.github.megatronking.netbare.sample E/System: * application will no longer be able to request it. Please see
2019-01-31 13:16:50.194 3029-5047/com.github.megatronking.netbare.sample E/System: * https://android-developers.googleblog.com/2018/03/cryptography-changes-in-android-p.html
2019-01-31 13:16:50.194 3029-5047/com.github.megatronking.netbare.sample E/System: * for more details.
2019-01-31 13:16:50.198 3029-5047/com.github.megatronking.netbare.sample E/System: ******** DEPRECATED FUNCTIONALITY ********
2019-01-31 13:16:50.198 3029-5047/com.github.megatronking.netbare.sample E/System: * The implementation of the Signature.SHA512WITHRSA algorithm from
2019-01-31 13:16:50.198 3029-5047/com.github.megatronking.netbare.sample E/System: * the BC provider is deprecated in this version of Android.
2019-01-31 13:16:50.198 3029-5047/com.github.megatronking.netbare.sample E/System: * It will be removed in a future version of Android and your
2019-01-31 13:16:50.198 3029-5047/com.github.megatronking.netbare.sample E/System: * application will no longer be able to request it. Please see
2019-01-31 13:16:50.198 3029-5047/com.github.megatronking.netbare.sample E/System: * https://android-developers.googleblog.com/2018/03/cryptography-changes-in-android-p.html
2019-01-31 13:16:50.198 3029-5047/com.github.megatronking.netbare.sample E/System: * for more details.

NetBare still works fine, in spite of the warning.

As stated, more information on how to resolve this issue can be found at https://android-developers.googleblog.com/2018/03/cryptography-changes-in-android-p.html. Reproducible on physical 9.0 device and also on 9.0 emulator.

Broken pipe error occurs occasionally

Device: Android 8.1
Latest version of NetBare Sample

I visited the following page: https://www.travelden.co.uk/20-forbidden-places-you-can-never-visit?utm_source=taboola&utm_medium=referral&utm_term=tmg-birminghammail&utm_content=157819389&utm_site=tmg-birminghammail&utm_campaign_item=157819389&utm_campaign=ForbiddenDestinations-GlobalC and an error was reported in logcat:

2019-02-18 13:39:36.083 11516-11581/com.github.megatronking.netbare.sample E/NetBare: Broken pipe
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:53)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:65)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:510)
at com.github.megatronking.netbare.tunnel.TcpTunnel.channelWrite(TcpTunnel.java:77)
at com.github.megatronking.netbare.tunnel.NioTunnel.onWrite(NioTunnel.java:114)
at com.github.megatronking.netbare.proxy.TcpProxyServer.process(TcpProxyServer.java:141)
at com.github.megatronking.netbare.proxy.BaseProxyServer.run(BaseProxyServer.java:62)
at com.github.megatronking.netbare.proxy.TcpProxyServer.run(TcpProxyServer.java:108)
at java.lang.Thread.run(Thread.java:764)
2019-02-18 13:39:36.108 11516-11581/com.github.megatronking.netbare.sample E/NetBare: Broken pipe
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:53)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:65)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:510)
at com.github.megatronking.netbare.tunnel.TcpTunnel.channelWrite(TcpTunnel.java:77)
at com.github.megatronking.netbare.tunnel.NioTunnel.onWrite(NioTunnel.java:114)
at com.github.megatronking.netbare.proxy.TcpProxyServer.process(TcpProxyServer.java:141)
at com.github.megatronking.netbare.proxy.BaseProxyServer.run(BaseProxyServer.java:62)
at com.github.megatronking.netbare.proxy.TcpProxyServer.run(TcpProxyServer.java:108)
at java.lang.Thread.run(Thread.java:764)

Was not able to reproduce this error by re-loading the page, so it may be a random error that is difficult to reproduce.

The page still loaded fine in Chrome, there were no visible signs of a problem.

EOFException: Read error

Device: Android 8.1
Latest version of NetBare Sample

I visited the following page: https://www.espn.com/soccer/scoreboard and an error was reported in logcat:

2019-02-18 13:27:30.008 11516-11581/com.github.megatronking.netbare.sample E/NetBare: java.io.EOFException: Read error

Here's the lines before and after the error:

2019-02-18 13:27:30.001 11516-11581/com.github.megatronking.netbare.sample I/URL: WKP Request: https://cto.fn-pz.com/v1/web
2019-02-18 13:27:30.002 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][54.242.122.197:443]Encode a http2 frame: HEADERS stream(1) length(386)
2019-02-18 13:27:30.002 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][54.242.122.197:443]Decode a http2 frame: DATA stream(1) length(817)
2019-02-18 13:27:30.002 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][54.242.122.197:443]Http2 stream end: 1
2019-02-18 13:27:30.002 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][54.242.122.197:443]Encode a http2 frame: DATA stream(1) length(817)
2019-02-18 13:27:30.003 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][104.89.45.59:443]Read from remote: 4096
2019-02-18 13:27:30.003 11516-11581/com.github.megatronking.netbare.sample W/NetBare: No enough ssl/tls packet length, packet: 16413 actual: 12288
2019-02-18 13:27:30.004 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][5.226.178.246:443]Read from remote: 841
2019-02-18 13:27:30.006 11516-11581/com.github.megatronking.netbare.sample I/URL: WKP Response: https://imstore.bet365affiliates.com/?AffiliateCode=365_374860&CID=196&DID=299&TID=1&PID=149&LNG=1
2019-02-18 13:27:30.006 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][5.226.178.246:443]Write to proxy: 419
2019-02-18 13:27:30.007 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][5.226.178.246:443]Write to proxy: 83
2019-02-18 13:27:30.007 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][5.226.178.246:443]Write to proxy: 161
2019-02-18 13:27:30.007 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][5.226.178.246:443]Write to proxy: 45
2019-02-18 13:27:30.008 11516-11581/com.github.megatronking.netbare.sample E/NetBare: java.io.EOFException: Read error
2019-02-18 13:27:30.009 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][5.226.178.246:443]Proxy tunnel is closed.
2019-02-18 13:27:30.009 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][5.226.178.246:443]Remote tunnel is closed.
2019-02-18 13:27:30.010 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][5.226.178.246:443]Gateway request finished!
2019-02-18 13:27:30.010 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][5.226.178.246:443]Gateway response finished!
2019-02-18 13:27:30.011 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][104.89.45.59:443]Read from remote: 4096

Was able to reproduce this error by re-loading the page, so hopefully it is easy to reproduce (did not try on other devices yet).

The page still loaded fine in Chrome, there were no visible signs of a problem.

Page not loading correctly on some URLs

We are seeing on some pages that they are not being loaded correctly and the screen remains blank in Chrome, e.g.

https://meteoexploration.com/

In logcat, we see several entries like this:

2019-02-15 13:07:48.896 1331-1569/com.github.megatronking.netbare.sample E/NetBare: java.io.EOFException: Read error

Error is reproducible. Tested with latest Netbare sample on Android 8.1 device

想通过这个库做一个简单代理,有好的意见吗?

比如https://github.com/MegatronKing/NetBare/issues/new
跳转到http://192.168.100.1:8888/MegatronKing/NetBare/issues/new
直接更改onRequestInject回调中header的uri是不行的。
现在我有两种想法。

  • 一种是直接在TcpVATunnel的的回调中的request中获取header、body、url等信息,然后自行发送网络请求。收到response之后,通过 mGateway.onResponse(buffer)来设置返回信息。

  • 另一种是和有赞团队一样https://www.infoq.cn/article/akWjUR42QEvqUF*bP9kE 从vpnService中直接获取数据然后通过Socket发送出去,在自己的服务器(192.168.100.1)中做转发。

不知哪种可行。

通过参数配置只拦截http

安卓 7.0 后不再默认信任用户证书
又不想使用虚拟环境
希望可以通过配置只拦截http
因为现在好像不安装证书的话https会出错。

INTERNAL_ERROR reported on some pages

Device: Android 8.1
Latest version of NetBare Sample

I visited the following page: http://www.espncricinfo.com/series/18913/game/1158065/uwi-vice-chancellors-xi-vs-england-tour-match-england-tour-of-wi-2018-19 and an error was reported in logcat:

2019-02-18 12:49:20.264 11516-11581/com.github.megatronking.netbare.sample E/NetBare: [TCP][2.21.170.249:443]Http2 TYPE_GOAWAY error code: INTERNAL_ERROR last stream: 0

Here's the lines before and after the error:

2019-02-18 12:49:18.480 11516-11579/com.github.megatronking.netbare.sample I/NetBare: [UDP][172.217.168.162:443]Write to remote: 28
2019-02-18 12:49:20.262 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][2.21.170.249:443]Read from remote: 77
2019-02-18 12:49:20.263 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][2.21.170.249:443]Decode a http2 frame: GOAWAY stream(0) length(8)
2019-02-18 12:49:20.264 11516-11581/com.github.megatronking.netbare.sample E/NetBare: [TCP][2.21.170.249:443]Http2 TYPE_GOAWAY error code: INTERNAL_ERROR last stream: 0
2019-02-18 12:49:20.264 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][2.21.170.249:443]Write to proxy: 46
2019-02-18 12:49:20.266 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][2.21.170.249:443]Read from remote: -1
2019-02-18 12:49:20.266 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][2.21.170.249:443]Remote tunnel is closed.
2019-02-18 12:49:20.266 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][2.21.170.249:443]Gateway request finished!
2019-02-18 12:49:20.269 11516-11579/com.github.megatronking.netbare.sample V/NetBare: ip: 10.1.10.1:39402 -> 2.21.170.249:44907
2019-02-18 12:49:20.269 11516-11579/com.github.megatronking.netbare.sample V/NetBare: tcp: ACKPSH 39402 -> 44907 -525999401:1783992291, size: 46
2019-02-18 12:49:20.270 11516-11579/com.github.megatronking.netbare.sample V/NetBare: ip: 10.1.10.1:44907 -> 2.21.170.249:443
2019-02-18 12:49:20.270 11516-11579/com.github.megatronking.netbare.sample V/NetBare: tcp: ACK 44907 -> 443 1783992291:-525999355, size: 0
2019-02-18 12:49:21.080 11516-11581/com.github.megatronking.netbare.sample E/NetBare: Connection reset by peer
2019-02-18 12:49:21.081 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][198.105.197.235:443]Proxy tunnel is closed.
2019-02-18 12:49:21.081 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][198.105.197.235:443]Remote tunnel is closed.
2019-02-18 12:49:21.082 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][198.105.197.235:443]Gateway request finished!
2019-02-18 12:49:21.083 11516-11581/com.github.megatronking.netbare.sample I/NetBare: [TCP][198.105.197.235:443]Gateway response finished!
2019-02-18 12:49:21.086 11516-11579/com.github.megatronking.netbare.sample V/NetBare: ip: 10.1.10.1:39402 -> 198.105.197.235:47768
2019-02-18 12:49:21.087 11516-11579/com.github.megatronking.netbare.sample V/NetBare: tcp: ACKFIN 39402 -> 47768 1693217116:1110106827, size: 0

Was able to reproduce this error by re-loading the page, and also saw it on other pages, so hopefully it is easy to reproduce (did not try on other devices yet).

The page still loaded fine in Chrome, there were no visible signs of a problem.

onResponseFinished没有被回调

使用SimpleHttpInjector进行http抓包时发现onResponseFinished有很大的概率不会被回调,只有少数请求会回调,这个是什么原因?

服务启动后无法关闭

我运行了NetBare-sample这个例子,可以正常关闭服务,但是当我把NetBareConfig.java中的

public static NetBareConfig defaultConfig() {
return new Builder()
.dumpUid(false)
.setMtu(4096)
.setAddress(new IpAddress("10.1.10.1", 32))
.setSession("NetBare")
.addRoute(new IpAddress("0.0.0.0", 0))
.build();
}

中的addRoute(new IpAddress("0.0.0.0", 0))更改为addRoute(new IpAddress("183.192.199.205", 32)),应用启动服务后无法关闭,然而在半年前的一个版本的源码里,这样的修改是没有问题的,这次clone了最新的源码,就出现了这样的问题

作者是否有打算支持TLS1.3的打算

Enforce TLS 1.2 if available, since it's not default up to Java 8.
由于JDK的限制,TLS1.3在Java8上无法支持,但是现在很多应用,如Ins已经全量切TLS1.3,这在日常竞品分析中有很大区别。所以作者打算一起研究支持TLS1.3吗?

Occasional SSL error

Device: Android 8.1
Latest version of NetBare Sample

I am occasionally seeing the following error being reported in logcat when navigating with Chrome to different pages:

2019-02-25 11:47:58.857 12434-13122/com.github.megatronking.netbare.sample E/NetBare: Read error: ssl=0xa807b180: Failure in SSL library, usually a protocol error
error:10000416:SSL routines:OPENSSL_internal:SSLV3_ALERT_CERTIFICATE_UNKNOWN (external/boringssl/src/ssl/tls_record.cc:579 0x97773ae0:0x00000001)

Was not able to reproduce this error by re-loading the page, so it may be a random error that is difficult to reproduce.

The page still loaded fine in Chrome, there were no visible signs of a problem.

Random crash on some web pages

We observed the following crash occurred randomly when loading some pages (this one happened when loading a page on https://www.marca.com, but was not reproducible as it did not crash when opening the same page a second time):

2019-01-31 12:16:14.369 27534-28206/? I/NetBare: [TCP][105.53.65.253:443]Write to proxy: 38
2019-01-31 12:16:14.369 27534-28206/? I/NetBare: [TCP][214.191.162.41:443]Read from remote: 3031
2019-01-31 12:16:14.384 27534-28206/? I/NetBare: [TCP][214.191.162.41:443]Write to remote: 318
2019-01-31 12:16:14.384 27534-28206/? E/NativeCrypto: AppData::create pipe(2) failed: Too many open files
2019-01-31 12:16:14.388 27534-28206/? E/AndroidRuntime: FATAL EXCEPTION: TcpProxyServer
Process: com.github.megatronking.netbare.sample, PID: 27534
java.lang.RuntimeException: javax.net.ssl.SSLException: Unable to create application data
at com.android.org.conscrypt.ConscryptEngine.newSsl(ConscryptEngine.java:198)
at com.android.org.conscrypt.ConscryptEngine.(ConscryptEngine.java:173)
at com.android.org.conscrypt.OpenSSLContextImpl.engineCreateSSLEngine(OpenSSLContextImpl.java:138)
at javax.net.ssl.SSLContext.createSSLEngine(SSLContext.java:363)
at com.github.megatronking.netbare.ssl.SSLEngineFactory.createServerEngine(SSLEngineFactory.java:122)
at com.github.megatronking.netbare.ssl.SSLRequestCodec.createEngine(SSLRequestCodec.java:65)
at com.github.megatronking.netbare.http.SSLHttpRequestCodec.createEngine(SSLHttpRequestCodec.java:50)
at com.github.megatronking.netbare.ssl.SSLCodec.decode(SSLCodec.java:137)
at com.github.megatronking.netbare.http.SSLCodecInterceptor.decodeRequest(SSLCodecInterceptor.java:204)
at com.github.megatronking.netbare.http.SSLCodecInterceptor.access$200(SSLCodecInterceptor.java:40)
at com.github.megatronking.netbare.http.SSLCodecInterceptor$1.onResult(SSLCodecInterceptor.java:126)
at com.github.megatronking.netbare.http.SSLHttpResponseCodec.decode(SSLHttpResponseCodec.java:74)
at com.github.megatronking.netbare.http.SSLCodecInterceptor.decodeResponse(SSLCodecInterceptor.java:232)
at com.github.megatronking.netbare.http.SSLCodecInterceptor.intercept(SSLCodecInterceptor.java:152)
at com.github.megatronking.netbare.http.HttpIndexInterceptor.intercept(HttpIndexInterceptor.java:71)
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:55)
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:32)
at com.github.megatronking.netbare.gateway.InterceptorChain.process(InterceptorChain.java:95)
at com.github.megatronking.netbare.http.HttpSniffInterceptor.intercept(HttpSniffInterceptor.java:71)
at com.github.megatronking.netbare.http.HttpIndexInterceptor.intercept(HttpIndexInterceptor.java:71)
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:55)
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:32)
at com.github.megatronking.netbare.gateway.InterceptorChain.process(InterceptorChain.java:95)
at com.github.megatronking.netbare.http.HttpVirtualGateway.onSpecResponse(HttpVirtualGateway.java:118)
at com.github.megatronking.netbare.gateway.SpecVirtualGateway.sendResponse(SpecVirtualGateway.java:78)
at com.github.megatronking.netbare.NetBareVirtualGateway.sendResponse(NetBareVirtualGateway.java:102)
at com.github.megatronking.netbare.tunnel.TcpVATunnel$2.onRead(TcpVATunnel.java:134)
at com.github.megatronking.netbare.tunnel.NioTunnel.onRead(NioTunnel.java:101)
at com.github.megatronking.netbare.proxy.TcpProxyServer.process(TcpProxyServer.java:139)
at com.github.megatronking.netbare.proxy.BaseProxyServer.run(BaseProxyServer.java:62)
at com.github.megatronking.netbare.proxy.TcpProxyServer.run(TcpProxyServer.java:108)
at java.lang.Thread.run(Thread.java:764)
Caused by: javax.net.ssl.SSLException: Unable to create application data
at com.android.org.conscrypt.NativeCrypto.SSL_new(Native Method)
at com.android.org.conscrypt.SslWrapper.newInstance(SslWrapper.java:58)
at com.android.org.conscrypt.ConscryptEngine.newSsl(ConscryptEngine.java:196)
at com.android.org.conscrypt.ConscryptEngine.(ConscryptEngine.java:173)
at com.android.org.conscrypt.OpenSSLContextImpl.engineCreateSSLEngine(OpenSSLContextImpl.java:138)
at javax.net.ssl.SSLContext.createSSLEngine(SSLContext.java:363)
at com.github.megatronking.netbare.ssl.SSLEngineFactory.createServerEngine(SSLEngineFactory.java:122)
at com.github.megatronking.netbare.ssl.SSLRequestCodec.createEngine(SSLRequestCodec.java:65)
at com.github.megatronking.netbare.http.SSLHttpRequestCodec.createEngine(SSLHttpRequestCodec.java:50)
at com.github.megatronking.netbare.ssl.SSLCodec.decode(SSLCodec.java:137)
at com.github.megatronking.netbare.http.SSLCodecInterceptor.decodeRequest(SSLCodecInterceptor.java:204)
at com.github.megatronking.netbare.http.SSLCodecInterceptor.access$200(SSLCodecInterceptor.java:40)
at com.github.megatronking.netbare.http.SSLCodecInterceptor$1.onResult(SSLCodecInterceptor.java:126)
at com.github.megatronking.netbare.http.SSLHttpResponseCodec.decode(SSLHttpResponseCodec.java:74)
at com.github.megatronking.netbare.http.SSLCodecInterceptor.decodeResponse(SSLCodecInterceptor.java:232)
at com.github.megatronking.netbare.http.SSLCodecInterceptor.intercept(SSLCodecInterceptor.java:152)
at com.github.megatronking.netbare.http.HttpIndexInterceptor.intercept(HttpIndexInterceptor.java:71)
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:55)
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:32)
at com.github.megatronking.netbare.gateway.InterceptorChain.process(InterceptorChain.java:95)
at com.github.megatronking.netbare.http.HttpSniffInterceptor.intercept(HttpSniffInterceptor.java:71)
at com.github.megatronking.netbare.http.HttpIndexInterceptor.intercept(HttpIndexInterceptor.java:71)
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:55)
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:32)
at com.github.megatronking.netbare.gateway.InterceptorChain.process(InterceptorChain.java:95)
at com.github.megatronking.netbare.http.HttpVirtualGateway.onSpecResponse(HttpVirtualGateway.java:118)
at com.github.megatronking.netbare.gateway.SpecVirtualGateway.sendResponse(SpecVirtualGateway.java:78)
at com.github.megatronking.netbare.NetBareVirtualGateway.sendResponse(NetBareVirtualGateway.java:102)
at com.github.megatronking.netbare.tunnel.TcpVATunnel$2.onRead(TcpVATunnel.java:134)
at com.github.megatronking.netbare.tunnel.NioTunnel.onRead(NioTunnel.java:101)
at com.github.megatronking.netbare.proxy.TcpProxyServer.process(TcpProxyServer.java:139)
at com.github.megatronking.netbare.proxy.BaseProxyServer.run(BaseProxyServer.java:62)
at com.github.megatronking.netbare.proxy.TcpProxyServer.run(TcpProxyServer.java:108)
at java.lang.Thread.run(Thread.java:764)
2019-01-31 12:16:14.408 1724-1811/? I/ActivityManager: Showing crash dialog for package com.github.megatronking.netbare.sample u0
2019-01-31 12:16:14.420 1724-28695/? W/DropBoxManagerService: Dropping: data_app_crash (3606 > 0 bytes)
2019-01-31 12:16:14.427 1724-1810/? W/BroadcastQueue: Background execution not allowed: receiving Intent { act=android.intent.action.DROPBOX_ENTRY_ADDED flg=0x10 (has extras) } to com.google.android.gms/.stats.service.DropBoxEntryAddedReceiver
2019-01-31 12:16:14.427 1724-1810/? W/BroadcastQueue: Background execution not allowed: receiving Intent { act=android.intent.action.DROPBOX_ENTRY_ADDED flg=0x10 (has extras) } to com.google.android.gms/.chimera.GmsIntentOperationService$PersistentTrustedReceiver
2019-01-31 12:16:14.502 30383-30383/? W/cr_Autocomplete: stopping autocomplete.

A similar crash happened on a different page. Not 100% reproducible, seems to happen randomly.

Some application do not work when NetBare is running

Android: 7.1.2
NetBare: the latest (commit a42f9e4)

Noticed that many applications (PlayStore, Twitter, Facebook, Reddit, Slack) on Android phone do not work properly or doesn't work at all when NetBare is running. My assumption is that this is due to certificate pinning. Sometimes I see errors in log: Handshake failed and Connection reset by peer

Do you have any idea why this is happening? Not sure what additional information I may provide to help resolve this. Anyway I am opened to help with this issues.

MeiZuPro7 Root手机SSL握手失败,证书已信任

2019-10-20 20:56:24.570 30291-30724/com.github.megatronking.netbare.sample E/NetBare: Handshake failed: Invalid handshake status: NOT_HANDSHAKING java.io.IOException: Handshake failed: Invalid handshake status: NOT_HANDSHAKING at com.github.megatronking.netbare.ssl.SSLCodec.handshake(SSLCodec.java:219) at com.github.megatronking.netbare.ssl.SSLCodec.startDecode(SSLCodec.java:192) at com.github.megatronking.netbare.ssl.SSLCodec.decode(SSLCodec.java:172) at com.github.megatronking.netbare.ssl.SSLCodec.decode(SSLCodec.java:143) at com.github.megatronking.netbare.http.HttpSSLResponseCodec.decode(HttpSSLResponseCodec.java:71) at com.github.megatronking.netbare.http.HttpSSLCodecInterceptor.decodeResponse(HttpSSLCodecInterceptor.java:233) at com.github.megatronking.netbare.http.HttpSSLCodecInterceptor.intercept(HttpSSLCodecInterceptor.java:154) at com.github.megatronking.netbare.http.HttpSSLCodecInterceptor.intercept(HttpSSLCodecInterceptor.java:41) at com.github.megatronking.netbare.gateway.IndexedInterceptor.intercept(IndexedInterceptor.java:73) at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:55) at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:32) at com.github.megatronking.netbare.gateway.InterceptorChain.process(InterceptorChain.java:104) at com.github.megatronking.netbare.http.HttpSniffInterceptor.intercept(HttpSniffInterceptor.java:71) at com.github.megatronking.netbare.http.HttpSniffInterceptor.intercept(HttpSniffInterceptor.java:34) at com.github.megatronking.netbare.gateway.IndexedInterceptor.intercept(IndexedInterceptor.java:73) at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:55) at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:32) at com.github.megatronking.netbare.gateway.InterceptorChain.process(InterceptorChain.java:104) at com.github.megatronking.netbare.http.HttpVirtualGateway.onSpecResponse(HttpVirtualGateway.java:126) at com.github.megatronking.netbare.gateway.SpecVirtualGateway.onResponse(SpecVirtualGateway.java:85) at com.github.megatronking.netbare.NetBareVirtualGateway.onResponse(NetBareVirtualGateway.java:102) at com.github.megatronking.netbare.tunnel.TcpVATunnel$2.onRead(TcpVATunnel.java:134) at com.github.megatronking.netbare.tunnel.NioTunnel.onRead(NioTunnel.java:101) at com.github.megatronking.netbare.proxy.TcpProxyServer.process(TcpProxyServer.java:139) at com.github.megatronking.netbare.proxy.BaseProxyServer.run(BaseProxyServer.java:62) at com.github.megatronking.netbare.proxy.TcpProxyServer.run(TcpProxyServer.java:108) at java.lang.Thread.run(Thread.java:761)

Crash on Android 8.1 devices

We are running some tests with netbare-sample. It is working fine on Android 7 and Android 9 devices that we have tested. However, on all Android 8.1 devices that we test, after turning on the VPN and browsing https pages using Chrome, the following crash occurs (100% reproducible on our devices):

2019-01-30 12:30:54.022 12391-12451/com.github.megatronking.netbare.sample E/NetBare: java.io.EOFException: Read error
javax.net.ssl.SSLException: java.io.EOFException: Read error
at com.android.org.conscrypt.SSLUtils.toSSLException(SSLUtils.java:295)
at com.android.org.conscrypt.ConscryptEngine.convertException(ConscryptEngine.java:1093)
at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:851)
at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:678)
at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:644)
at com.github.megatronking.netbare.ssl.SSLCodec.engineUnwrap(SSLCodec.java:368)
at com.github.megatronking.netbare.ssl.SSLCodec.unwrap(SSLCodec.java:303)
at com.github.megatronking.netbare.ssl.SSLCodec.startDecode(SSLCodec.java:175)
at com.github.megatronking.netbare.ssl.SSLCodec.decode(SSLCodec.java:166)
at com.github.megatronking.netbare.ssl.SSLCodec.decode(SSLCodec.java:137)
at com.github.megatronking.netbare.http.SSLHttpResponseCodec.decode(SSLHttpResponseCodec.java:70)
at com.github.megatronking.netbare.http.SSLCodecInterceptor.decodeResponse(SSLCodecInterceptor.java:232)
at com.github.megatronking.netbare.http.SSLCodecInterceptor.intercept(SSLCodecInterceptor.java:152)
at com.github.megatronking.netbare.http.HttpIndexInterceptor.intercept(HttpIndexInterceptor.java:71)
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:55)
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:32)
at com.github.megatronking.netbare.gateway.InterceptorChain.process(InterceptorChain.java:95)
at com.github.megatronking.netbare.http.HttpSniffInterceptor.intercept(HttpSniffInterceptor.java:71)
at com.github.megatronking.netbare.http.HttpIndexInterceptor.intercept(HttpIndexInterceptor.java:71)
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:55)
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:32)
at com.github.megatronking.netbare.gateway.InterceptorChain.process(InterceptorChain.java:95)
at com.github.megatronking.netbare.http.HttpVirtualGateway.onSpecResponse(HttpVirtualGateway.java:117)
at com.github.megatronking.netbare.gateway.SpecVirtualGateway.sendResponse(SpecVirtualGateway.java:78)
at com.github.megatronking.netbare.NetBareVirtualGateway.sendResponse(NetBareVirtualGateway.java:102)
at com.github.megatronking.netbare.tunnel.TcpVATunnel$2.onRead(TcpVATunnel.java:134)
at com.github.megatronking.netbare.tunnel.NioTunnel.onRead(NioTunnel.java:101)
at com.github.megatronking.netbare.proxy.TcpProxyServer.process(TcpProxyServer.java:137)
at com.github.megatronking.netbare.proxy.BaseProxyServer.run(BaseProxyServer.java:62)
at com.github.megatronking.netbare.proxy.TcpProxyServer.run(TcpProxyServer.java:106)
at java.lang.Thread.run(Thread.java:764)
Caused by: java.io.EOFException: Read error
at com.android.org.conscrypt.NativeCrypto.ENGINE_SSL_read_direct(Native Method)
at com.android.org.conscrypt.SslWrapper.readDirectByteBuffer(SslWrapper.java:492)
at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataDirect(ConscryptEngine.java:1052)
at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataHeap(ConscryptEngine.java:1072)
at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1044)
at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:805)
at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:678) 
at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:644) 
at com.github.megatronking.netbare.ssl.SSLCodec.engineUnwrap(SSLCodec.java:368) 
at com.github.megatronking.netbare.ssl.SSLCodec.unwrap(SSLCodec.java:303) 
at com.github.megatronking.netbare.ssl.SSLCodec.startDecode(SSLCodec.java:175) 
at com.github.megatronking.netbare.ssl.SSLCodec.decode(SSLCodec.java:166) 
at com.github.megatronking.netbare.ssl.SSLCodec.decode(SSLCodec.java:137) 
at com.github.megatronking.netbare.http.SSLHttpResponseCodec.decode(SSLHttpResponseCodec.java:70) 
at com.github.megatronking.netbare.http.SSLCodecInterceptor.decodeResponse(SSLCodecInterceptor.java:232) 
at com.github.megatronking.netbare.http.SSLCodecInterceptor.intercept(SSLCodecInterceptor.java:152) 
at com.github.megatronking.netbare.http.HttpIndexInterceptor.intercept(HttpIndexInterceptor.java:71) 
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:55) 
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:32) 
at com.github.megatronking.netbare.gateway.InterceptorChain.process(InterceptorChain.java:95) 
at com.github.megatronking.netbare.http.HttpSniffInterceptor.intercept(HttpSniffInterceptor.java:71) 
at com.github.megatronking.netbare.http.HttpIndexInterceptor.intercept(HttpIndexInterceptor.java:71) 
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:55) 
at com.github.megatronking.netbare.http.HttpResponseChain.processNext(HttpResponseChain.java:32) 
at com.github.megatronking.netbare.gateway.InterceptorChain.process(InterceptorChain.java:95) 
at com.github.megatronking.netbare.http.HttpVirtualGateway.onSpecResponse(HttpVirtualGateway.java:117) 
at com.github.megatronking.netbare.gateway.SpecVirtualGateway.sendResponse(SpecVirtualGateway.java:78) 
at com.github.megatronking.netbare.NetBareVirtualGateway.sendResponse(NetBareVirtualGateway.java:102) 
at com.github.megatronking.netbare.tunnel.TcpVATunnel$2.onRead(TcpVATunnel.java:134) 
at com.github.megatronking.netbare.tunnel.NioTunnel.onRead(NioTunnel.java:101) 
at com.github.megatronking.netbare.proxy.TcpProxyServer.process(TcpProxyServer.java:137) 
at com.github.megatronking.netbare.proxy.BaseProxyServer.run(BaseProxyServer.java:62) 
at com.github.megatronking.netbare.proxy.TcpProxyServer.run(TcpProxyServer.java:106) 
at java.lang.Thread.run(Thread.java:764) 

Looks like there is a problem with ConscryptEngine on Android 8.1 devices that causes this error.

Can anyone reproduce this on an Android 8.1 device?

This site can't be reached error on some pages

We are seeing a reproducible error in some URLs, e.g.

https://www.elotrolado.net/index.php

In logcat we see:

2019-02-15 13:35:06.912 1331-1569/com.github.megatronking.netbare.sample E/NetBare: Read error: ssl=0x966eca80: Failure in SSL library, usually a protocol error
error:10000103:SSL routines:OPENSSL_internal:INVALID_ALPN_PROTOCOL (external/boringssl/src/ssl/t1_lib.cc:1464 0xa6f2c9e0:0x00000000)
error:10000095:SSL routines:OPENSSL_internal:ERROR_PARSING_EXTENSION (external/boringssl/src/ssl/t1_lib.cc:2979 0x96551a00:0x00000001)
error:100000be:SSL routines:OPENSSL_internal:PARSE_TLSEXT (external/boringssl/src/ssl/handshake_client.cc:1049 0xa6f2c9e0:0x00000000)

and in Chrome it shows:

This site can't be reached.
ERR_CONNECTION_CLOSED

and does not load the page.

Tested with latest NetBare sample on Android 8.1 and 9.0 devices

Not in GZIP format

When my response body is too large,I met this promblem:
java.util.zip.ZipException: Not in GZIP format
and this is my code:
var res=GZIPInputStream(HttpBodyInputStream(body)).bufferedReader(UTF_8).readText()
how to solve this problem

Crash on Android 8.0 (SDK 26) devices

I did some tests on an Android 8.0 (SDK 26) device and also with the Android Studio 8.0 emulator. On both devices, as soon as the VPN tries to connect, the device crashes (and actually goes into an unstable state which needs a reboot to return to normal). Issue is 100% reproducible, I was never able to start the VPN without this crash occurring. Please let me know if you need more information

Crash when visiting cnn.com

Tested and reproducible on Android 7, 8.1 and 9 devices

On visiting cnn.com, the page is redirected to https://edition.cnn.com

If this page is opened in Chrome, it is not loaded correctly (screen remains blank) and the following crash occurs:

2019-01-31 13:23:32.418 3029-5047/com.github.megatronking.netbare.sample E/NetBare: Connection reset by peer
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:55)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:65)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:512)
at com.github.megatronking.netbare.tunnel.TcpTunnel.channelWrite(TcpTunnel.java:77)
at com.github.megatronking.netbare.tunnel.NioTunnel.onWrite(NioTunnel.java:114)
at com.github.megatronking.netbare.proxy.TcpProxyServer.process(TcpProxyServer.java:141)
at com.github.megatronking.netbare.proxy.BaseProxyServer.run(BaseProxyServer.java:62)
at com.github.megatronking.netbare.proxy.TcpProxyServer.run(TcpProxyServer.java:108)
at java.lang.Thread.run(Thread.java:764)

Increasing targetSdkVersion to 28 for netbare-sample causes it to fail on Android 9+ devices

Hi,

Device: Android 9 or Android Q Beta 2
Netbare sample not installed when starting test

targetSdkVersion for netbare-sample is currently set to 27. Increasing it to 28 (as it is for the netbare-injector and netbare-core) causes netbare-sample to report the following error after install:

2019-04-05 17:48:59.765 9972-9993/com.github.megatronking.netbare.sample E/NetBare: cannot create signer: The BC provider no longer provides an implementation for Signature.SHA512WITHRSA. Please see https://android-developers.googleblog.com/2018/03/cryptography-changes-in-android-p.html for more details.

Trying to start the service has no effect.

If NetBare Sample was installed with an older version with the certificate installed, the following error occurs after starting the service:

2019-04-05 11:49:00.375 10021-10112/com.github.megatronking.netbare.sample E/NetBare: Failed to create server SSLEngine: org.bouncycastle.operator.OperatorCreationException: cannot create signer: The BC provider no longer provides an implementation for Signature.SHA512WITHRSA. Please see https://android-developers.googleblog.com/2018/03/cryptography-changes-in-android-p.html for more details.

Looks like the issue is related to https://github.com/MegatronKing/NetBare/issues/6. This functionality is deprecated in SDK 28.

Since Google will be forcing us to increase targetSDK to 28 on 1 August 2019, this issue is quite serious and would need to be resolved soon: https://developer.android.com/distribute/best-practices/develop/target-sdk

Note that in order to build with targetSDK 28, it is necessary to add the following to the manifest:

<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />

    android:extractNativeLibs="true"

Possible false positive error being reported?

Device: Android 8.1
Latest version of NetBare Sample

On many different pages an error is sometimes reported in logcat, e.g.:

2019-02-18 11:16:33.998 10376-10444/com.github.megatronking.netbare.sample E/NetBare: [TCP][62.210.94.130:443]Http2 TYPE_GOAWAY error code: NO_ERROR last stream: 3

The number for last stream varies, but the error ode NO_ERROR seems to indicate that no problem has occurred.

Should the code be changed to not report this as an ERROR in the case where errorCode == "NO_ERROR" ? Something like:

if (errorCode == "NO_ERROR")
mLog.i("Http2 TYPE_GOAWAY error code: " + errorCode + " last stream: " + lastStreamId);
else
mLog.e("Http2 TYPE_GOAWAY error code: " + errorCode + " last stream: " + lastStreamId);

The page still loaded fine in Chrome, there were no visible signs of a problem.

suddenly crashes

V/NetBare: ip: 10.1.10.1:54181 -> 108.177.97.188:59351
V/NetBare: tcp: ACKPSH 54181 -> 59351 1195385186:312892864, size: 225
V/NetBare: ip: 10.1.10.1:59351 -> 108.177.97.188:5228
V/NetBare: tcp: ACK 59351 -> 5228 312892864:1195385411, size: 0
I/totypeg.androi: Thread[3,tid=23987,WaitingInMainSignalCatcherLoop,Thread*=0x7f0475e000,peer=0x78480088,"Signal Catcher"]: reacting to signal 3
I/totypeg.androi: Wrote stack traces to '[tombstoned]'

H2协议支持问题

hi ,
我最近用NetBare抓取apk信息时,发现如果apk同时包含http1.1和http2协议时,http2协议的网址均没有访问记录,请问是不支持H2协议吗?
举例:用httpcanary抓取阴阳师游戏的包体时,阴阳师游戏无法登录。通过wireshark看到是带有https://g37.update.netease.com 开头的网址均无访问记录。
补充说明:我已经在httpcanary中安装了平行空间,用平行空间打开阴阳师游戏仍然不行,应该和CA证书限制无关?
经过排查后,发现一个问题,比如在访问https://drpf-g37.proxima.nie.netease.com/ 请求时,其实是h2协议,但是代码中选择的是Http1.1
I/NetBare: TCP connects to: 42.186.69.71:443
I/NetBare: [TCP][42.186.69.71:443]Connect to remote server /42.186.69.71:443
I/NetBare: [TCP][42.186.69.71:443]Remote tunnel is connected.
I/NetBare: [TCP][42.186.69.71:443]Read from proxy: 198
I/NetBare: [TCP][42.186.69.71:443]host is: drpf-g37.proxima.nie.netease.com
I/NetBare: [TCP][42.186.69.71:443]Read from remote: 2895
I/NetBare: [TCP][42.186.69.71:443]Server selected ALPN protocol: HTTP/1.1
I/NetBare: [TCP][42.186.69.71:443]Write to proxy: 1923
麻烦大神帮忙看下,感谢!

how to modify the content of a html page?

override fun onResponseInject(response: HttpResponse, body: HttpBody, callback: InjectorCallback)

this function is called for several times, how i kown when the body is finished?
and how to get the string format of body and modify the body then send out?

Sample not working on Google Pixel 3a with Android 10

I tried your application on different devices and versions of Android and I didn't have any issue until I tried it on the Google Pixel 3a. After enabling the VPN from the sample application the network stops working. The following image is the screenshot of the error message Google Chrome gives me.

screenshot

The error it gives me is DNS_PROBE_FINISHED_NXDOMAIN which just means it was not able to resolve the name. Below is the log of the sample app, I hope it will be helpful.

                NetBare  I  Start NetBare service!
                         V  [TCP]proxy server: 10.1.10.1:40797
                         W  Unknown ip protocol: 0
                         W  Unknown ip protocol: -128
                         W  Unknown ip protocol: -128
                         I  [TCP]Server starts running.
                         W  Unknown ip protocol: -128
                         I  [UDP]Server starts running.
                         W  Unknown ip protocol: -128
                         W  Unknown ip protocol: -128
                         V  ip: 10.1.10.1:37873 -> 216.58.208.164:443
                         V  udp: 37873 -> 443, size: 1350
                         I  UDP connects to: 216.58.208.164:443
                         I  [UDP][216.58.208.164:443]Connect to remote server /216.58.208.164:443
                         I  [UDP][216.58.208.164:443]Write to remote: 1350
                         I  [UDP][216.58.208.164:443]Read from remote: 1350
                         I  [UDP][216.58.208.164:443]Read from remote: 1350
                         V  ip: 10.1.10.1:37873 -> 216.58.208.164:443
                         V  udp: 37873 -> 443, size: 36
                         I  [UDP][216.58.208.164:443]Write to remote: 36
                         V  ip: 10.1.10.1:37873 -> 216.58.208.164:443
                         V  udp: 37873 -> 443, size: 1350
                         I  [UDP][216.58.208.164:443]Write to remote: 1350
                         V  ip: 10.1.10.1:37873 -> 216.58.208.164:443
                         V  udp: 37873 -> 443, size: 488
                         I  [UDP][216.58.208.164:443]Write to remote: 488
                         W  Unknown ip protocol: -128
                         I  [UDP][216.58.208.164:443]Read from remote: 1350
                         I  [UDP][216.58.208.164:443]Read from remote: 20
                         I  [UDP][216.58.208.164:443]Read from remote: 373
                         I  [UDP][216.58.208.164:443]Read from remote: 220
                         V  ip: 10.1.10.1:37873 -> 216.58.208.164:443
                         V  udp: 37873 -> 443, size: 28
                         I  [UDP][216.58.208.164:443]Write to remote: 28
                         V  ip: 10.1.10.1:37873 -> 216.58.208.164:443
                         V  udp: 37873 -> 443, size: 28
                         I  [UDP][216.58.208.164:443]Write to remote: 28
                         W  Unknown ip protocol: -128
                         V  ip: 10.1.10.1:37873 -> 216.58.208.164:443
                         V  udp: 37873 -> 443, size: 23
                         I  [UDP][216.58.208.164:443]Write to remote: 23
                         I  [UDP][216.58.208.164:443]Read from remote: 20
                         V  ip: 10.1.10.1:37514 -> 74.125.71.188:5228
                         V  tcp: SYN 37514 -> 5228 -782190166:0, size: 0
                         V  ip: 10.1.10.1:40797 -> 74.125.71.188:37514
                         V  tcp: SYNACK 40797 -> 37514 1203069732:-782190165, size: 0
                         V  ip: 10.1.10.1:37514 -> 74.125.71.188:5228
                         V  tcp: ACK 37514 -> 5228 -782190165:1203069733, size: 0
                         V  ip: 10.1.10.1:37514 -> 74.125.71.188:5228
                         V  tcp: ACKPSH 37514 -> 5228 -782190165:1203069733, size: 533
                         V  ip: 10.1.10.1:40797 -> 74.125.71.188:37514
                         I  TCP connects to: 74.125.71.188:5228
                         V  tcp: ACK 40797 -> 37514 1203069733:-782189632, size: 0
                         I  [TCP][74.125.71.188:5228]Connect to remote server /74.125.71.188:5228
                         I  [TCP][74.125.71.188:5228]Remote tunnel is connected.
                         I  [TCP][74.125.71.188:5228]Read from proxy: 533
                         I  detect whitelist ip 74.125.71.188
                         I  [TCP][74.125.71.188:5228]Write to remote: 533
                         I  [TCP][74.125.71.188:5228]Read from remote: 203
                         I  [TCP][74.125.71.188:5228]Write to proxy: 203
                         V  ip: 10.1.10.1:40797 -> 74.125.71.188:37514
                         V  tcp: ACKPSH 40797 -> 37514 1203069733:-782189632, size: 203
                         V  ip: 10.1.10.1:37514 -> 74.125.71.188:5228
                         V  tcp: ACK 37514 -> 5228 -782189632:1203069936, size: 0
                         V  ip: 10.1.10.1:37514 -> 74.125.71.188:5228
                         V  tcp: ACKPSH 37514 -> 5228 -782189632:1203069936, size: 64
                         V  ip: 10.1.10.1:40797 -> 74.125.71.188:37514
                         I  [TCP][74.125.71.188:5228]Read from proxy: 64
                         I  [TCP][74.125.71.188:5228]Write to remote: 64
                         V  tcp: ACK 40797 -> 37514 1203069936:-782189568, size: 0
                         V  ip: 10.1.10.1:37514 -> 74.125.71.188:5228
                         V  tcp: ACKPSH 37514 -> 5228 -782189568:1203069936, size: 375
                         I  [TCP][74.125.71.188:5228]Read from proxy: 375
                         I  [TCP][74.125.71.188:5228]Write to remote: 375
                         V  ip: 10.1.10.1:40797 -> 74.125.71.188:37514
                         V  tcp: ACK 40797 -> 37514 1203069936:-782189193, size: 0
                         I  [TCP][74.125.71.188:5228]Read from remote: 529
                         I  [TCP][74.125.71.188:5228]Write to proxy: 529
                         I  [TCP][74.125.71.188:5228]Read from remote: 225
                         I  [TCP][74.125.71.188:5228]Write to proxy: 225
                         V  ip: 10.1.10.1:40797 -> 74.125.71.188:37514
                         V  tcp: ACKPSH 40797 -> 37514 1203069936:-782189193, size: 529
                         V  ip: 10.1.10.1:37514 -> 74.125.71.188:5228
                         V  tcp: ACK 37514 -> 5228 -782189193:1203070465, size: 0
                         V  ip: 10.1.10.1:40797 -> 74.125.71.188:37514
                         V  tcp: ACKPSH 40797 -> 37514 1203070465:-782189193, size: 225
                         V  ip: 10.1.10.1:37514 -> 74.125.71.188:5228
                         V  tcp: ACK 37514 -> 5228 -782189193:1203070690, size: 0
                         V  ip: 10.1.10.1:5741 -> 8.8.8.8:53
                         V  udp: 5741 -> 53, size: 28
                         I  UDP connects to: 8.8.8.8:53
                         I  [UDP][8.8.8.8:53]Connect to remote server /8.8.8.8:53
                         I  [UDP][8.8.8.8:53]Write to remote: 28
                         I  [UDP][8.8.8.8:53]Read from remote: 44
                         W  Unknown ip protocol: -128
                         V  ip: 10.1.10.1:54823 -> 8.8.8.8:53
                         V  udp: 54823 -> 53, size: 28
                         I  UDP connects to: 8.8.8.8:53
                         I  [UDP][8.8.8.8:53]Connect to remote server /8.8.8.8:53
                         I  [UDP][8.8.8.8:53]Write to remote: 28
                         I  [UDP][8.8.8.8:53]Read from remote: 44
                         W  Unknown ip protocol: -128
                         V  ip: 10.1.10.1:20158 -> 8.8.8.8:53
                         V  udp: 20158 -> 53, size: 28
                         I  UDP connects to: 8.8.8.8:53
                         I  [UDP][8.8.8.8:53]Connect to remote server /8.8.8.8:53
                         I  [UDP][8.8.8.8:53]Write to remote: 28
                         I  [UDP][8.8.8.8:53]Read from remote: 44
                         V  ip: 10.1.10.1:48052 -> 216.58.198.10:443
                         V  tcp: SYN 48052 -> 443 -2111608431:0, size: 0
                         V  ip: 10.1.10.1:40797 -> 216.58.198.10:48052
                         V  tcp: SYNACK 40797 -> 48052 -2045457795:-2111608430, size: 0
                         V  ip: 10.1.10.1:48052 -> 216.58.198.10:443
                         V  tcp: ACK 48052 -> 443 -2111608430:-2045457794, size: 0
                         I  TCP connects to: 216.58.198.10:443
                         V  ip: 10.1.10.1:48052 -> 216.58.198.10:443
                         V  tcp: ACKPSH 48052 -> 443 -2111608430:-2045457794, size: 563
                         I  [TCP][216.58.198.10:443]Connect to remote server /216.58.198.10:443
                         V  ip: 10.1.10.1:40797 -> 216.58.198.10:48052
                         V  tcp: ACK 40797 -> 48052 -2045457794:-2111607867, size: 0
                         I  [TCP][216.58.198.10:443]Remote tunnel is connected.
                         I  [TCP][216.58.198.10:443]Read from proxy: 563
                         I  detect whitelist ip 216.58.198.10
                         I  [TCP][216.58.198.10:443]Write to remote: 563
                         I  [TCP][216.58.198.10:443]Read from remote: 218
                         I  [TCP][216.58.198.10:443]Write to proxy: 218
                         V  ip: 10.1.10.1:40797 -> 216.58.198.10:48052
                         V  tcp: ACKPSH 40797 -> 48052 -2045457794:-2111607867, size: 218
                         V  ip: 10.1.10.1:48052 -> 216.58.198.10:443
                         V  tcp: ACK 48052 -> 443 -2111607867:-2045457576, size: 0
                         V  ip: 10.1.10.1:48052 -> 216.58.198.10:443
                         V  tcp: ACKPSH 48052 -> 443 -2111607867:-2045457576, size: 64
                         I  [TCP][216.58.198.10:443]Read from proxy: 64
                         I  [TCP][216.58.198.10:443]Write to remote: 64
                         V  ip: 10.1.10.1:40797 -> 216.58.198.10:48052
                         V  tcp: ACK 40797 -> 48052 -2045457576:-2111607803, size: 0
                         V  ip: 10.1.10.1:48052 -> 216.58.198.10:443
                         V  tcp: ACKPSH 48052 -> 443 -2111607803:-2045457576, size: 3250
                         I  [TCP][216.58.198.10:443]Read from proxy: 3250
                         I  [TCP][216.58.198.10:443]Write to remote: 3250
                         V  ip: 10.1.10.1:40797 -> 216.58.198.10:48052
                         V  tcp: ACK 40797 -> 48052 -2045457576:-2111604553, size: 0
                         I  [TCP][216.58.198.10:443]Read from remote: 1531
                         I  [TCP][216.58.198.10:443]Write to proxy: 1531
                         V  ip: 10.1.10.1:40797 -> 216.58.198.10:48052
                         V  tcp: ACKPSH 40797 -> 48052 -2045457576:-2111604553, size: 1531
                         V  ip: 10.1.10.1:48052 -> 216.58.198.10:443
                         V  tcp: ACK 48052 -> 443 -2111604553:-2045456045, size: 0
                         I  Stop NetBare service!
                         I  [UDP][8.8.8.8:53]Remote tunnel is closed.
                         I  [TCP]Server stops running.
                         I  [UDP][8.8.8.8:53]Gateway request finished!
                         I  [UDP][8.8.8.8:53]Gateway response finished!
                         I  [UDP][8.8.8.8:53]Remote tunnel is closed.
                         I  [UDP][8.8.8.8:53]Gateway request finished!
                         I  [UDP][8.8.8.8:53]Gateway response finished!
                         I  [UDP][216.58.208.164:443]Remote tunnel is closed.
                         I  [UDP][216.58.208.164:443]Gateway request finished!
                         I  [UDP][216.58.208.164:443]Gateway response finished!
                         I  [UDP][8.8.8.8:53]Remote tunnel is closed.
                         I  [UDP][8.8.8.8:53]Gateway request finished!
                         I  [UDP][8.8.8.8:53]Gateway response finished!
                         I  [UDP]Server stops running.

Here some more information on the device.

version

On a side note, the application works on the emulator device with Android 10.

How can I change the response to json?

The browser will always request when running the program.
Always request

Here is my code:

    override fun sniffResponse(response: HttpResponse): Boolean {
        val shouldInject = response.url().startsWith("https://xx.xx/a")
        if (shouldInject) {
            Log.i(TAG, "Start test injection!")
        }
        return shouldInject
    }

    override fun onResponseInject(header: HttpResponseHeaderPart, callback: InjectorCallback) {
        callback.onFinished(header)
    }

    override fun onResponseInject(response: HttpResponse, body: HttpBody, callback: InjectorCallback) {
          val jsonStr = "{\"a\": 1}"

          Log.i(TAG, "Inject body completed!")

          callback.onFinished(StringStream(jsonStr))
    }

I tried a lot of urls and only succeeded once.
Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.