Giter VIP home page Giter VIP logo

wechat_automated_jump_game's Introduction

自动玩微信跳一跳小游戏

利用MonkeyRunner(WebDriverAgent)和OpenCV在Android(iOS)设备上自动玩微信跳一跳小游戏。

效果:http://t.cn/RH939gQ

依赖

运行服务端

控制Android设备

  • monkeyrunner
    • 位于 Android SDK 中:tools/bin/monkeyrunner
  • adb
    • 位于 Android SDK 中:platform-tools/adb

控制iOS设备

原理说明

本项目通过图像处理的方式从屏幕截图中判断棋子当前位置和目标棋盘位置。

  1. 使用 Template Matching 的方式在截图中识别棋子当前位置。
  2. 对截图进行边缘检测,然后在合适的区域内自上而下寻找第一个形状的顶部中心点和最右点,再计算出目标跳跃位置。
  3. 计算棋子当前位置到目标跳跃位置的距离。
  4. 将距离映射为按压时间。
  5. 控制设备进行模拟点击操作。

项目结构说明

项目主要分为两部分:计算跳跃时间的服务端(server.py)和执行设备控制操作的客户端,分为控制Android设备的MonkeyRunner脚本(monkeyrunner.py)和控制iOS设备的WDA脚本(wda.py)。

执行设备控制操作的客户端首先对设备进行截图,然后将截图通过POST方式发送到服务端的HTTP接口上,计算跳跃时间的服务端对截图进行处理并返回对应的按压时间,客户端在设备上模拟点击操作。

操作步骤

Step 1 - 构建服务端C拓展(可选)

在项目根目录下运行 python setup.py build_ext --inplace 即可。该步骤可选,不执行该步骤则关键步骤使用纯Python实现,执行该步骤则使用Cython实现。

Step 2 - 启动服务端

在项目根目录下运行 python server.py 即可。服务端默认监听 127.0.0.1:5000。可选启动参数见 python server.py -h

Step 3 - 启动设备控制脚本

Android

  1. Android手机开启USB调试,通过USB线连接到电脑。
  2. 使用ADB列出连接的Android设备:adb devices,并记录设备ID如 WTKDU1670700000
  3. 启动MonkeyRunner:monkeyrunner monkeyrunner.py WTKDU1670700000 http://127.0.0.1:5000。注意将 WTKDU1670700000 替换为上一步记录的设备ID,如果启动服务端时修改了监听端口,则第二个参数也需要对应修改。
  4. MonkeyRunner提示 Press enter to start 后,在微信中打开跳一跳并开始游戏,然后在MonkeyRunner中按下回车键。

iOS

  1. 在手机上启动 WebDriverAgentRunner,并记录设备URL如 http://10.0.0.100:8100
  2. 启动脚本:python wda.py http://10.0.0.100:8100 http://127.0.0.1:5000。注意将 http://10.0.0.100:8100 替换为上一步记录的设备ID,如果启动服务端时修改了监听端口,则第二个参数也需要对应修改。
  3. 提示 Press enter to start 后,在微信中打开跳一跳并开始游戏,然后在脚本中按下回车键。

幺蛾子

本项目做了以下幺蛾子操作。

人工增加随机跳跃误差

monkeyrunner.pywda.py 均接受 --jitter JITTER 参数,若设置该值,则会将控制设备按压的时间乘以 [1 - JITTER, 1 + JITTER] 区间内的随机值。可以先尝试 0.01 然后再根据效果调整。

随机跳跃间隔

两次跳跃间的等待时间为固定 1.75s0.5s 内的随机值。

已知问题

  • iOS下WDA截图有损,杂噪点可能会影响位置判断,但应该不致命。有解决方法,见QA。
  • iOS下可能需要微调长按时间的修正系数(wda.py文件中的 CORRECTION_RATIO 变量)。
  • 距离到时间的映射大概也许可能是有问题的(calculate_time函数)。必要的话可以自己微调里面的系数,或者干脆重写这个函数。
  • 圆形和长方形的棋盘位置判断可能会有偏差,但不致命。
  • 音乐盒的音符可能会干扰棋盘位置判断,致命。(已尝试修复,未确认)

TODO

  • 增加iOS脚本?
  • 也许可以把服务端部署在服务器上。 见FAQ。
  • 优化性能。
  • 进一步优化性能。
  • 优化圆形和长方形棋盘的处理。

FAQ

Android手机需要ROOT吗?

不需要ROOT。但需要电脑安装 Android SDK。只需要命令行工具即可,不需要Android Studio

提示 Broken Pipe 该如何处理?

Ctrl + C退出一次MonkeyRunner再重新启动即可。

MonkeyRunner提示 Press enter to start 后卡住

下载2.5.4rc1版本的Jython替换 Android SDK 中 tools/lib/jython-standalone-2.5.3.jar 文件。

会不会被封号?

这个……封号应该是不会的,并没有对微信做任何的hack。不过删榜倒是有可能的。另外刷分太多会没朋友喔。

可以多个设备同时操作吗?

可以,开一个服务端和多个MonkeyRunner即可。

iOS上运行时定位不准或服务端有大量 Ignored shape 提示

是WDA默认截图为有损格式所致。可以自行修改代码 WebDriverAgentLib/Categories/XCUIDevice+FBHelpers.m- (NSData *)fb_screenshotWithError:(NSError*__autoreleasing*)error 函数里 quality 变量值为 0 。但可能会导致不稳定,原因见该变量注释。

服务端支持WSGI么?

支持,使用 server.py 中的 app 变量即可。配置可在 server_config.py 中修改。

比如用 gunicorn 跑:gunicorn -w 4 -k gevent server:appNOT PRODUCTION READY!

wechat_automated_jump_game's People

Contributors

microdog avatar

Stargazers

 avatar  avatar  avatar  avatar Kai Zhong avatar Sawyer Gao avatar  avatar  avatar devl avatar wuxiaolong avatar  avatar yuyue avatar  avatar Ben Zhang avatar River.nate avatar gudezhi avatar Maggie Yang avatar 张雷 avatar  avatar Chengyu Zhang avatar  avatar lyh2668 avatar Frank avatar xucheng avatar  avatar Lin Jie avatar LucasLin avatar Herrington Darkholme avatar  avatar San avatar

Watchers

gudezhi avatar James Cloos avatar  avatar Lin Jie avatar  avatar  avatar

wechat_automated_jump_game's Issues

无法截图:E/Screenshot: Unsupported protocol: 2

JPython 已经替换成 FAQ 里写的 2.5.4rc1

  • 按回车开始之后报截图错误,日志如下
PS D:\Android_sdk\tools> .\monkeyrunner.bat D:\Web\wechat_automated_jump_game\monkeyrunner.py XXXXXXXXXXXXXX http://127.0.0.1:5000
Connecting XXXXXXXXXXXXXX
Connected
Press enter to start
Taking snapshot...
03:27:17 E/Screenshot: Unsupported protocol: 2
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions] Script terminated due to an exception
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]Traceback (most recent call last):
  File "D:\Web\wechat_automated_jump_game\monkeyrunner.py", line 67, in <module>
    main()
  File "D:\Web\wechat_automated_jump_game\monkeyrunner.py", line 29, in main
    img_data = snapshot.convertToBytes('png')
        at com.android.chimpchat.adb.image.ImageUtils.convertImage(ImageUtils.java:47)
        at com.android.chimpchat.adb.image.ImageUtils.convertImage(ImageUtils.java:63)
        at com.android.chimpchat.adb.AdbChimpImage.createBufferedImage(AdbChimpImage.java:41)
        at com.android.chimpchat.core.ChimpImageBase.getBufferedImage(ChimpImageBase.java:65)
        at com.android.chimpchat.core.ChimpImageBase.convertSnapshot(ChimpImageBase.java:125)
        at com.android.chimpchat.core.ChimpImageBase.convertToBytes(ChimpImageBase.java:72)
        at com.android.monkeyrunner.MonkeyImage.convertToBytes(MonkeyImage.java:66)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)

java.lang.NullPointerException: java.lang.NullPointerException

171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.core.Py.JavaError(Py.java:495)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.core.Py.JavaError(Py.java:488)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:188)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:204)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.core.PyObject.__call__(PyObject.java:404)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.core.PyObject.__call__(PyObject.java:408)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.core.PyMethod.__call__(PyMethod.java:124)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.pycode._pyx0.main$1(D:\Web\wechat_automated_jump_game\monkeyrunner.py:26)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.pycode._pyx0.call_function(D:\Web\wechat_automated_jump_game\monkeyrunner.py)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.core.PyTableCode.call(PyTableCode.java:165)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.core.PyBaseCode.call(PyBaseCode.java:120)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.core.PyFunction.__call__(PyFunction.java:307)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.pycode._pyx0.f$0(D:\Web\wechat_automated_jump_game\monkeyrunner.py:67)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.pycode._pyx0.call_function(D:\Web\wechat_automated_jump_game\monkeyrunner.py)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.core.PyTableCode.call(PyTableCode.java:165)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.core.PyCode.call(PyCode.java:18)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.core.Py.runCode(Py.java:1275)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.core.__builtin__.execfile_flags(__builtin__.java:522)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.util.PythonInterpreter.execfile(PythonInterpreter.java:225)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at com.android.monkeyrunner.ScriptRunner.run(ScriptRunner.java:116)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at com.android.monkeyrunner.MonkeyRunnerStarter.run(MonkeyRunnerStarter.java:77)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at com.android.monkeyrunner.MonkeyRunnerStarter.main(MonkeyRunnerStarter.java:189)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]Caused by: java.lang.NullPointerException
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at com.android.chimpchat.adb.image.ImageUtils.convertImage(ImageUtils.java:47)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at com.android.chimpchat.adb.image.ImageUtils.convertImage(ImageUtils.java:63)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at com.android.chimpchat.adb.AdbChimpImage.createBufferedImage(AdbChimpImage.java:41)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at com.android.chimpchat.core.ChimpImageBase.getBufferedImage(ChimpImageBase.java:65)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at com.android.chimpchat.core.ChimpImageBase.convertSnapshot(ChimpImageBase.java:125)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at com.android.chimpchat.core.ChimpImageBase.convertToBytes(ChimpImageBase.java:72)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at com.android.monkeyrunner.MonkeyImage.convertToBytes(MonkeyImage.java:66)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at java.lang.reflect.Method.invoke(Unknown Source)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:186)
171231 15:27:17.799:S [MainThread] [com.android.monkeyrunner.MonkeyRunnerOptions]       ... 19 more

monkeyrunner报错

File "monkeyrunner.py", line 7, in
from com.android.monkeyrunner import MonkeyRunner, MonkeyDevice, MonkeyView
ModuleNotFoundError: No module named 'com'

could not convert string to float

Traceback (most recent call last):
File "wda.py", line 105, in
main()
File "wda.py", line 58, in main
press_time_raw = float(res.content) / 1000.0
ValueError: could not convert string to float: {
"value" : "Unhandled endpoint: / -- http://127.0.0.1:8100/ with parameters {\n wildcards = (\n ""\n );\n}",
"sessionId" : "0096483F-9F7A-4776-BD92-C86AD389DA6C",
"status

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.