calabash / calabash-android-server Goto Github PK

Automated Functional testing for Android based on cucumber

License: Other

JavaScript 0.30% Shell 0.24% Java 99.09% HTML 0.34% Ruby 0.02% Gherkin 0.01%

calabash-android-server's Introduction

master	develop

calabash-android-server

The test-server for Calabash-Android

Automated Functional testing for Android based on cucumber

Building

Requirements:

Java 8.
Ruby >= 2.3.*
Android build-tools and Android Platform (will be installed by gradle).
Android device/emulator and ADB for local testing.

$ git clone https://github.com/calabash/calabash-android-server.git
$ cd calabash-android-server/server
$ ./gradlew clean assembleAndroidTest

The gradle task will stage the TestServer.apk and the AndroidManifest.xml to the root directory of this repository.

Testing

Start Android device/emulator and make sure that device is visible via ADB before executing tests:

$ adb devices

Execute test runner:

$ cd server/integration-tests
$ ./test.sh

Troubleshooting

If you have issues with build:

Make sure that you have correct version of required dev tools/SDKs.
Verify environment variables on your machine.

If you have issues with installing/running test apk:

Make sure that device/emulator is visible via ADB.

Modify the server/integration-tests/04-run-tests.sh as follows to emit adb logcat logs after the tests.

# Disable "exit immediately" mode
# set -e

# ...

# Add this line to clear device logs
adb logcat -c

(
  cd calabash-test-suite
  SKIP_VERSION_CHECK=1 bundle exec calabash-android run \
    unittest.apk \
    --format pretty \
    --format junit --out test_report
)

# Add this line to receive device logs
adb logcat -d

calabash-android-server's People

Contributors

Stargazers

Watchers

calabash-android-server's Issues

Crash in ViewMapper.getRectForView: "Cannot get a dirty matrix!"

Every few test runs, a random test case will fail due to this crash. We have seen this crash for a couple of years and to get around it we usually just choose a different device to test against. We recently started seeing it again, so I figured I would file it as a bug.

Here is the full stack trace:

F/DEBUG   (15453): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
F/DEBUG   (15453): Build fingerprint: 'samsung/starltexx/starlte:10/QP1A.190711.020/G960FXXU7DTAA:user/release-keys'
F/DEBUG   (15453): Revision: '26'
F/DEBUG   (15453): ABI: 'arm64'
F/DEBUG   (15453): Timestamp: 2020-03-17 23:51:25+0100
F/DEBUG   (15453): pid: 14165, tid: 15450, name: Thread-276  >>> com.convoy.driver.uitest <<<
F/DEBUG   (15453): uid: 17808
F/DEBUG   (15453): signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
F/DEBUG   (15453): Abort message: 'Cannot get a dirty matrix!'
F/DEBUG   (15453):     x0  0000000000000000  x1  0000000000003c5a  x2  0000000000000006  x3  00000079f4afa230
F/DEBUG   (15453):     x4  feff71647164636d  x5  feff71647164636d  x6  feff71647164636d  x7  7f7f7f7f7f7f7f7f
F/DEBUG   (15453):     x8  00000000000000f0  x9  0000007afc33e5e0  x10 0000000000000000  x11 0000000000000001
F/DEBUG   (15453):     x12 00000079f4afa3c0  x13 ffffffffffffffff  x14 0000000000000000  x15 ffffffffffffffff
F/DEBUG   (15453):     x16 0000007afc40c8c0  x17 0000007afc3e9870  x18 00000079ad074000  x19 00000000000000ac
F/DEBUG   (15453):     x20 0000000000003755  x21 00000000000000b2  x22 0000000000003c5a  x23 00000000ffffffff
F/DEBUG   (15453):     x24 00000000133f73d8  x25 0000000012d4e778  x26 00000000132af3d8  x27 0000000012d4da10
F/DEBUG   (15453):     x28 0000000013462870  x29 00000079f4afa2e0
F/DEBUG   (15453):     sp  00000079f4afa210  lr  0000007afc39d330  pc  0000007afc39d360
F/DEBUG   (15453): 
F/DEBUG   (15453): backtrace:
F/DEBUG   (15453):       #00 pc 0000000000083360  /apex/com.android.runtime/lib64/bionic/libc.so (abort+176) (BuildId: 220051b49364b1c2da3adf10c30832cc)
F/DEBUG   (15453):       #01 pc 0000000000008a74  /system/lib64/liblog.so (__android_log_assert+324) (BuildId: 388c836dee4b1e7598e1cafc587689b3)
F/DEBUG   (15453):       #02 pc 00000000001468ec  /system/lib64/libandroid_runtime.so (android::android_view_RenderNode_hasIdentityMatrix(long)+116) (BuildId: 97c11c0a9e40704eea4a584db87b34e1)
F/DEBUG   (15453):       #03 pc 00000000002bfa50  /system/framework/arm64/boot-framework.oat (art_jni_trampoline+64) (BuildId: 6704c3411337007ce3127c4cd1d512d3f2375e11)
F/DEBUG   (15453):       #04 pc 0000000000486e40  /system/framework/arm64/boot-framework.oat (android.graphics.RenderNode.hasIdentityMatrix+48) (BuildId: 6704c3411337007ce3127c4cd1d512d3f2375e11)
F/DEBUG   (15453):       #05 pc 0000000000a58538  /system/framework/arm64/boot-framework.oat (android.view.View.transformFromViewToWindowSpace+440) (BuildId: 6704c3411337007ce3127c4cd1d512d3f2375e11)
F/DEBUG   (15453):       #06 pc 0000000000a413ec  /system/framework/arm64/boot-framework.oat (android.view.View.getLocationInWindow+76) (BuildId: 6704c3411337007ce3127c4cd1d512d3f2375e11)
F/DEBUG   (15453):       #07 pc 0000000000a414a0  /system/framework/arm64/boot-framework.oat (android.view.View.getLocationOnScreen+48) (BuildId: 6704c3411337007ce3127c4cd1d512d3f2375e11)
F/DEBUG   (15453):       #08 pc 000000000209485c  /memfd:/jit-cache (deleted) (sh.calaba.instrumentationbackend.query.ViewMapper.getRectForView+220)
F/DEBUG   (15453):       #09 pc 00000000020a9dd8  /memfd:/jit-cache (deleted) (sh.calaba.instrumentationbackend.query.ast.UIQueryUtils.isViewSufficientlyShown+168)
F/DEBUG   (15453):       #10 pc 00000000020a9e10  /memfd:/jit-cache (deleted) (sh.calaba.instrumentationbackend.query.ast.UIQueryUtils.isViewSufficientlyShown+224)
F/DEBUG   (15453):       #11 pc 00000000020a9e10  /memfd:/jit-cache (deleted) (sh.calaba.instrumentationbackend.query.ast.UIQueryUtils.isViewSufficientlyShown+224)
F/DEBUG   (15453):       #12 pc 00000000020a9f48  /memfd:/jit-cache (deleted) (sh.calaba.instrumentationbackend.query.ast.UIQueryVisibilityMatcher.matchForUIObject+136)
F/DEBUG   (15453):       #13 pc 00000000020bf8c4  /memfd:/jit-cache (deleted) (sh.calaba.instrumentationbackend.query.ast.UIQueryMatcher.call+884)
F/DEBUG   (15453):       #14 pc 00000000020ac1a0  /memfd:/jit-cache (deleted) (sh.calaba.instrumentationbackend.query.ast.UIQueryUtils.isVisible+112)
F/DEBUG   (15453):       #15 pc 00000000020aa620  /memfd:/jit-cache (deleted) (sh.calaba.instrumentationbackend.query.ViewMapper.extractDataFromView+1456)
F/DEBUG   (15453):       #16 pc 00000000020eb810  /memfd:/jit-cache (deleted) (sh.calaba.instrumentationbackend.query.ViewMapper.mapResult+864)
F/DEBUG   (15453):       #17 pc 00000000020b6da4  /memfd:/jit-cache (deleted) (sh.calaba.instrumentationbackend.query.ViewMapper.mapList+260)
F/DEBUG   (15453):       #18 pc 00000000001375b8  /apex/com.android.runtime/lib64/libart.so (art_quick_invoke_static_stub+568) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #19 pc 00000000001460cc  /apex/com.android.runtime/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+276) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #20 pc 00000000002e3be8  /apex/com.android.runtime/lib64/libart.so (art::interpreter::ArtInterpreterToCompiledCodeBridge(art::Thread*, art::ArtMethod*, art::ShadowFrame*, unsigned short, art::JValue*)+384) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #21 pc 00000000002dec74  /apex/com.android.runtime/lib64/libart.so (bool art::interpreter::DoCall<false, false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, art::JValue*)+900) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #22 pc 00000000005a5340  /apex/com.android.runtime/lib64/libart.so (MterpInvokeStatic+368) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #23 pc 0000000000131994  /apex/com.android.runtime/lib64/libart.so (mterp_op_invoke_static+20) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #24 pc 000000000008f8d0  [anon:dalvik-classes.dex extracted in memory from /data/app/com.convoy.driver.uitest.test-iKBlgGp0gFOmMLqQhwr0Dw==/base.apk] (sh.calaba.instrumentationbackend.query.QueryResult.asMappedList+4)
F/DEBUG   (15453):       #25 pc 00000000005a2ab0  /apex/com.android.runtime/lib64/libart.so (MterpInvokeVirtual+1432) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #26 pc 0000000000131814  /apex/com.android.runtime/lib64/libart.so (mterp_op_invoke_virtual+20) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #27 pc 000000000007f85a  [anon:dalvik-classes.dex extracted in memory from /data/app/com.convoy.driver.uitest.test-iKBlgGp0gFOmMLqQhwr0Dw==/base.apk] (sh.calaba.instrumentationbackend.FranklyResult.asMap+46)
F/DEBUG   (15453):       #28 pc 00000000005a2ab0  /apex/com.android.runtime/lib64/libart.so (MterpInvokeVirtual+1432) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #29 pc 0000000000131814  /apex/com.android.runtime/lib64/libart.so (mterp_op_invoke_virtual+20) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #30 pc 000000000007f808  [anon:dalvik-classes.dex extracted in memory from /data/app/com.convoy.driver.uitest.test-iKBlgGp0gFOmMLqQhwr0Dw==/base.apk] (sh.calaba.instrumentationbackend.FranklyResult.asJson)
F/DEBUG   (15453):       #31 pc 00000000002b4518  /apex/com.android.runtime/lib64/libart.so (_ZN3art11interpreterL7ExecuteEPNS_6ThreadERKNS_20CodeItemDataAccessorERNS_11ShadowFrameENS_6JValueEbb.llvm.14549768435847923494+240) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #32 pc 0000000000593c38  /apex/com.android.runtime/lib64/libart.so (artQuickToInterpreterBridge+1024) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #33 pc 0000000000140468  /apex/com.android.runtime/lib64/libart.so (art_quick_to_interpreter_bridge+88) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #34 pc 0000000002097d44  /memfd:/jit-cache (deleted) (sh.calaba.instrumentationbackend.actions.HttpServer.serve+11860)
F/DEBUG   (15453):       #35 pc 000000000203c5e8  /memfd:/jit-cache (deleted) (sh.calaba.instrumentationbackend.actions.NanoHTTPD$HTTPSession.run+3208)
F/DEBUG   (15453):       #36 pc 00000000001a5568  /system/framework/arm64/boot.oat (java.lang.Thread.run+72) (BuildId: b92495b97bd2b9b94ff49baeb11f3347ca279c52)
F/DEBUG   (15453):       #37 pc 0000000000137334  /apex/com.android.runtime/lib64/libart.so (art_quick_invoke_stub+548) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #38 pc 00000000001460ac  /apex/com.android.runtime/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+244) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #39 pc 00000000004b2aa0  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::InvokeWithArgArray(art::ScopedObjectAccessAlreadyRunnable const&, art::ArtMethod*, art::(anonymous namespace)::ArgArray*, art::JValue*, char const*)+104) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #40 pc 00000000004b3b50  /apex/com.android.runtime/lib64/libart.so (art::InvokeVirtualOrInterfaceWithJValues(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jmethodID*, jvalue const*)+416) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #41 pc 00000000004f4600  /apex/com.android.runtime/lib64/libart.so (art::Thread::CreateCallback(void*)+1176) (BuildId: 646dedb2d1e7f719c8ef5314f8894d5e)
F/DEBUG   (15453):       #42 pc 00000000000e3b14  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+36) (BuildId: 220051b49364b1c2da3adf10c30832cc)
F/DEBUG   (15453):       #43 pc 0000000000085330  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64) (BuildId: 220051b49364b1c2da3adf10c30832cc)

Broken Touch and query visibility

Background
We have around 1400 tests running stable on Calabash 0.9.2 with Oreo emulators. I tried to update to 0.9.6 and about 30 different tests always fail due to below reasons:

The query sometimes returns blank results
Touch was successful but a completely wrong point on screen was touched than desired

I observed that:

query("* id:'btn'") yields empty results but query("all * id:'btn'") returns desired result which has visible => false even though element is visible. This can be easily reproduced by repeating query in a loop
On enabling pointer location in developer options (adb shell settings put system pointer_location 1), I saw wrong point was touched once in 10 attempts

On further debugging on server side, I found the casue was change in getViewLocationOnScreen(View view) in UIQueryUtils class. This is used for visibility as well as for touch gestures

If I change this method back to what it was as below, everythig works fine:

	public static int[] getViewLocationOnScreen(View view) {
		ViewWrapper viewWrapper = new ViewWrapper(view);

		return viewWrapper.getLocationOnScreen();
	}

This is the change from 3a91292

        public static int[] getViewLocationOnScreen(View view) {
-               ViewWrapper viewWrapper = new ViewWrapper(view);
+               if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.ICE_CREAM_SANDWICH_MR1
+                               && Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP_MR1) {
+                       ViewWrapper viewWrapper = new ViewWrapper(view);

-               return viewWrapper.getLocationOnScreen();
+                       return viewWrapper.getLocationOnScreen();
+               }
+
+               int[] location = new int[2];
+               view.getLocationOnScreen(location);
+
+               return location;
        }

how to support custom TextView

com.dianping.picasso.view.PicassoTextView
extend com.dianping.richtext.BaseRichTextView;
extend android.support.v7.widget.AppCompatTextView
extend android.widget.TextView

query("* marked:'a name'"), can not find the element

any way to support the custom TextView ? modify the WILDCARD branch ?

public static UIQueryAST uiQueryFromAst(CommonTree step) {
String stepType = UIQueryParser.tokenNames[step.getType()];
switch (step.getType()) {
case UIQueryParser.QUALIFIED_NAME:
return UIQueryASTClassName.fromQualifiedClassName(step.getText());
case UIQueryParser.NAME:
return UIQueryASTClassName.fromSimpleClassName(step.getText());
case UIQueryParser.WILDCARD:
return UIQueryASTClassName.fromQualifiedClassName("android.view.View");

Get the server to compile in Android Studio

How can I modify the version code of TestServer.apk when I use bundle exec rake build?

It reminds me that the version number of client and server is inconsistent when I use the latest compiled TestServer.apk to run calabash-android.

Can I specify the path to the step_definitions file

I don't want to put the step_definitions file in the step_definitions folder, I created a lib folder to put step_definitions files, can I specify a step_definitions file path for the feature file?

Get build.sh to work in travis

Move the server

start_test_server_in_background report failed to connect to socket 'tcp:7102'

I follow https://github.com/calabash/calabash-android-server to execute build for test project and tests
$ ./run-tests.sh
it report:

Scenario: Running the unit test # features/unittest.feature:2
Then I install the app # features/steps/shared.rb:1
Then I start the app # features/steps/shared.rb:5
App did not start see adb logcat for details (RuntimeError)
/calabash-android-server/server/integration-tests/calabash-android/ruby-gem/lib/calabash-android/operations.rb:743:in `block in start_test_server_in_background' /calabash-android-server/server/integration-tests/calabash-android/ruby-gem/lib/calabash-android/retry.rb:10:in` block in retry'
/calabash-android-server/server/integration-tests/calabash-android/ruby-gem/lib/calabash-android/retry.rb:8:in `times' /calabash-android-server/server/integration-tests/calabash-android/ruby-gem/lib/calabash-android/retry.rb:8:in` retry'
/calabash-android-server/server/integration-tests/calabash-android/ruby-gem/lib/calabash-android/operations.rb:742:in `start_test_server_in_background' /calabash-android-server/server/integration-tests/calabash-android/ruby-gem/lib/calabash-android/operations.rb:152:in` start_test_server_in_background'
./features/steps/shared.rb:7:in `/^I start the app$/' features/unittest.feature:4:in` Then I start the app'

I use "adb logcat -d", it print:

=============
8-27 17:57:56.629 2075 2075 D ImsResolver: maybeAddedImsService, packageName: sh.calaba.unittester.test
08-27 17:57:56.637 2075 2075 D CarrierSvcBindHelper: No carrier app for: 0
08-27 17:57:56.645 2350 2350 E adbd : failed to connect to socket 'tcp:7102': Connection refused
08-27 17:57:56.647 2350 2350 E adbd : failed to connect to socket 'tcp:7102': Connection refused

what's wrong?

Why I have no calabash-android-server after I install calabash-android

I have install calabash-android, and it will install instrumentation-backend when I run calabash-android. But I found I have no calabash-android-server in my Mac. Do I need to install and configure it? And how to install it?

[Request for Help] How to define multiple devices?

How to define a various type of devices with different Hardware configurations? Such as the work that Xamarin did; But on my server:

🙏

enter text on input field of email does not work on Chrome WebView

It enters the input above the input field. This is due to input of email type does not support selectionStart on ChromeWebView.

Migrate project structure to newer Android Studio

The project currently is setup with the old eclipse-style project structure. Since the official supported tooling is Android Studio, we should migrate the project to that.

http://developer.android.com/sdk/installing/migrate.html

I am guessing our CI setup would probably have to change as well.

no query results for custom class names due to cache

In my app, I have a facebook login dialog with class "com.facebook.internal.WebDialog$3"
In one case I poll for this dialog to appear and it never succeeds even if it appears on UI.
Reason:

Initially "com.facebook.internal.WebDialog$3" is NOT present and I do query("com.facebook.internal.WebDialog$3")
This results in [] result and at the same time:
executeQuery() method optimize the query. However, at this point, qualifiedClassName of optimised query is detected as null and it gets cached
Now, when dialog appears on screen and I do a query("com.facebook.internal.WebDialog$3") again, it should return results BUT, it returns [].
This is because: optimizedQuery is not null and at the same time it has qualifiedClassName = null in cache from previous step.

to solve this, I have modified the executeQuery() method as below

public QueryResult executeQuery() {
    List<UIQueryAST> queryPath = parseQuery(this.queryString);
    List<UIQueryAST> optimizedQuery = QueryOptimizationCache.getCacheFor(this.queryString);

    boolean needOptimization = false;
    if (optimizedQuery != null) {
        for (int i = 0; i < optimizedQuery.size(); i++) {
            UIQueryAST step = optimizedQuery.get(i);
            if (step instanceof UIQueryASTClassName) {
                if (((UIQueryASTClassName) step).qualifiedClassName == null) {
                    needOptimization = true;
                }
            }
        }
    }

    if (needOptimization || optimizedQuery == null) {
        QueryOptimizer queryOptimizer = new GeneralUIQueryOptimizer();
        optimizedQuery = queryOptimizer.optimize(queryPath);
        QueryOptimizationCache.cache(this.queryString, optimizedQuery);
    }

            return UIQueryEvaluator.evaluateQueryWithOptions(optimizedQuery, rootViews(),
                    parseOperations(this.operations));
}

http://calaba.sh leads to porn

http://calaba.sh link is leading to a porn site. can we please remove it from 1. About section and 2. Readme
cc:// @jmoody

[Android N] Text input fails because private API was removed

How to reproduce:

Run the following built-in step into an Android N (API 24) emulator or device:

I enter text "any text" into field with id "edit_text"
# enter_text("android.widget.EditText id:'edit_text'", "any text")

What's expected:
"any text" is typed into the EditText with id edit_text

What happens instead:
Calabash CLI prints:

Action 'set_selection' unsuccessful: java.lang.NoSuchFieldException: No field mServedInputConnection in class Landroid/view/inputmethod/InputMethodManager; (declaration of 'android.view.inputmethod.InputMethodManager' appears in /system/framework/framework.jar:classes2.dex) (RuntimeError)

Calabash Server prints:

I/System.out: URI: //map
I/System.out: params: {json={"query":"android.widget.EditText id:'edit_text'","operation":{"method_name":"query","arguments":[]}}
I/System.out: }
I/System.out: URI: //gesture
I/System.out: params: {json={"query_timeout":5.0,"gestures":[{"query_string":"android.widget.EditText id:'edit_text'","touches":[{"x":50,"y":50,"offset_x":0,"offset_y":0,"wait":0.0,"time":0.2,"release":true,"query_string":null}]}]}
I/System.out: }
I/System.out: URI: //
I/System.out: params: {json={"command":"set_selection","arguments":[-1,-1]}
I/System.out: }
I/System.out: header: {content-type=application/json;charset=utf-8, date=Fri, 08 Jul 2016 15:34:00 GMT, accept=*/*, host=127.0.0.1:34778, content-length=47, user-agent=HTTPClient/1.0 (2.8.0, ruby 2.0.0 (2015-12-16))}
I/System.out: params: {json={"command":"set_selection","arguments":[-1,-1]}
I/System.out: }
I/System.out: ProP json = {"command":"set_selection","arguments":[-1,-1]}
I/System.out: files: {}
I/System.out: command: {"command":"set_selection","arguments":[-1,-1]}
I/InstrumentationBackend: Got command:'Command:'Line:null,set_selection', arguments:'[-1, -1]'
I/System.out: result:{"bonusInformation":[],"message":"java.lang.NoSuchFieldException: No field mServedInputConnection in class Landroid/view/inputmethod/InputMethodManager; (declaration of 'android.view.inputmethod.InputMethodManager' appears in /system/framework/framework.jar:classes2.dex)","success":false}
I/System.out: URI: //kill
I/System.out: params: {json={}
I/System.out: }
I/System.out: Stopping test server