Giter VIP home page Giter VIP logo

deviceagent.ios's Introduction

master develop versioning license contributing
Build Status Build Status GitHub version License Contributing

Calabash

Automated Acceptance Testing for Mobile Apps.

Rake

TODO Release instructions for Android and iOS.

$ rake -T
rake android:build     # Build the Android test server
rake build             # Build calabash-1.9.9.pre2.gem into the pkg directory
rake ctags             # Generate ctags in ./git/tags
rake install           # Build and install calabash-1.9.9.pre2.gem into system gems
rake release           # Create tag v1.9.9.pre2 and build and push calabash-1.9.9.pre2.gem to Rubygems
rake spec              # Run RSpec code examples
rake unit              # Run RSpec code examples
rake yard              # Generate YARD Documentation
rake yard:publish      # Generate and publish docs

Testing

rspec

$ be rake unit # All unit tests.
$ be guard     # Run unit tests as you develop.

Integration tests

$ rake integration:page-object-model  # Run POM tests
$ rake integration:ruby               # Run tests ensuing correct Ruby interfacing
$ rake integration:cli                # Run command line interface tests

deviceagent.ios's People

Contributors

alexeybudynkov avatar anatolypristensky avatar andreymaslennikov avatar denis-panov avatar dmitriykirakosyan avatar dmitry-shibanov avatar dsame avatar fr0l avatar ilyabausovakvelon avatar ivan-nosar avatar jescriba avatar jmoody avatar joesss avatar leffsu avatar maksimzhukov avatar owenniblock avatar physx23 avatar polyakrecords avatar sergey-plevako-badoo avatar stoneman avatar tikhon-polyakov avatar vsafonkin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

deviceagent.ios's Issues

CI should run static analyzer

The static analyzer is picking up a couple of errors. We should be running in CI and failing on non-zero exit.

@sapieneptus One of the problems is in CBXDevice#dictionaryRepresentation; I seem to recall that you had a macro or there was a builtin macro for ensuring object or NSNull? Am I wrong?

XCUITest Spikes

What

Spikes for integrating CalabashXCUITestServer with the rest of calabash ecosystem.

Spike Acceptance Criteria

  • Get a calabash cucumber test running (with the following limited API)
  • Have some sort of API for the following actions:
  • tap
  • swipe
  • type_text

Spike 1: Self Contained XCUITest API

Description

This API should query CalabashXCUITestServer directly and perform gestures directly.

Projects Touched:

  • Need to tweak existing Calabus engine to handle requests with index
  • Need to create an API in calabash-ios

Extra TODOs

  • query(ID or Marked, { index: optional } )

CC: @krukow @jmoody I would love feedback as to whether this is a sane way to go about this.
@jraczak

unknown gesture should raise error

I expected return an HTTP error.

May 17 19:46:38 aft XCTRunner[33527]: POST /1.0/gesture {
        gesture = tap;
        options =     {
        };
        specifiers =     {
            coordinate =         {
                x = 50;
                y = 50;
            };
        };
    }

server should return valid JSON for invalid routes

RunLoop::XCUITest::HTTPError: Could not parse response 'Unhandled endpoint: POST /1.0/tap -- http://127.0.0.1:27753/
Params: {
    wildcards =     (
        "1.0/tap"
    );
}
Body: {
    id = General;
}'; the app has probably crashed

I could check the response first but I would rather reason with and report a JSON object in the error message.

xcodebuild is blocking indexing in Xcode.app

The xcodebuild process running the DeviceAgent is blocking the Xcode.app from performing indexing which in turn blocks commands like indexing.

This is not a good experience for the user. I am not sure yet if this affects projects other than the DeviceAgent project. If it does, this effectively means the end of Calabash as a BDD tool.

There are two cases to understand and deal with:

  1. Orphaned xcodebuild processes. I took a shot at managing the xcodebuild process in RunLoop earlier and the results were not good. Sending TERM to xcodebuild threw a bunch of UI crash reports. Users may or may not see these - my machine is configured to generate a UI crash report for every crash. This is not the default.
  2. User is developing an app using Calabash BDD. A Calabash test has run (the AUT is still open) or tests are running. The user goes back to Xcode to make some change and finds Xcode is not in a good state for development (stuck indexing). The user will have to quit the Calabash test.

ATTN @krukow This needs to be considered when deciding how to distribute/launch the runner.

Can /session route detect if the app has actually launched?

On iOS Simulators, we can detect that the app has launched using ps.

Can the server itself tell us that the app has been launched?

This Scenario is failing

Scenario: app with bundle id is not installed
Given com.example.MyApp is not installed on target device
And I call /session with com.example.MyApp
Then an error is raised

What actually happens is that /session replies that it has launched the app.

/shutdown is not stopping the server

I have confirmed that the server responds to /health for at least a second after the /shutdown route is called.

This causes problems when launching on physical devices because the server is available even while the xcodebuild is in progress.

Integration testing?

@sapieneptus

I saw in a recent PR that you tested rotate against Google Maps and a drawing application.

I would like to stand up some kind of integration tests in this repo.

I have some recommendations, but am interested to know if you have any suggestions.

I want to make a decision on this quickly because I want to have cucumbers for the 2-finger tap gesture.

UnitTestApp

This is what I do in the LPServer. I use the LPTestTarget as the unit test target and cucumber test target. These cucumber tests are transient - they are eventually deleted when the feature stabilizes. Most of the time I move the feature to Permissions, CalSmokeApp, or CalWebApp. Sometimes I just delete the feature (if it is not interesting).

There are some drawbacks to this approach.

  1. I end up duplicating the feature and the implementation (e.g. a view that responds to double tap) later in another app.
  2. The UnitTest app has two roles - unit testing and integration testing. I have never had a problem with this on the LPServer, but the unit test coverage is light in that repo.

The benefit to this approach is that there is no other target to maintain. It also supports a BDD workflow. I want to implement 2 finger tap? I write the cucumber, I add the server side implementation, and then I write the application code to add a view that demonstrates 2 finger tapping.

(Yet) Another App Target

This is only interesting if there is some problem using the UnitTest app as the integration test target.

The drawback here is that the we have to manage (yet) another target in the DeviceAgent target. This is not terrible, just something to consider.

CalSmoke-cal.app

We do like UITest does - we add a CalSmoke-cal.app directly to this repo. The benefit is that the CalSmoke-cal.app has views that cover the majority of gestures.

The drawback here is that maintaining the CalSmoke-cal.app. The static binary will need to be updated everytime there is a release or a change in the app. There is already a lot of work around what happens after a Calabash iOS release. And the CalSmokeApp will require more updates.

I would prefer not to use the CalSmokeApp in this way. This app has a clear role these days: demonstrate good Cucumber and that the Calabash API works. It should always be passing and always in the best possible state.

CalSmoke git submodule

I don't want to do this. I mention it as an option so you can argue for it if you want.

Add CORS response headers

These were added to the LPServer, but I don't recall why.

@krukow Should these be part of the DeviceAgent response headers? If so, assign to me.

 @"Access-Control-Allow-Methods" : @"GET, POST, PUT, DELETE, OPTIONS",
 @"Access-Control-Allow-Origin" : @"*",
 @"Access-Control-Allow-Credentials" : @"true",
 @"Access-Control-Max-Age" : @"3000",

remove `enter_text_in` gesture

The server API should not have compound gestures.

This enter-text-in requires a touch, a check for the keyboard, and then a keyboard interaction.

There are at least three possible error conditions here:

  1. can't find the thing to touch
  2. can't touch
  3. keyboard doesn't appear
  4. text does not get entered.

The client will have to handle these cases to produce a meaningful error for the user. This is on top of the implementation for enter_text.

I think it is easier for enter_text_in to be implemented on the client side.

Device functions

  • rotate_device
  • screenshot
  • send_app_to_background

Optional

  • shake

What about

  • set_location ?

tap, double tap, and multi-finger tap should return :error response if duration > 0.5

  # DoubleTap.m
   //TODO: should this throw an exception?
    if ([self duration] >= 0.5) {
        NSLog(@"WARNING: Having a tap duration greater than 0.5 will trigger a LongPress, not a tap.");
    }

It makes sense to confine :duration to a range. Would it make more sense to drop the :duration key? The tap gestures have a fixed duration. Everything else is a long press?

Regardless of the outcome of this discussion, an invalid duration should return an :error response to the client.

/shutdown or DELETE /session kill the current AUT

It gets a little murky here when the server could launch more than one application.

Should the client be keeping track of the sessions that were created and be responsible for deleting each one?

The problem with the client keeping track is that when attaching to an existing CBXRunner (from the console), the client cannot know what apps have been launched with the CBXRunner. Does that make sense?

Can we extract an interface from the two implementations of gestures?

Implementing the same logic twice is a pain in the ass, especially when you don't know what implementation will be used.

- (XCSynthesizedEventRecord *)eventWithCoordinates:(NSArray<Coordinate *> *)coordinates {
- (XCTouchGesture *)gestureWithCoordinates:(NSArray<Coordinate *> *)coordinates {

long long ==> NSInteger type conversion problem in CBXTouchEvent.m

/Users/moody/git/calabash/xcuitest/Server/AutomationActions/Gestures/EventSynthesis/CBXTouchEvent.m:19:22: implicit conversion loses integer precision: 'long long' to 'NSInteger' (aka 'int') [-Wshorten-64-to-32]

    te.orientation = path.orientation;
                   ~ ^~~~~~~~~~~~~~~~
  • Turn on warnings fail the build for the cli unit test runner
  • Turn on warnings fail the build for the CBXRunner build

How will XCUITServer and CBServer know each other's ports?

Given the architecture we have decided, calabash-ios-server (henceforth CBServer) will need to POST some stuff to calabash-xcuitest-server (henceforth XCUITServer). Therefore, CBServer needs to know XCUITServer's port.

Approaches

  1. XCUITServer has a static port. This is hardcoded into CBServer.
  2. XCUITServer checks a range of ports and uses the first available. It must then POST its port to CBServer to be stored for the remainder of the session.
    As a result, since CBServer's port is technically dynamic as well, the calabash-ios-client will need to read the CBServer's port from its Info.plist and write that into XCUITServer's info plist prior to installation on device. This of course requires resigning.
  3. User is required to add CBServer's port to XCUITServer's info.plist (somehow). Then XCUITServer can POST its port to CBServer for registration, however it may be determined.

For now, I'm using approach 1 since it's the easiest and quickest.

CC: @jmoody @krukow

Prefer NS and CGFloat types over primitives

I am concerned about the use of int, float, and double primitive types.

We had many headaches upgrading the LPServer between Xcode 5 and 6 because we used primitives instead of the NS and CGFloat types. This was the switch between NSInteger being an int and a long (et. al.).

I propose we convert all int to NSInteger and unsigned int to NSUInteger. We should also use NSUInteger whenever we expect an unsigned integer.

We had a number of problems in the LPServer with long double type encoding and double vs. float values. If we use CGFloat everywhere we can avoid these problems.

App crashed and launched FaceTime sign in?

I was testing cucumbers on a physical device this morning and had a FaceTime signing dialog pop up, then the UnitTestApp crashed.

2016-04-19 08:54:14.454 XCTRunner[1377:766729] POST /1.0/query {
    id = "gesture performed";
}
    t =     5.24s     Find the Application "sh.calab.UnitTestApp" 0x136e56150
    t =     6.25s         Find the Application "sh.calab.UnitTestApp" 0x136e56150 (retry 1)
    t =     7.26s         Find the Application "sh.calab.UnitTestApp" 0x136e56150 (retry 2)
Provided dSYM: [/var/containers/Bundle/Application/AE6506D1-22FF-432C-B3B0-98555202252E/CBX-Runner.app/PlugIns/CBX.xctest.dSYM/Contents/Resources/DWARF/CBX] does not match symbol owner 0x136f82da0
    t =     7.53s         Assertion Failure: UI Testing Failure - Application is not running, unable to get Accessibility data. Did you call -launch?
<unknown>:0: error: -[XCUITestDriver testRunner] : UI Testing Failure - Application is not running, unable to get Accessibility data. Did you call -launch?
    t =     7.54s     Get all elements bound by index for: Elements matching predicate '"gesture performed" IN identifiers'
2016-04-19 08:54:16.768 XCTRunner[1377:766729] (
    0   CoreFoundation                      0x0000000183e6ae50 <redacted> + 148
    1   libobjc.A.dylib                     0x00000001834cff80 objc_exception_throw + 56
    2   CoreFoundation                      0x0000000183e6ad80 <redacted> + 0
    3   XCTest                              0x00000001000680cc +[_XCTestCaseInterruptionException interruptTest] + 44
    4   XCTest                              0x000000010007c760 -[XCAXClient_iOS screenshotData] + 412
    5   XCTest                              0x0000000100099c28 __43-[XCUIElementQuery allElementsBoundByIndex]_block_invoke + 64
    6   XCTest                              0x000000010006c8f0 -[XCTestCase startActivityWithTitle:block:] + 364
    7   XCTest                              0x0000000100099bd0 -[XCUIElementQuery allElementsBoundByIndex] + 176
    8   CBX                                 0x00000001023bc624 -[Query execute] + 612
    9   CBX                                 0x000000010237b7d0 __24+[QueryRoutes getRoutes]_block_invoke_2 + 332
    10  CBX                                 0x00000001023b1488 __39+[CBXRoute handleRequestAndExceptions:]_block_invoke + 140
    11  CBX                                 0x00000001023866e4 -[RoutingHTTPServer handleRoute:withRequest:response:] + 376
    12  CBX                                 0x0000000102387584 __72-[RoutingHTTPServer routeMethod:withPath:parameters:request:connection:]_block_invoke + 5

The client replied with:

   Expected status code < 200, found 500.

      Server replied with:

      {"error"=>"Interrupting test"}
       (RunLoop::XCUITest::HTTPError)

Cleanup

  • Remove 'CB' prefix wherever posible
  • Remove copyright comments from tops of all files
  • Ensure that fb driver files reference the correct license

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.