Giter VIP home page Giter VIP logo

aws-device-farm-gradle-plugin's Introduction

AWS Device Farm Gradle Plugin

AWS Device Farm integration with the Android Gradle Build system

This plugin provides AWS Device Farm functionality from your Android gradle environment, allowing you to kick off tests on real Android phones and tablets hosted in the AWS Cloud.

For more information see the AWS Device Farm Developer Guide.

Usage

  1. Add the Device Farm plugin artifact. Paste the following into your top-level build.gradle.
buildscript {

    repositories {        
        mavenCentral()            
    }
    
    dependencies {        
        classpath 'com.amazonaws:aws-devicefarm-gradle-plugin:1.3'
    }        
}
  1. Configure the Device Farm plugin in your module’s build.gradle file. This is usually app/build.gradle.

Minimal configuration

This will do the following:

  • Start a Run under the Project "My Project"
  • Use the "Top Devices" curated Device Pool
  • Run your instrumentation tests under androidTest
apply plugin: 'devicefarm'

devicefarm {

    // Required. The Project must already exist. You can create a project in the AWS console.
    projectName "My Project" 
    
    // Required. You must specify either accessKey and secretKey OR roleArn. roleArn takes precedence. 
    authentication {
        accessKey "aws-iam-user-accesskey"
        secretKey "aws-iam-user-secretkey"
        
        // OR
        
        roleArn "My role arn"
    }
}

Advanced configuration

apply plugin: 'devicefarm'

devicefarm {

    // Required. The Project must already exist. You can create a project in the AWS console.
    projectName "My Project" // required: Must already exist.
    
    // Optional. Defaults to "Top Devices"
    devicePool "My Device Pool Name"
    
    // Optional. Default is 150 minutes
    executionTimeoutMinutes 150
    
    // Optional. Set to "off" if you want to disable device video recording during a run. Default is "on"
    videoRecording "on"
    
    // Optional. Set to "off" if you want to disable device performance monitoring during a run. Default is "on"
    performanceMonitoring "on"
    
    // Optional. Add this if you have a subscription and want to use your unmetered slots
    useUnmeteredDevices()
    
    // Required. You must specify either accessKey and secretKey OR roleArn. roleArn takes precedence. 
    authentication {
        accessKey "aws-iam-user-accesskey"
        secretKey "aws-iam-user-secretkey"
        
        // OR
        
        roleArn "My role arn"
    }

    // Optional block. Radios default to 'on' state, all parameters are optional
    devicestate {
        extraDataZipFile file("path/to/zip") // or ‘null’ if you have no extra data. Default is null.
        auxiliaryApps files(file("path/to/app"), file("path/to/app2")) // or ‘files()’ if you have no auxiliary apps. Default is an empty list.
        wifi "on"
        bluetooth "off"
        gps "off"
        nfc "on"
        latitude 47.6204 // default
        longitude -122.3491 // default
    }
 
    // Optional. Set the test type. Default is instrumentation. 
    // You can only set one test type. 
    // See "Test Type configuration" below for configuration details

    // Fuzz
    fuzz {
    }

    // Instrumentation
    instrumentation {
        // Optional. See the AWS Developer docs for filter rules
        filter "my-filter"
    }

    // Calabash        
    calabash {
       tests file("path-to-features.zip")
    }
}
  1. Run your configured test on Device Farm with the devicefarmUpload task. ( ./gradlew devicefarmUpload)
  2. The build output will print out a link to the AWS Device Farm console where you can monitor your test execution.

Generating a proper IAM user:

  1. Log into your AWS web console UI.
  2. Click "Identity & Access Management".
  3. On the left-hand side of the screen, click "Users".
  4. Click "Create New Users".
  5. Enter a user name of your choice.
  6. Leave the "Generate an access key for each user" checkbox checked.
  7. Click "Create".
  8. View or optionally download the User security credentials that were created; you will them them later.
  9. Click "Close" to return to the IAM screen.
  10. Click your user name in the list.
  11. Under the Inline Policies header, click the "click here" link to create a new inline policy.
  12. Select the "Custom Policy" radio button.
  13. Click "Select".
  14. Give your policy a name under "Policy Name".
  15. Copy/paste the following policy into "Policy Document"
{
     "Version": "2012-10-17",
     "Statement": [
         {
             "Sid": "DeviceFarmAll",
             "Effect": "Allow",
             "Action": [ "devicefarm:*" ],
             "Resource": [ "*" ]
         }
     ]
 }
  1. Click "Apply Policy".

Test Type configuration

Appium

Device Farm provides support for Appium Java TestNG , Appium Java JUnit, Appium python, Appium Node.js and Appium Ruby for Android.

You can choose to useTestNG() or useJUnit() or usePython() or useNode()`` or useRuby()`.

JUnit is the default and does not need to be explicitly specified.

appium {
    tests file("path to zip file") // Required
    useTestNG() // or useJUnit() or usePython() or useNode() or useRuby()
    testSpecName "My Test Spec Name" // if you want to use Custom Mode // Optional for TestNG, JUnit and Python
}

Use AWS Device Farm's app explorer to test user flows through your app without writing custom test scripts. A username and password may be specified in the event your app requires account log-in.

appexplorer {
    username "my-username"
    password "my-password"
}

Device Farm provides a built-in fuzz test type.

The built-in fuzz test randomly sends user interface events to devices and then reports results.

fuzz {
    eventThrottle 50 // Optional. Default is 50
    eventCount 6000  // Optional. Default is 6000
    randomizerSeed 1234 // Optional. Default is blank
}

Device Farm provides support for Calabash for Android.

calabash {
    tests file("path to zip file") // Required
    tags "my tags" // Optional. Calabash tags
    profile "my profile" // Optional. Calabash profile
}

Device Farm provides support for Instrumentation (JUnit, Espresso, Robotium, or any Instrumentation-based tests) for Android.

When running an instrumentation test in gradle the apk generated from your androidTest directory will be used as the source of your tests.

instrumentation { 
    filter "test filter per developer docs" // Optional
    testSpecName "My Test Spec Name" // if you want to use Custom Mode // Optional
}

Upload your app as well as your UI Automator based tests packaged in a jar file

uiautomator {
    tests file("path to uiautomator jar file") // Required
    filter "test filter per developer docs" // Optional
}

Building the plugin

Building the plugin is optional. The plugin is published through Maven Central.

  1. Clone the GitHub repository.
  2. Clean, assemble and test the plugin using ./gradlew clean build
  3. Install the plugin into your local maven directory using ./gradlew install.
  4. The plugin will be installed to your local maven repository.

Adding New Frameworks

As Device Farm supports additional test types the plugin must be extended to enable them into the DSL.

  1. Define what the DSL should look like.
  2. Create a class in the com.amazonaws.devicefarm.extension package to support the DSL, it must extend the ConfiguredTest abstract class.
  3. If the test type requires a test artifact package to be uploaded in addition to the app then make sure the new class extends the TestPackageProvider trait.
  4. If the test type supports the 'filter' parameter extend the HasFilter trait.
  5. Modify DeviceFarmExtension to add a method for the new test type, like so:
void mynewtesttype(final Closure closure) {
    NewTestType newTest = new NewTestType()
    project.configure newTest, closure
    test = newTest
}        
  • This will enable the configuration of the new test type within the devicefarm plugin.

aws-device-farm-gradle-plugin's People

Contributors

ahawker avatar andreas-marschke avatar appwiz avatar artemnikitin avatar bradley-curran avatar michael-b-willingham avatar misumirize avatar nikhil-dabhade avatar r7o avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-device-farm-gradle-plugin's Issues

How to specify build variant/apk file?

When running "gradle devicefarmUpload" my tests run using a debug version of the app.
I need to run against a release version of the app.

How can I specify a different version of the app to upload?

Issue using within Java 7 project

This plugin is compiled using Java 8 which is causing issues within my Android project (using Java 7). I receive the following error within my Android project when trying to use this plugin: "Unsupported major.minor version 52.0".

If I change my Android project to use JDK 8, then the plugin works but I would rather not use JDK 8 within my project since it is not yet officially supported.

Is there something I am doing wrong, or is it possible to update this plugin so it is compatible with Java 7?

Flutter support

Can this be used with flutter integration_test?
e.g. 1) I can run tests locally which pass with
./gradlew app:connectedAndroidTest -Ptarget=pwd/../integration_test/app_test.dart
2) I can manually upload to AWS device farm
3) if i use this plugin, how do i specify app_test.dart?

What is the zip file of appium ?

In the gradle plugin there is this part:

appium {
    tests file("path to zip file") // Required
    useTestNG() // or useJUnit()
}

What is the path to zip file ?
I'm using gradle of course.
Thanks!

AWS doesn't support Appium with Serenity BDD & Gradle configuration?

Hi,
I have seen AWS is supporting the below

Test Type Configuration

  • Appium JUnit
  • Appium TestNG
  • Built-in: Explorer
  • Built-in: Fuzz
  • Calabash

but I'm not sure exactly supporting below

  • Appium- Junit With Serenity BDD & Gradle

If it does support under which type I should use the configuration from the list above.
Let me know, please?

Thanks
Vamc

Running the plugin says tests were skipped

The build is successful and I get a link to devicefarm and the test run. However, all the tests are skipped. Please can you advise the options available?

apply plugin: 'devicefarm'

devicefarm {

    projectName "*******" // required: Must already exists.

    devicePool "*********" // optional: Defaults to "Top Devices"

    //useUnmeteredDevices() // optional if you wish to use your un-metered devices


    authentication {
        accessKey "******"
        secretKey "*****"
    }

    // optional block, radios default to 'on' state, all parameters optional
    devicestate {
        wifi "on"
        bluetooth "off"
        gps "off"
        nfc "on"
        latitude 47.6204 // default
        longitude -122.3491 // default
    }
    instrumentation {
        
    }
}

Using roleArn results in error 'Access key cannot be null'

Found in 1.3

Using roleArn instead of accessKey & secretKey, as follows:

authentication {
    roleArn "arn:aws:iam::[redacted]:role/[role_name]"
}

results in Access key cannot be null error when executing devicefarmUpload.

Adding an accessKey then results in Secret key cannot be null.

Adding an accessKey & secretKey means the plugin attempts to use them over the roleArn (I used dummy keys when attempting this).

`testSpecName` in Instrumentation tests

Hello,

I would like to know how can I use testSpecName in Instrumentation tests as declared in README.

With the version 1.3 of the plugin it doesn't seem to recognize it.

Regards,
Maciej Górski

Failed to apply plugin [id 'devicefarm'] The android or android-library has not been applied yet

I'm getting the following error when trying to setup the device farm gradle plugin.

Failed to apply plugin [id 'devicefarm'] The android or android-library has not been applied yet

Here is an example project where this is failing.

https://github.com/twilly86/AWSDeviceFarmTest

Could you see what I'm doing wrong?

The instructions aren't saying anything about needing the android plugin to get this to work.

devicefarmUpload fail with NullPointerException on android library (aar) projects

I've an android library project and am trying to test it using AWS Device Farm, when i execute it I get this error:

:my-library:devicefarmUpload FAILED
:my-library:devicefarmUpload (Thread[Task worker Thread 7,5,main]) completed. Took 0.889 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':my-library:devicefarmUpload'.
> java.lang.NullPointerException (no error message)

* Try:
Run with --debug option to get more log output.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':currency-converter-datalayer:devicefarmUpload'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:84)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:55)
        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:62)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:88)
        at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:51)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker$1.execute(DefaultTaskGraphExecuter.java:236)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker$1.execute(DefaultTaskGraphExecuter.java:228)
        at org.gradle.internal.Transformers$4.transform(Transformers.java:169)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:106)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:61)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:228)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:215)
        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:77)
        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:58)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
        at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:46)
Caused by: java.lang.NullPointerException
        at com.amazonaws.devicefarm.DeviceFarmUploader.upload(DeviceFarmUploader.java:64)
        at com.amazonaws.devicefarm.DeviceFarmServer.uploadApks(DeviceFarmServer.java:87)
        at com.android.build.gradle.internal.tasks.TestServerTask.sendToServer(TestServerTask.java:45)
        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.doExecute(DefaultTaskClassInfoStore.java:141)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:134)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:123)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:632)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:615)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:95)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:76)
        ... 20 more


BUILD FAILED

The configuration is really basic:

buildscript {
    repositories {
        jcenter()
    }
    dependencies {
        classpath 'com.android.tools.build:gradle:2.3.0'
        classpath 'com.amazonaws:aws-devicefarm-gradle-plugin:1.2'
    }
}

and in the library project below the android library setup:

apply plugin: 'devicefarm'

devicefarm {
    projectName "Bootstrap Android"
    devicePool "Last Nexus/Pixel"
    authentication {
        accessKey "[redacted]"
        secretKey "[redacted]"
    }

    // optional block, radios default to 'on' state, all parameters optional
    devicestate {
    }

    instrumentation {
        filter "my.package"
    }
}

The exact same configuration succeed on an application (apk).

connectedAndroidTests are executed just fine on the library when i use my own device or an emulator.

Test run only gets enqueued, plugin doesn't wait for tests to finish

Running version 1.2 of the plugin I was trying to execute some Instrumentation (Espresso) tests.

There are two things that happen that I think are wrong.

  1. The build finishes very quickly with a link to the device farm console, where all devices show up as unavailable. If I run the tests through the Device Farm UI on the web (same tests, same device pool, same apk) it all works fine
    screen shot 2016-01-13 at 3 21 39 pm
  2. The build is reported as successfully passing, which it obviously is not. Should be reported as failure.

AWS Device Farm configuration is NOT VALID (with valid config)

I get this message:

AWS Device Farm configuration is NOT VALID
AWS Device Farm configuration is NOT VALID

(the message appears twice) for the config below (the projectName, accessKey and secretKey are working, I can use them to send tests through the command line):

devicefarm {

  projectName "My Android App"

  authentication {
    accessKey "XXXXXXXXXXXXXXXX"
    secretKey "YYYYYYYYYYYYYYYYYYYY"
  }

}

The devicefarmUpload task is therefore skipped (I'm guessing because the config is considered not valid).

CI Integration (wait for test results in the gradle plugin)

The current behavior of the plugin is uploading the artifact on the AWS device farm and then provide an URL to follow it.

For a CI system this is not what you want: you want the test to WAIT to complete and fail if any test has failed / succeed if all test failed.

I'd like to have a task like: devicefarmUploadAndWait that makes the gradle script stop and wait for the result (possibly sending multiple modules projects in parallel and wait for all of them).

Without this I can't use AWS Device Farm for my CI.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.