defra / quke Goto Github PK

A gem that lets you focus on your Cucumber acceptance tests, and not worry about configuring the drivers

License: Other

Ruby 99.86% Shell 0.14%

quke acceptance-testing selenium cucumber poltergeist browserstack capybara

quke's Introduction

Quke is a gem that helps you to build a suite of Cucumber acceptance tests.

Quke tries to simplify the process of writing and running these tests by setting up Cucumber for you. It handles the config to allow you to run them in Firefox (via Selenium) and Chrome via Selenium. It also has out of the box support for using Browserstack automate. This leaves you to focus on just your tests.

It was born out of trying to make the process for team members without a development background as simple as possible, and to get them involved in writing and building acceptance tests that they then manage.

It also includes the ability to run those tests using Browserstack. Browserstack gives you the ability to test your application with different combinations of platform, OS, and browser all in the cloud.

Pre-requisites

You'll need Ruby installed (ideally the latest version available) plus the Bundler gem.

The only other dependency this project has is the browsers you intend to use it with. It is currently setup to work with

Chrome (via Selenium)
Firefox (via Selenium)

Installation

Add this line to your application's Gemfile

gem 'quke'

And then execute:

$ bundle

Or install it yourself as

$ gem install quke

Configuration

You can configure Quke using a .config.yml file. What can be configured essentially falls into one of 3 groups.

Quke configuration - things like which driver to use, whether to pause between steps, or if Quke should stop in the event of an error
Custom configuration - add your own custom values using the custom: node
Browserstack configuration - specify exactly how you want your Browserstack session to run, for example which browser and OS to test against, the project name, and your credentials.

See .config.example.yml for full details of the options you can specify in your .config.yml.

Multiple configs

When Quke runs it will default to looking for .config.yml. However you can override this and tell Quke which one to use. This allows you to create multiple config files.

You do this using an environment variable. The most flexible way is to set the variable as part of your command.

QUKE_CONFIG='chrome.config.yml' bundle exec quke

The use case is to allow you to have different configs setup ready to go, and enable you to switch between them. For example when testing with Chrome and Firefox you set a 1 second delay between steps so you can observe the tests as they run through, but in your default .config.yml you want no pauses.

Security

If you are testing a site that requires some form of login, or you are using Browserstack you'll need to tell Quke about the credentials. Generally we advise not committing your .config.yml to source control, but we understand you may well build up a suite of them that you want to store with the project and share with others.

If that's the case make use of the following features.

Browserstack environment variables

.config.example.yml is an example of a config file which contains username, auth_key, and if local testing local_key. You really don't want these becoming known so if you have to commit the config file, remove them and instead set the following environment variables wherever you need to run the tests.

BROWSERSTACK_USERNAME
BROWSERSTACK_AUTH_KEY
BROWSERSTACK_LOCAL_KEY

Quke looks for them when configuring the Browserstack driver and only if not found does it then go looking in the config file.

Ruby's ENV classs

You can access any environment variable from within your project using ENV.

secret_key = ENV['SUPER_SECRET_KEY']

You can set them by calling export SUPER_SECRET_KEY="password123" in your terminal prior to running Quke. In this way for example, the passwords featured in the custom section in .config.example.yml could be removed, and instead your step could read the value it needs to submit from an environment variable.

Usage

TODO: Write usage instructions here

Behaviours

You should be aware of some default behaviours included in Quke.

Displaying web pages on fail

Capybara includes the ability to save the source of the current page at any point. Quke has been configured so that if you are not using a headless browser and a step fails it will save the source to file and then use a tool called Launchy to open it in your default browser.

You can disable this behaviour using display_failures: false in your .config.yml

Quit on 5 failures

If you are running using Chrome or Firefox after the 5th failure Quke will automatically stop. This is to prevent scores of tabs being opened in the browser when an error is found and Quke is set to show failures, which may just be the result of an error in the test code.

Automatically setting Browserstack session status

If you are using the Browserstack driver, Quke will automatically update the status of the session recorded in Browserstack for you. If all tests pass, it will be marked as 'passed', else it will be marked as 'failed' (it defaults to 'completed').

Browser drivers

A very simple overview of how things work is that

Cucumber runs the tests
Capbybara is used in the tests to simplify working with Selenium
Selenium is used to tell the browsers what to do
each browser has a driver (Chromedriver, Geckodriver etc) which can interpret Selenium commands into actual actions

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Running

Whilst just developing a gem like Quke you can execute it with

$ ruby -Ilib ./exe/quke

and pass in arguments as well

$ ruby -Ilib ./exe/quke --tags @wip

Contributing to this project

If you have an idea you'd like to contribute please log an issue.

All contributions should be submitted via a pull request.

License

THIS INFORMATION IS LICENSED UNDER THE CONDITIONS OF THE OPEN GOVERNMENT LICENCE found at:

http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3

The following attribution statement MUST be cited in your products and applications when using this information.

Contains public sector information licensed under the Open Government license v3

About the license

The Open Government Licence (OGL) was developed by the Controller of Her Majesty's Stationery Office (HMSO) to enable information providers in the public sector to license the use and re-use of their information under a common open licence.

It is designed to encourage use and re-use of information freely and flexibly, with only a few conditions.

quke's People

Contributors

Stargazers

Watchers

Forkers

timstone jskirst uk-gov-mirror

quke's Issues

Safari driver support

Currently quke doesn't have support to run tests using the Safari browser, this is needed to test MS dynamics on OSX as other browsers aren't supported
https://technet.microsoft.com/en-us/library/hh699710.aspx#BKMK_support_nonIE

Replace chrome and gecko drivers with WebDrivers gem

The maintainer of Chromedriver-helper has announced that support for the gem will cease 31-03-2019

flavorjones/chromedriver-helper#83

This is because he feels the Webdrivers gem has almost reached feature parity, and has more support for maintenance, plus it covers all selenium drivers in one package.

I would have just swapped chromedriver-helper out for WebDrivers, but for a note on the WebDrivers README about using a proxy.

So I just need a bit more time to work out whether proxy support should be handled in the gem config, or maintained in the driver config as it currently is.

save_and_open_page() outputs to project root

Currently if an error occurs save_and_open_page will be called to take a snapshot of the failing page. However these files are saved to the root which overtime gets in the way of developers selecting files to be edited.

These should be saved to a folder at least which can then be cleaned out periodically as needed

Don't use driver as an assumption of running on CI

Currently Quke will assume you are running on a CI server if the driver phantomjs is selected. It then changes its error behaviour so that as soon as an error has been hit it will fail rather than waiting to run the rest.

https://github.com/EnvironmentAgency/quke/blob/eb8fb5fdc8522497d2fd547f3e73be3459048a41/lib/features/support/after_hook.rb#L9

However experience has now shown that some people don't always use phantomjs, for example going with a combo of XVFB and Chrome. Or it could be that they wish to see all tests attempted rather than failing immediately.

Therefore this is a note to remove this assumption and instead use a flag set in the config file to determine how the after_hook should work, i.e. should it error immediately or not.

focus label left against rspec tests stops all tests running

In PR #42 the label focus: true was left against the tests written for changes which were being done to the Quke::Configuration class.

This means that when Travis-CI now runs its only running these tests. These labels need to be removed and ideally this needs to be prevented from happening again.

Set whether JavaScript errors will throw an exception

This was highlighted by someone using Quke with PhantomJS as their selected 'driver'.

Currently Quke was configuring PhantomJS via Poltergeist to throw an error if a page had a JavaScript error. However they were looking to test other functionality within the application, and whether the page threw a JavaScript error was outside the scope and ability of the team to fix. Hence they wanted to override this setting.

This issue covers how Quke should configure JavaScript error handling across the browsers.

Event listener

Using selenium it's possible to output a log of the events for certain actions to enhance debugging.
http://bit.ly/saucecon2018-15
Can this be hooked into quke somehow?

Ability to specify a proxy for each driver

We have found that in some build environments access to the internet is provided by a proxy. Therefore for Quke to work we need to be able to tell PhantomJS and Selenium what the address and port for the proxy server is.

Handling tests using javascript during headless testing

v0.1.0

Currently default driver is set to poltergeist, if there is a test that requires javascript to execute then the test will fail when the test is run.

There either needs to be a way to run any javascript dependant tests using another driver or tagging the test not to run during headless testing.

Running tests against different environments

It's possible to set the url using app_host (https://github.com/jnicklas/capybara/tree/2.1.0#calling-remote-servers)

Would there be a way to run against different environments (urls') by setting the environment when running the tests?

Possibly having set environments dev, QA, staging, prod for example rather than providing the actual url.

Use Capybara's implicit wait system

SitePrism by default will return immediately if an element or section is not found on the page. One of the benefits of Capybara is that you don't have to manually add these into your steps, which reduces the noise and possibility of errors, and means tests run quicker.

You can tell SitePrism to use Capybara's implicit wait system as per https://github.com/natritmeyer/site_prism#using-capybara-implicit-waits and ideally Quke should come with this set.

Passing tags when parallel enabled causes no scenarios to run

For some reason if you pass in --tags ~@email or something of that nature with parallel enabled it returns immediately saying no tests have been run.

It works fine however if you do not have parallel enabled.

The frustration is behind the scenes ParallelTests is being used in both cases, it's just a case of what args are being passed to it.

Add `no_proxy` support for PhantomJS

PR #49 added support for specifying a no_proxy option in the proxy configuration details but only for selenium based drivers (Firefox and Chrome). Ideally we'd like to extend this support to PhantomJS as well but research at the time seemed to indicate this wasn't possible.

We'd like to go back and revisit this properly to confirm if that is the case, and if so do any workarounds exist.

Enable ability to override driver settings

One of the main things Quke does for users is remove the requirement to setup the underlying drivers used to run the tests.

This tends to be a complex area and how you do it differs depending on the driver used. Quke is based on the presumption that a user simply wants to run their tests against browser x, hence it takes care of this.

However there may be times a user needs to configure a driver differently. For example when using Quke to test an app recently we found that when using Chrome on OSX an app would default to its mobile rather than desktop view. Manually we could bypass this by spoofing the agent in the browser, but because these settings are hidden from Quke users, we didn't have the means of doing the same for our tests.

Therefore this feature is about providing a mechanism to allow users to tell Quke how a driver should be configured, overriding what it would do normally.

Need to investigate why this is happening. When this was happening phantomjs was the selected driver.