QA Suite

QA suite for running automated tests on websites, powered by Docksal.

Features:

Behat 3.x
BackstopJS 2.x
PhantomJS, SlimerJS (with Firefox ESR), CasperJS

Setup instructions

Step #1: Docksal environment setup

This is a one time setup - skip this if you already have a working Docksal environment.

Follow Docksal environment setup instructions

Step #2: Project setup

Clone this repo into your Projects directory

git clone [email protected]:docksal/qa-suite.git qa-suite
cd qa-suite

Initialize the stack
```
fin init
```
Make sure to follow on-screen directions.

Usage

Behat

Run all features under tests/behat/features

fin behat

Run a particular feature

fin behat features/blackbox.feature

BackstopJS

Take reference screenshots

fin backstop reference --config=backstop-example.json

Take test screenshots and compare

fin backstop test --config=backstop-example.json

By default, BackstopJS is using PhantomJS to take screenshots.
You can also use SlimerJS/Firefox by setting "engine": "slimerjs" in the configuration file.

Pre-configured sample commands

Pre-configured commands are in .docksal/commands.
They can be used as shortcuts/alternative to typing long lists of arguments every time.

Run sample BDD (behavior-driven development) tests with Behat

fin behat-example

Open http://qa-suite.docksal/tests/behat/reports/html/

Run sample VRT (visual regression testing) tests with BackstopJS

fin backstop-example

Open http://qa-suite.docksal/tests/backstop/data/example/reports/html/

Debugging

For tests that utilize Chrome/Firefox via Selenium (@javascript tests), you can watch the browser in real time inside the container with VNC Viewer.

Connect to 192.168.64.100:5900, password: secret.

sh: drush: not found

This suite works great until I try to use any of the Drupal specific steps.

Here is my behat.yml:

# Local overrides to the default profile
default:
  suites:
    default:
      contexts:
        - Drupal\DrupalExtension\Context\DrupalContext
        - Drupal\DrupalExtension\Context\MinkContext
        - Drupal\DrupalExtension\Context\MessageContext
        - Drupal\DrupalExtension\Context\DrushContext
        - FeatureContext:
            parameters:
              screenshot_dir: %paths.base%/screenshots

  extensions:
    Behat\MinkExtension:
      goutte:
        guzzle_parameters:
          verify: false
      selenium2: ~
      # Use goutte (basic PHP browser, super fast) as the default driver.
      default_session: goutte
      # For real browser testing and tests requiring JS use selenium2 driver.
      # Tag features/scenarious with @javascript to use the selenium2 driver.
      javascript_session: selenium2
      # Configure browser to be used. Browser must be available on wd_host.
      # Stick with chrome by default. It's 2x faster than firefox or phantomjs (your results may vary).
      browser_name: chrome
      selenium2:
        wd_host: http://browser:4444/wd/hub
        capabilities: { "browser": "chrome", "version": "*" }

    Drupal\DrupalExtension:
      drush:
        alias: 'self'
      blackbox: ~
      api_driver: 'drush'
      # Map template regions.
      region_map:
        header: "#header"
        content: "#content"
        footer: "#footer"
      # Map selectors.
      selectors:
        message_selector: '#messages'
        error_message_selector: '#messages .messages.error'
        success_message_selector: '#messages .messages.status'

    # Behat HTML formatter.
    emuse\BehatHTMLFormatter\BehatHTMLFormatterExtension:
      name: html
      renderer: Twig,Behat2
      file_name: index
      print_args: true
      print_outp: true
      loop_break: true

docker:
  extensions:
    Behat\MinkExtension:
      goutte: ~
      selenium2: ~

Here is my feature:

Feature: Test Boilerplate d8 Project Administrator Role
  As an administrator user
  I should be able to log in

  @api
  Scenario: Open home page and find text
    Given I am on "http://drupal8.docksal:8080"
    Given I am logged in as a user with the "administrator" role
    Then I should see text matching "My account"

I get the following result:

$ fin behat features/d8-admin.feature
Feature: Test Boilerplate d8 Project Administrator Role
  As an administrator user
  I should be able to log in

  @api
  Scenario: Open home page and find text                         # features/d8-admin.feature:6
    Given I am on "http://drupal8.docksal:8080"                  # Drupal\DrupalExtension\Context\MinkContext::visit()
    Given I am logged in as a user with the "administrator" role # Drupal\DrupalExtension\Context\DrupalContext::assertAuthenticatedByRole()
      sh: drush: not found
       (RuntimeException)
    Then I should see text matching "My account"                 # Drupal\DrupalExtension\Context\MinkContext::assertPageMatchesText()

--- Failed scenarios:

    features/d8-admin.feature:6

1 scenario (1 failed)
3 steps (1 passed, 1 failed, 1 skipped)
0m0.19s (11.40Mb)

fin drush does run fine on the qa-suite container which I wouldn't expect from the error message. Is there some kind of magic with where I'm supposed to put the aliases for these sites? I'm testing against other docksal sites, so should I create aliases for those sites somewhere in this qa-suite container? Is there a way for me to specify the alias in the feature itself? I'd like to be able to test many different sites using this one container.

docksal / qa-suite Goto Github PK

qa-suite's Introduction

QA Suite

Setup instructions

Step #1: Docksal environment setup

Step #2: Project setup

Usage

Behat

BackstopJS

Pre-configured sample commands

Debugging

qa-suite's People

Contributors

Stargazers

Watchers

Forkers

qa-suite's Issues

Recommend Projects

Recommend Topics

Recommend Org