Sauce Labs Testrunner Toolkit is a containerized testing solution that simplifies user setup, speeds up test execution time and supports native Javascript frameworks like Puppeteer and Playwright for running end-to-end web tests with Sauce Labs.
- Docker installed
- Make sure the Docker daemon is running (e.g.
docker info
works in your terminal) - A Sauce Labs account; if you don't have one, start a free trial
Using NPM:
npm i -g saucectl
or via our one line installer:
curl -L https://saucelabs.github.io/saucectl/install | bash
Would you like to inspect the content of our one line installer? Download it, have a look and execute it:
curl -fsSL -o get_saucectl.sh https://saucelabs.github.io/saucectl/install
chmod 700 get_saucectl.sh
./get_saucectl.sh
A Sauce Labs user name and access key are needed to post the test results. You can get your SAUCE_ACCESS_KEY
from
Account > User Settings in Sauce Labs.
To authenticate yourself, the following environment variables need to be set:
SAUCE_USERNAME
SAUCE_ACCESS_KEY
You can export them as follows:
export SAUCE_USERNAME=<your-username>
export SAUCE_ACCESS_KEY=<your-access-key>
If you are using a cloud CI/CD tool, we strongly suggest protecting these values through secrets or context variables.
saucectl new
This command will ask you to choose one of the frameworks:
After that, a ./sauce/config.yml
file and an example test under
the tests
directory will be created, where you can start working from.
saucectl run
This command will run the example test based on the ./.sauce/config.yml
file.
To learn more about saucectl
and its commands and flags,
please visit the saucectl repository.
saucectl
requires a configuration file to know what tests to run and what
framework to use. By default, .sauce/config.yml
will be the place where
saucectl
will look for its configuration.
# Simple puppeteer example config
apiVersion: v1
metadata:
name: Feature XYZ
tags:
- e2e
- release team
- other tag
build: Release $CI_COMMIT_SHORT_SHA
files:
- ./tests/**/*.js
image:
base: saucelabs/stt-puppeteer-jest-node
version: latest
sauce:
region: us-west-1
If you wish to use more than one framework, or to configure different sets of
tests separately, you could use any name for the configuration file, and
specify it through saucectl run -c ./path/to/config.yml
.
As an example, this repository uses two configurations for its pipeline. One for Puppeteer, and one for Playwright.
NOTE: Test files need to match
(spec|test)
in their file name so they will be automatically detected as testfiles.
All images are hosted on Docker Hub.
Base image
is called testrunner
. It contains the tooling necessary to record videos, VNC etc. Plus Chrome, and a Firefox version.
Base image + Playwright contains saucectl with different versions of Playwright.
Base image + Puppeteer contains saucectl with different versions of Puppeteer.
Base image + TestCafe contains saucectl with different versions of TestCafe.
Base image + Cypress contains saucectl with different versions of Cypress.
The examples here show how Pipeline testing can be used. Try them and find your own use cases. Every testrunner image comes with a preconfigured setup that allows you to focus on writing tests instead of tweaking with the configurations. Our initial testrunner flavors come either with Puppeteer or Playwright as an automation framework. They will start the browser for you and expose the browser
object (Puppeteer / Playwright) to the global scope for you to be accessible in the test:
Our Puppeteer testrunner image exposes browser
into the global scope which represents an instance of its Browser class
. The browser will be initiated and shutdown by the testrunner setup.
describe('saucectl demo test', () => {
test('should verify title of the page', async () => {
const page = (await browser.pages())[0]
await page.goto('https://www.saucedemo.com/');
expect(await page.title()).toBe('Swag Labs');
});
});
The Playwright testrunner image also exposes a global browser
variable that represents Playwright's Browser class
. In addition to that you also have access to a pre-generated browser context via context
as well as to a page frame via page
.
describe('saucectl demo test', () => {
test('should verify title of the page', async () => {
await page.goto('https://www.saucedemo.com/');
expect(await page.title()).toBe('Swag Labs');
});
});
import { Selector } from 'testcafe';
fixture `Getting Started`
.page `http://devexpress.github.io/testcafe/example`
const testName = 'testcafe test'
test(testName, async t => {
await t
.typeText('#developer-name', 'devx')
.click('#submit-button')
.expect(Selector('#article-header').innerText).eql('Thank you, devx!');
});
context('Actions', () => {
beforeEach(() => {
cy.visit('https://example.cypress.io/commands/actions')
})
it('.type() - type into a DOM element', () => {
// https://on.cypress.io/type
cy.get('.action-email')
.type('[email protected]').should('have.value', '[email protected]')
})
})
Native JavaScript testing is achieved through the combination of Sauce Labs, Jest, and the JavaScript framework of your choice. In the current beta, the toolkit supports Puppeteer, Playwright and TestCafe. Cypress. This approach gives you the power and expressiveness of different test frameworks with the dashboards, infrastructure, and analytics of Sauce Labs.
The specific framework you want to use to for testing should be based on the types of tests you need to run and the environment in which you are running them. In this beta you will be able to run tests in your existing CI pipeline and benefit from the low latency.
When tests are completed, logs, results, and videos will be uploaded to Sauce Labs to your account. After that, you can review, share, and analyze those results just as you would from any other test executed on Sauce Labs.
To learn more about:
- Jest, visit https://jestjs.io/
- The Google Puppeteer project, visit https://developers.google.com/web/tools/puppeteer
- The Microsoft Playwright project, visit https://github.com/microsoft/playwright
- TestCafe, visit https://devexpress.github.io/testcafe/
- Cypress, visit https://github.com/cypress-io/cypress
This repository includes examples of CI/CD in GitHub Actions Workflows and CircleCI Pipelines. Although the GitHub Actions and CircleCI examples are included, the mechanism works with any CI/CD provider that supports containers.
There is a lot we can imagine doing next. It starts with hearing from you. Submit issues and features here - everything helps!
The Sauce Labs Testrunner Toolkit is part of our commitment to a world of digital confidence where each of our digital lives and experiences are magical. If you are thinking about getting involved, please do. This repository is focused on helping people learn how to test their user experience. More is welcome.