Giter VIP home page Giter VIP logo

playwright-bdd's Introduction

playwright-bdd

lint test npm version npm downloads license

Run BDD tests with Playwright runner.

Inspired by the issue in the Playwright repo microsoft/playwright#11975

Tip

Playwright-bdd v6 is out! Check out Cucumber reporters and share your feedback

Why Playwright runner?

Both Playwright and CucumberJS have their own test runners. You can use CucumberJS runner with Playwright as a library to test BDD scenarios. This package offers an alternative way: convert BDD scenarios into Playwright tests and run them with Playwright runner as usual. Such approach brings all the benefits of Playwright runner:

Documentation

Check out documentation website.

Example

Fork and play with playwright-bdd-example repo.

Feedback

Feel free to share your feedback in issues.

Changelog

Inspect the latest changes in the CHANGELOG.md.

Contributing

Your contributions are welcome! Review DEVELOPMENT.md for playwright-bdd local setup and development.

Sponsors

Great thanks to the sponsors for supporting playwright-bdd project ❤️ Become a sponsor

Currents.devArturs LeščinskisAntoine Heinrich

How to make BDD valuable for my project?

Have a look on this section.

License

MIT

playwright-bdd's People

Contributors

cassus avatar ideadapt avatar nonjerry avatar ron-myers avatar ronvoluted avatar vitalets avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

playwright-bdd's Issues

Support custom fixtures

Thank you for starting this project, it's great to see progress on the playwright-bdd front!

I'm wondering how would it be possible to use custom fixtures in the steps. This would enable component testing (mount), or other custom fixtures I'm using in my project (especially worker fixtures which are a great performance optimization)

Snapshot folder and file not being created when we use ToHaveScreenshot()

Hi @vitalets,

because of the change (issue 40, concerning the Simplify generated directories structure) I had to delete the features-gen folder.

Now after I run the 'npx bddgen' all features are being re-generated with the new structure and all is good with them.

The problem comes when we want to use visual comparison tests.
So after I clean I do not have anymore the snapshots folder and the snapshots inside.

Just for info, when using clean playwright that snapshot folder with the snapshot file will be automatically generated for me (cause they do not exist) when I use the ToHaveScreenshot() method.

Unfortunatelly with playwright-bdd when the test come to the method ToHaveScreenshot it hangs and after a while test fails.
The message I see is -> Internal error: worker process exited unexpectedlly (code=134, signal=null)

Last time I think I fix that when I manually created snapshot folder with a file inside, will try to do the same, but I was wondering if you can help with real fix, so that we do not need to create snapshot folder manually.

Question: is there a possibility to import defineStep instead of gherkin keywords?

Is there a possibility to import defineStep instead of Given, When, Then to create a step definitions?

I'm using playwright style to create a step definition and some of the steps might be reused as Given/When and I don't want to duplicate the step definition. With defineStep, there is no need to specify the Gherkin keyword in the step definition.

Add config option to set quotes type in generated tests: single / double / backtick

Currently generated tests use double quotes for strings. At the same time, most of feature files that I see also use double quotes for string parameters (including samples on cucumber.io). That causes a lot of escapes in test files and reports:

await When("I click link \"Get started\"", null, { homePage });
await Then("I see in title \"Installation\"", null, { page });

I would look more nicely if there will be single quotes:

await When('I click link "Get started"', null, { homePage });
await Then('I see in title "Installation"', null, { page });

Suggestion is to add config option quotes: 'double' | 'single' | 'backtick' to allow user control it.

Project dependencies not working when using an authentication setup

I'm trying to implement authentication setup using a project dependency as it is shown in this doc https://playwright.dev/docs/auth

So for that I've created an auth.setup.ts file that includes the loging process and then I configured the playwright.config.ts with:

  projects: [
    { 
      name: 'setup',
      testMatch: '**/*setup.ts' 
    },
    {
      name: 'Regresion tests using chromium',
      dependencies: ['setup'],
      use: { 
        ...devices['Desktop Chrome'],
        storageState: STORAGE_STATE, 
      },
    },

When I run my test cases I get this error:
Error: Error reading storage state from myprojectdir\.auth\user.json: ENOENT: no such file or directory

It seems that the project-dependency is not triggered.

Simplify generated directories structure

Currently directory structure is replicated inside .features-gen. For example, having the following .feature file:

features
├── path
    └── to
        └── features
            └── example.feature

bddgen will generate the output:

.features-gen
└── features
   └── path
      └── to
         └── features
            └── example.feature.spec.js

Although it can be simplified to:

.features-gen
└── example.feature.spec.js

Simplified structure is especially useful for checking generated snapshots (stored next to test files itself).
This is breaking change as all previously saved snapshots will be invalid.

Is there an example of how to handle projects using POM pattern

Hi
I have followed the steps and trying to figure out if the is a config I need to change to generate PW tests for a project that is using Page Object Model Pattern.

when I run playwright-bdd

I get the following spec.js format

/** Generated from: features\tools\DataExport.feature */
import {test} from "playwright-bdd";

test.describe("Data Export", () => {

    test("Create Data Export", async ({Given, When, Then}) => {
        await Given("Go to login page");
        await When("User logs in user name \"User1\" and password \"Passw0rd1\", \"\"");
        await Then("I select \"Tools\" \"Data Export\" view");
        await Then("I create an Data Export");
    });

});

Which is missing any page object references - for example this how the original step file looks

Then('I create Data Export', async()=>{
    const dataexport = new DataExportPage();
    await dataexport.createDataExport();

});

I have tried including the page classes in the cucumber config

Allow users to specify a custom path for feature files.

First of all, thank you for putting together this library! I am excited!

At the moment the bddgen bin expects the feature files to be found at feature/*.feature, however I'd love to define them in tests/features/.feature.

Creating an issue now and might work on a PR if there is interest!

Some inconsistencies within the html report when using POM

So I have following structure:
features
steps
pages/sections

In the html report I sometimes see wrong snapshot of the code in that second and third level.

See following examples:

image

Also when we have iframes it does not look very nice.

Another example:
image

Wrap each cucumber step in `test.step`

Hello,

I would like to combine using playwright-bdd with allure-playwright. The latter one allows to generate rich HTML reports of test runs. However, instead of showing details of execution with low-level playwright API steps, I would like it to show details in the form cucumber feature steps.

Right now, if I set details: false in allure-playwright, nothing is shown.
image

It's caused by the fact, that with details: false it only shows steps defined with test.step function.

Generated test file:

/** Generated from: features\example.feature */
import { test } from "playwright-bdd";

test.describe("Playwright site", () => {

  test("Check title", async ({ Given, When, Then }) => {
    await Given("I open url \"https://playwright.dev\"");
    await When("I click link \"Get started\"");
    await Then("I see in title \"Playwright\"");
  });

});

And if I wrap each cucumber step with test.step

/** Generated from: features\example.feature */
import { test } from "playwright-bdd";

test.describe("Playwright site", () => {

  test("Check title", async ({ Given, When, Then }) => {
    await test.step("Given I open url \"https://playwright.dev\"", () => Given("I open url \"https://playwright.dev\""));
    await test.step("When I click link \"Get started\"", () => When("I click link \"Get started\""));
    await test.step("Then I see in title \"Playwright\"", () => Then("I see in title \"Playwright\""));
  });

});

everything looks great in allure report:
image

So my questions are:

  1. Would it be possible to wrap each cucumber step with playwright's test.step during test generation?
  2. If not, is there a possibility to hook into to test generation process and make slight modifications? I think this could be useful not only in this case, but also in different scenarios as well.

Actually, if I understand the source code correctly and if it makes sense to always define playwright step, then maybe it could be changed directly on fixture level, somwhere inside src/run/invoke.ts:L6-L14

Reverse mode: auto-generate BDD scenarios

In Playwright there is a codegen feature that generates test-code while you perform actions in browser. What I'm thinking about is to generate BDD code in some similar way.

Imagine the following scenario:

  1. open todo app
  2. create two todos
  3. mark the first todo as completed
  4. filter only completed todos

In codegen mode Playwright will generate the test:

import { test, expect } from '@playwright/test';

test('test', async ({ page }) => {
  await page.goto('https://demo.playwright.dev/todomvc/#/');
  await page.getByPlaceholder('What needs to be done?').fill('todo 1');
  await page.getByPlaceholder('What needs to be done?').press('Enter');
  await page.getByPlaceholder('What needs to be done?').fill('todo 2');
  await page.getByPlaceholder('What needs to be done?').press('Enter');
  await expect(page.getByTestId('todo-title')).toHaveCount(2); // <- added manually
  await page.getByRole('listitem').filter({ hasText: 'todo 1' }).getByRole('checkbox', { name: 'Toggle Todo' }).check();
  await page.getByRole('link', { name: 'Completed' }).click();
  await expect(page.getByTestId('todo-title')).toHaveCount(1); // <- added manually
});

The code looks rather structured and it would be possible to convert it into BDD scenario:

Scenario: Scenario 1
    Given I am on page "https://demo.playwright.dev/todomvc/#/"
    When I fill "todo 1" and press Enter
    And I fill "todo 2" and press Enter
    Then Count of "todo-title" equals to 2
    When I enable checkbox for "todo 1"
    And I click on "Completed"
    Then Count of "todo-title" equals to 1

Necessary step definitions can be generated as well:

Given('I am on page {string}', async ({ page }, url: string) => {
  await page.goto(url);
});

When('I fill {string} and press Enter', async ({ page }, text: string) => {
  await page.getByPlaceholder('What needs to be done?').fill(text);
  await page.getByPlaceholder('What needs to be done?').press('Enter');
});

Then('Count of {string} equals to {int}', async ({ page }, testId: string, count: number) => {
  await expect(page.getByTestId(testId)).toHaveCount(count);
});

When('I enable checkbox for {string}', async ({ page }, hasText: string) => {
  await page.getByRole('listitem').filter({ hasText }).getByRole('checkbox', { name: 'Toggle Todo' }).check();
});

When('I click on {string}', async ({ page }, name: string) => {
  await page.getByRole('link', { name }).click();
});

If I record second scenario existing step definitions can be reused.

I feel that there can be edge cases, but developer can always fix it manually.
In general having such easy way to generate BDD scenarios looks useful to define project requirements and behavior.

Would appreciate your thoughts on that :)

Feature: support several custom test instances

Describe the problem
Currently only one custom test instance can be provided via importTestFrom option. In many cases this is no enough. Depending on project structure, different groups of tests may require own auto-fixtures, not applicable for other tests. Even without auto-fixtures sometimes it's more convenient to keep fixtures separated instead of having all-in-one test.extend.

Imagine we have an app that has 3 main parts: user login, todo list and administration of uses. Each part could have own Page Object Models and fixtures. Example structure:

features
    login.feature
    todolist.feature
    admin.feature
steps
   Login
      LoginPage.ts
      LogoutPage.ts
      fixtures.ts
   TodoList
      TodoListPage.ts
      TodoDetailsPage.ts
      fixtures.ts
   Admin
      UserList.ts
      UserDetails.ts
      fixtures.ts

Currently to run such tests with playwright-bdd we need to combine all fixtures.ts into one and point to it with importTestFrom.

Describe the solution
The solution I'm thinking about - is to refuse using importTestFrom at all. Technically, when we know all steps used in particular feature file, we can detect what test instance should be imported. To achieve it we need to switch from Cucumber's steps loading to own implementation (that is already used for decorator steps). To make it backwards compatible I can add new config option steps: string | string[] that should point to all steps definitions including fixtures. When steps option is defined, no need for Cucumber's require or import options and laso no need for importTestFrom. The config will looks very simple:

const testDir = defineBddConfig({
  paths: ['features'],
  steps: './steps',
});

Describe alternatives you've considered
Alternatively we can allow passing object to importTestFrom mapping different features with different test instances, e.g.:

const testDir = defineBddConfig({
  importTestFrom: {
    'features/Login': 'features/Login/fixtures.ts',
    'features/TodoList': 'features/TodoList/fixtures.ts',
    'features/Admin': 'features/Admin/fixtures.ts',
  },
  paths: ['features'],
  steps: './steps',
});

This is less convenient as we need manually sync changes in files structure with config.

Use decorators to define steps in Page Object Models

Imagine I use TodoPage fixture (page object model) from Playwright docs:

// fixtures.ts

export class TodoPage {
  private readonly inputBox: Locator;
  private readonly todoItems: Locator;

  constructor(public readonly page: Page) {
    this.inputBox = this.page.locator('input.new-todo');
    this.todoItems = this.page.getByTestId('todo-item');
  }

  async goto() {
    await this.page.goto('https://demo.playwright.dev/todomvc/');
  }

  async addToDo(text: string) {
    await this.inputBox.fill(text);
    await this.inputBox.press('Enter');
  }
}

Typical BDD step definitions for todoPage will be:

// steps.ts

Given('I am on todo page', async ({ todoPage }) => {
  await todoPage.goto();
});

When('I add todo {string}', async ({ todoPage }, text: string) => {
  await todoPage.addToDo(text);
});

For me such step definitions look like a boilerplate code. It would be more convenient to just mark todoPage methods with step text - the same way as it's done in Cucumber Java. Having new JS decorators syntax it should be possible:

// fixtures.ts

export class TodoPage {
  private readonly inputBox: Locator;
  private readonly todoItems: Locator;

  constructor(public readonly page: Page) {
    this.inputBox = this.page.locator('input.new-todo');
    this.todoItems = this.page.getByTestId('todo-item');
  }

  @Given('I am on todo page')
  async goto() {
    await this.page.goto('https://demo.playwright.dev/todomvc/');
  }

  @When('I add todo {string}')
  async addToDo(text: string) {
    await this.inputBox.fill(text);
    await this.inputBox.press('Enter');
  }
}

In that case there is no need for additional steps.ts file. We just define fixture and write scenarios.

Possible downside is that VSCode BDD extensions maynot recognize such step definitions (to be checked).

Would appreciate your ideas on that.

Example for processing tabular data defined in feature file

Would it be possible to extend the current example to show how tabular data would be handled in the step definitions?

For instance, the step definition might be something like:
When('I create an entity with properties', async ({ page }, data: DataTable) => { /* do stuff */ });

Generate snippets for undefined steps

Running cucumber with undefined step:

> cucumber-js

Failures:

1) Scenario: Check home # features/scenario.feature:3
   ? Given I open url "https://playwright.dev"
       Undefined. Implement with the following snippet:
       
         Given('I open url {string}', async function (string) {
           // Write code here that turns the phrase above into concrete actions
           return 'pending';
         });

Running playwright-bdd with undefined step:

  1) [chromium] › features/sample.feature.spec.js:6:7 › Playwright site › Check title ──────────────

    Error: Unknown step: I open url "https://playwright.dev"

See: https://github.com/cucumber/cucumber-js/blob/main/docs/snippets.md

Not able to run visual comparison tests - toHaveScreenshot() is not working

Hello @vitalets I am trying to run a test that should compare screenshots. -> https://playwright.dev/docs/test-snapshots

When I use await expect(page).toHaveScreenshot(); and I do not have a master/golden file, this should generate for me a .png file in a specific directory
image

Unfortunatelly when I run the test this snapshot directory is not created automatically and get some internal error

image

Could you please try to run some visual comparison test in your end to see it is working.

Support UI Mode

Maybe I am wrong, but for me Playwright UI Mode is not finding my tests when using playwright-bdd.
https://playwright.dev/docs/test-ui-mode

I try to run it by using: npx bddgen && npx playwright test --ui

Supporting UI Mode would bring a lot of functions and productivity in my opinion.

Let me know what you think or if i initialize it wrong.

Random test failures when running >40 tests in parallel

I have a test suite with about 45 Scenarios, split across about 30 feature files.
When I run them individually, all is good.
When I run less than 40, fully parallel, all is good
When I run more than 40, fully parallel, I experience random failures, always from the tests executed towards the end
If I shard them and run them in batches of 20 or 30, no failure happens

This is the error being thrown:

    TypeError: Cannot read properties of null (reading 'includes')

        at World.invokeStep (/Users/myName/workspace/myProject/node_modules/playwright-bdd/src/run/world.ts:53:41)
        at /Users/myName/workspace/myProject/starter-kit/async /Users/myName/workspace/myProject/starter-kit/tests/pwFeature/.feature-gen/tests/pwFeature/textInput.feature.spec.js:19:5
        at /Users/myName/workspace/myProject/starter-kit/async /Users/myName/workspace/myProject/node_modules/@playwright/test/lib/worker/workerMain.js:351:9
        at /Users/myName/workspace/myProject/starter-kit/async /Users/myName/workspace/myProject/node_modules/@playwright/test/lib/worker/workerMain.js:347:7

My configuration:
OS: Mac OSx Ventura i7 32GB RAM
playwright-bdd: 3.2.1
@playwright/test: 1.33.0
cucumber: 9.1.0
chrome: 114
nodejs: 16.16.0
ts-node: 10.9.1

To write my step definitions, I am using cucumber style:

example:

Given('I am on the {string} page', async function(pageName: string) {
  const currentPage: PageInterface = globalThis[`${pageName}Page`];
  return currentPage.open();
});

To run my tests, I first run bddgen (there is nothing wrong with the generated spec files), then I use npx playwright test --config path/to/my/config

This is what my feature files look like:

Feature: Switch

  Background:
    Given create "switch" page

  Scenario: Switch displayed correctly
    Given I am on the "switch" page
    Then Switch is displayed correctly
    And Switch label is displayed correctly

They all include a background and are written in exactly the same way

Question: unable to run the CLI command

seems like the bddgen is no longer available?

✗  npx bddgen
npm ERR! code E404
npm ERR! 404 Not Found - GET https://registry.npmjs.org/bddgen - Not found
npm ERR! 404
npm ERR! 404  'bddgen@*' is not in this registry.
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

Generate test files close to Gherkin document structure

Currently generator of test files uses Cucumber Pickles that are intermediate data structures derived from Gherkin document. It's better to generate tests by Gherkin document itself, that gives the following benefits:

  • use the same keywords as in original Gherkin file (e.g. And / But)
  • map Background to test.beforeEach instead of inserting into every test
  • support Rule syntax (see #7)

Feature: Tags at the example level are not being picked up by the playwright grep

Describe the problem
If we want to run specific tests in a feature file by the tags at the example level, the only way is to use $tags and add logic based on that. Cucumber lets us run tests at the example level by those tags but not through playwright-bdd.

Describe the solution
Probably similar to how we are adding the @tags to the scenario description in order to let playwright grep recognize them, it might be possible to add the tag at the example level as well

Describe alternatives you've considered
Alternative I have tried is to pull the cucumber tags in steps via the $tags and added logic accordingly, but this is an overhead where as in cucumber I can directly run specific tests by using the tags

Additional context
N/A

Duplicate tests getting generated to handle Outline Scenarios

I started exploring "Playwright-BDD" extension and I see the test cases are getting duplicates for "Scenario Outline" which is not the case in Cucumber.js. Is there a way we can handle this as well in future ? Basically instead of generating duplicate tests, we need to find a way to consume data from parameters grid while executing tests. I do understand this is best option you had for handling Scenario Outline. Is it possible for Playwright Test and Playwright BDD extension to work together to come up with better solution ?

Potential Solution:
Generate in-line csv in to "test.describe" block and make the Tests repeat using the csv ? Is something like that possible using this playwright pattern for csv test repeats ?

Some background why I ask for this:

  • We use automation patterns to synchronize tests from Feature Files to Azure DevOps. Outline Feature (lets say I have a test case to test home page of website. I have 50 test data combinations cause we run some of our applications home page in 50 languages with 50 different URLs ). I have only ONE test case in ADO TestPlans with these 50 data combinations
  • Next when I run tests..it will test with all parameter combinations given in parameters section. Azure DevOps TesPlans stores test results under one test case.
  • When I Synchronize test results...I use Cucumber Json File which updates the test results for all combinations under one test case.
  • If I use this extension I need to have 50 test cases created in Azure DevOps and the Testers and Developers will get confused while reading tests as they all look same.

image

image

ERROR: When using several calls of defineBddConfig() after installing v4.0.0

Hello,
After installing v4.0.0, I get this error when doing the npx bddgen:

ERROR: When using several calls of defineBddConfig() please manually provide different "outputDir" option.

I'm running my features in different projects and I have differents outputDir for each project:

projects: [
    { 
      name: 'setup',
      testDir: defineBddConfig({
        outputDir: '.features-gen/login',
        importTestFrom: 'page-objects/basePage.ts',
        paths: ['features/login.feature'],
        require: ['steps/loginSteps.ts'],
      }),
    },
    {
      name: 'Regresion tests using chromium',
      testDir: defineBddConfig({
        outputDir: '.features-gen/regression',
        importTestFrom: 'page-objects/basePage.ts',
        paths: ['features/project.feature'],
        require: ['steps/projectSteps.ts'],
      }),
      dependencies: ['setup'],
      use: { 
        ...devices['Desktop Chrome'],
        storageState: STORAGE_STATE, 
      },
     {
      name: 'Regresion tests using edge',
      testDir: defineBddConfig({
        outputDir: '.features-gen/regression',
        importTestFrom: 'page-objects/basePage.ts',
        paths: ['features/project.feature'],
        require: ['steps/projectSteps.ts'],
      }),
      dependencies: ['setup'],
      use: { 
        ...devices['Desktop Edge'],
        storageState: STORAGE_STATE, 
      },
    },

Note: when rolling back to 3.3.0, it works just fine

Support hooks (Before, BeforeAll, After, AfterAll)

As already described hooks are currently not supported.
When generating the spec files from the feature files you can also not know that hooks are defined.

Is it an option to add a config option where you can define the location of the file(s) with hooks?
In that way you can add hooks to the generated spec files.

Support Cucumber reporters

Cucumber has several built-in reporters (also called formatters). It would be useful to output results of Playwright test runner to these reporters (especially html-reporter or messages reporter).
There are different approaches:

  1. Write custom Playwright reporter that will create Cucumber report
  2. Use Playwright json report and convert it into Cucumber messages, and then convert to html

Ideally, final usage should allow to use any Cucumber report:

import { defineConfig, devices } from '@playwright/test';
import { cucumberReporter } from 'playwright-bdd';

export default defineConfig({
  // ...
  reporter: cucumberReporter('json:cucumber-report.json'),
});

Known problem - reporting Scenario Outline.
In Cucumber it is possible to have the same test title with different data. In Playwright each test must have unique title (it was also discussed in #3 (comment)). Example of Cucumber html report for Scenario Outline:

I would appreciate more ideas on that from your reporting usage.

Set up SyntaxError

Hi, I'm trying to set up playwright and cucumber from scratch in a project, but keep on getting this syntax error:

/Users/user/WebstormProjects/project/tests/steps/steps.ts:38
import { createBdd } from 'playwright-bdd';
^^^^^^

SyntaxError: Cannot use import statement outside a module

I've tried adding "type" : "module" in package.json file, but then I'm getting:

Error: Cucumber expected a CommonJS module at '/Users/user/WebstormProjects/project/tests/steps/steps.ts' but found an ES module.
      Either change the file to CommonJS syntax or use the --import directive instead of --require.
      
      Original error message: Must use import to load ES Module: /Users/user/WebstormProjects/project/tests/steps/steps.ts
require() of ES modules is not supported.
require() of /Users/user/WebstormProjects/project/tests/steps/steps.ts from /Users/user/WebstormProjects/project/node_modules/@cucumber/cucumber/lib/try_require.js is an ES module file as it is a .ts file whose nearest parent package.json contains "type": "module" which defines all .ts files in that package scope as ES modules.
Instead change the requiring code to use import(), or remove "type": "module" from /Users/user/WebstormProjects/project/package.json.

steps.ts file:

import { expect } from '@playwright/test'
import { createBdd } from 'playwright-bdd';

const { Given, When, Then } = createBdd();

Given('I open url {string}', async ({ page }, url) => {
    await page.goto(url);
});

When('I click link {string}', async ({ page }, name) => {
    await page.getByRole('link', { name }).click();
});

Then('I see in title {string}', async ({ page }, keyword) => {
    await expect(page).toHaveTitle(new RegExp(keyword));
});

playwright.config.js

import { defineConfig } from '@playwright/test';
import { defineBddConfig } from 'playwright-bdd';

const testDir = defineBddConfig({
    paths: ['tests/features/**/*.feature'],
    require: ['tests/steps/steps.ts'],
    requireModule: ['ts-node/register'],
});

export default defineConfig({
    testDir,
    reporter: 'html',
});

Any help would be appreciated!:)

Possibility of adding Playwright-bdd extension in to playwright.conf.ts

Currently we need to run two commands together "npx bddgen && npx playwright test". Can we configure the extension in to playwright.conf.ts like we configure various reporter extensions ? I understand there is no option to add third party extensions in Playwright Test currently other than reporters section. Can you add "playwright-bdd" in to reporter section similar to this Playwright Azure DevOps TestPlans Extension. This will make users run one command directly.

One problem to think of here is how will VS Code Extension work ? Can it identify the tests since tests are not auto generated at this point ? Can Playwright team make change to "VS Code Playwright extension" to run "npx bddgen" command based on "playwright.conf.ts" and detect the tests ?

Bug: scenarios in the same feature create tests using different fixtures

short description
I am using classes extending one from others to share functionalities.
But as you guys know class can hold state as member variables.
even though I use the method that is inherited from the parent class, the state can be different.
in below test, even though I used the "Given" decorator to choose the child class, the result is just chosen based on where the specific decorator is located (parent class).
When the feature file is marked with specific fixture, it should consistently use the same fixture, but in below example, you will see the fixture for each tests are dynamically chosen based on the location of the each decorator.
To fix this temporarily, I had to use a specific decorator that only belong to the fixture and it makes my BDD more verbose.

Given

playright config

import { devices, defineConfig } from "@playwright/test";

/**
 * Read environment variables from file.
 * https://github.com/motdotla/dotenv
 */
import dotenv from "dotenv";
import { defineBddConfig } from "playwright-bdd";


dotenv.config({
	path: "./.env",
});
console.log("setting on env:"+ process.env.NODE_ENV);

dotenv.config({
	path: `./.env.${process.env.NODE_ENV}`,
});
dotenv.config({
	path: `./.env.${process.env.NODE_ENV}.local`,
});

console.log("public url is:", process.env.PUBLIC_URL);
const isUi = process.env.UI === 'true';
const testDir = defineBddConfig({
	paths: ["./features"],
	importTestFrom: './steps/fixtures.ts',
	require: ['./steps/*.ts'],
	quotes: 'backtick' // when fixture filles are converted to spec.ts files, quotes will be replaced to the backticks
});

// 
export default defineConfig({
	testDir,
	timeout: 60 * 1000,
	expect: {
		/**
		 * Maximum time expect() should wait for the condition to be met.
		 * For example in `await expect(locator).toHaveText();`
		 */
		timeout: 10000,
	},
	/* Run tests in files in parallel */
	fullyParallel: true,
	/* Fail the build on CI if you accidentally left test.only in the source code. */
	forbidOnly: !!process.env.CI,
	/* Retry on CI only */
	retries: process.env.CI ? 2 : 0,
	/* Opt out of parallel tests on CI. */
	workers: process.env.CI ? 1 : undefined,
	reporter: [["html", { outputFolder: "./reports" }]],
	use: {
	  	actionTimeout: 0,
		/* Base URL to use in actions like `await page.goto('/')`. */
		baseURL: process.env.PUBLIC_URL ?? 'http://my-dev.dimensional.com',
		screenshot: 'only-on-failure',
		/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
		trace: "on-first-retry",
		testIdAttribute: "data-qa",
	},
	projects: [
		{
			name: "setup",
			testMatch: /\/__[a-zA-Z0-9].[a-zA-Z0-9_-]+\.feature\.spec\.js$/,
		},
		{
			name: "chromium",
			use: {
				...devices["Desktop Chrome"],
				// Use prepared auth state.
				storageState: ".auth/user.json",
			},
			testMatch: /\/[a-zA-Z0-9].[a-zA-Z0-9_-]+\.feature\.spec\.js$/,
			...(isUi? {}: {
				dependencies: ["setup"], // when UI mode, don't make dependency to save time.
			}),
		},
});

feature

@fixture:fundCenter
Feature: Some Feature
  Given: 
    Given I am on fund center  <=== choosing fund center. this whole feature should use fund center

  Scenario: Scenario 1
    Then I see "Some button" button
  Scenario: Scenario 2
    Then I see "Some tab" tab   <==== this decorator is in model center (parent class)

fixture

export
@Fixture<typeof test>("fundCenter")
class FundCenter extends ModelCenter{ // <==== extended from ModelCenter
        constructor(){
           super('fund');        
        }

        @Given("I am in fund center")
	async onFundCenter()
	{
		await this.openList('fund');
		this.modelStates.currentPage = "fund list";
	}	
       
        @Then("I see {string} button")
        async shouldSeeButton(button: string){
           ...
        }
}


export
@Fixture<typeof test>("modelCenter")
class ModelCenter {
     listType:string;
    constructor(listType: string = "model"){
       this.listType = listType; // by default, it is model, but it can be changed by child class
     }

    @Given("I am in model center")
	async onModelCenter()
	{
		await this.openList('fund');
		this.modelStates.currentPage = "fund list";
	}
        @Then("I see {string} tab")
        async shouldSeeTab(tab: string){
           ...
        }
}

fixtures.ts

type Pages = {
	modelCenter: ModelCenter;
	fundCenter: FundCenter;
};

export const test = base.extend<Pages>({
	modelCenter:async ({ page }, use) => use(new ModelCenter(page)),
	fundCenter:async ({ page }, use) => use(new FundCenter(page)),
});

When

npx bddgen

Then

It creates test using different fixtures

test.describe(`Some Feature`, () => {
  test.beforeEach(async ({ Given, fundCenter }) => {
    await Given(`I am on fundCenter`, null, { fundCenter });
  });
  test(`Scenario 1`, async ({ Then, fundCenter }) => {
    await Then(`I  see "some button" button`, null, { fundCenter });
  });

  test(`Scenario 2`, async ({ Then, modelCenter }) => {
    await Then(`I  see "some tab" tab`, null, { modelCenter }); // <== using modelCenter instead of fundCenter
  });

});

Expected behavior

for the same fixture, (especially when I explicitly defined @fixture: fundCenter), it should use the same fixture instead of using the class that holds original decorator.

Isolated demo

Environment

Playwright-bdd environment info:

platform: darwin
node: v16.14.2
playwright-bdd: v5.2.0
@playwright/test: v1.37.1
@cucumber/cucumber: v9.4.0
Playwright config file: playwright.config.ts

Customize test titles for scenario outline

Currently for scenario outline each example line converted to test with title like "Example #{row}".
For example:

Feature: calculator

    Scenario Outline: Check doubled
      Then Doubled <start> equals <end>

      Examples:
          | start | end |
          |    2  |   4 |
          |    3  |   6 |

converted to:

test.describe("calculator", () => {

  test.describe("Check doubled", () => {

    test("Example #1", async ({ Then }) => {
      await Then("Doubled 2 equals 4");
    });

    test("Example #2", async ({ Then }) => {
      await Then("Doubled 3 equals 6");
    });

The problem is that it's not reliable for reporting. If I comment some rows or insert rows in the table - test titles will shift.
Reporters that keep track of test history will show wrong results.

Suggested solution is to allow to provide format for test title using gherkin comment.
For example:

Feature: calculator

    Scenario Outline: Check doubled
      Then Doubled <start> equals <end>

      # title-format: Example for <start>
      Examples:
          | start | end |
          |    2  |   4 |
          |    3  |   6 |

In that case test titles will be bound to example data and will not change in case of adding/removing rows:

test.describe("calculator", () => {

  test.describe("Check doubled", () => {

    test("Example for 2", async ({ Then }) => {
      await Then("Doubled 2 equals 4");
    });

    test("Example for 3", async ({ Then }) => {
      await Then("Doubled 3 equals 6");
    });

Gherkin parser provides all comments with location in the file, so it should be possible to find and use such comments.

[Question] Data tables typescript

Hello @vitalets , great work with playwright-bdd 💯

One question concering datatables, as we are using a lot of parameters in our scenarious.

Do we have some examples on data tables?

E.g.

Given I am in registration page
And I provide below information during registeration
  | Fields        | Inputs                                  |
  | firstName     | Tom                                    |
  | lastName      | Jerry                                   |
  |  phone        | 123456789                           |
  |  email        | [email protected]  | 
  Scenario Outline: Create <NAME>
    Given a user called "<NAME>"  with email address "<EMAIL>" who lives in "<CITY>" and was born "<DOB>"

    Examples: 
      | NAME            | EMAIL           | CITY          | DOB        |
      | Vivienne Senger | [email protected]  | Everett       | 1965-11-09 |
      | Mallory Wisoky  | [email protected]  | Boynton Beach | 1927-12-09 |

Run only one scenario / skip scenario

In Playwright it's possible to focus / skip particular test with test.only / test.skip.
In Cucumber it's achieved with tags defined by user.
The idea is to add special handling of @only and @skip cucumber tags:

  • scenario with @only converted to test.only(...)
  • scenario with @skip converted to test.skip(...)

The same can be applied to Feature and Rule tags.
Related to #8.

Bug: duplicated text does not support

Given
What is your setup: paste your playwright config, feature file and directories structure

import { devices, defineConfig } from "@playwright/test";

/**
 * Read environment variables from file.
 * https://github.com/motdotla/dotenv
 */
import dotenv from "dotenv";
import { defineBddConfig } from "playwright-bdd";


dotenv.config({
	path: "./.env",
});
console.log("setting on env:"+ process.env.NODE_ENV);

dotenv.config({
	path: `./.env.${process.env.NODE_ENV}`,
});
dotenv.config({
	path: `./.env.${process.env.NODE_ENV}.local`,
});

console.log("public url is:", process.env.PUBLIC_URL);

const testDir = defineBddConfig({
	paths: ["./features"],
	importTestFrom: './steps/fixtures.ts',
	require: ['./steps/*.ts'],
	quotes: 'backtick' // when fixture filles are converted to spec.ts files, quotes will be replaced to the backticks
});

// 
export default defineConfig({
	testDir,
	timeout: 60 * 1000,
	expect: {
		/**
		 * Maximum time expect() should wait for the condition to be met.
		 * For example in `await expect(locator).toHaveText();`
		 */
		timeout: 10000,
	},
	/* Run tests in files in parallel */
	fullyParallel: true,
	/* Fail the build on CI if you accidentally left test.only in the source code. */
	forbidOnly: !!process.env.CI,
	/* Retry on CI only */
	retries: process.env.CI ? 2 : 0,
	/* Opt out of parallel tests on CI. */
	workers: process.env.CI ? 1 : undefined,
	reporter: [["html", { outputFolder: "./reports" }]],
	use: {
	  	actionTimeout: 0,
		/* Base URL to use in actions like `await page.goto('/')`. */
		baseURL: process.env.PUBLIC_URL ?? 'http://my-dev.dimensional.com',
		screenshot: 'only-on-failure',
		/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
		trace: "on-first-retry",
		testIdAttribute: "data-qa",
	},
	projects: [
		{
			name: "setup",
			testMatch: /\/__[a-zA-Z0-9].[a-zA-Z0-9_-]+\.feature\.spec\.js$/,
		},
		{
			name: "chromium",
			use: {
				...devices["Desktop Chrome"],
				// Use prepared auth state.
				storageState: ".auth/user.json",
			},
			testMatch: /\/[a-zA-Z0-9].[a-zA-Z0-9_-]+\.feature\.spec\.js$/,
			dependencies: ["setup"],
		},
		
	],
	/* Run your local dev server before starting the tests */
	...((!process.env.CI || process.env.CI === 'false') && process.env.NODE_ENV === 'development' ? {
			webServer: {
			   command: 'echo \"running dev server\";cd ../../src/cra;npm run start',
			   port: 3006,
			   reuseExistingServer: (!process.env.CI || process.env.CI === 'false')
			 },
		}: 
		{}
	)
});


When

npx bddgen

Then
I have 2 @When decorators which have the same text. in the other fixtures i.e.


export 
@Fixture<typeof test>("smaBuildModal")
class SmaBuildModal extends Common {
   @When("I add direct equity")
	async addDirectEquity() {
		await this.page.getByTestId("tab-item-strategyDesign-sma").click();
		await this.page.locator(".direct-equity-on-sma-modal").first().click({timeout: 3000});
	}
}

export 
@Fixture<typeof test>("modelBuildModal")
class ModelBuildModal extends Common {
	@When("I add direct equity")
	async addDirectEquity() {
                // different selectors from smaBuildModal
		await this.page.getByTestId("tab-item-strategyDesign-model").click();
		await this.page.locator(".direct-equity-on-model-modal").first().click({timeout: 3000});
	}
}

my fixture file:

@fixture:smaBuildModal 
Feature: SMA strategy design
    Scenario: Direct equity select and deselect
        When I add direct equity

my fixtures.ts file


type Pages = {
	smaBuildModal: SmaBuildModal;
	modelBuildModal: ModelBuildModal;
};

export const test = base.extend<Pages>({
	smaBuildModal: async ({ page }, use) => use(new SmaBuildModal(page)),
	modelBuildModal:async ({ page }, use) => use(new ModelBuildModal(page)),
});

Expected behavior
What did you expect?
Since they have to be compiled using different fixture class, both should be parsed properly.
when I didn't have the second fixture(ModelBuildModal), I could get below file. I am expecting to have similar file for the second fixture using modelBuildModal instead of smaBuildModal.

/** Generated from: features/sma-build/sma-create-new-sma/sma-direct-equity-only.feature */
import { test } from "../../../../steps/fixtures.ts";

test.describe(`SMA strategy design`, () => {

  test.beforeEach(async ({ Given, smaBuildModal }) => {
    await Given(`I am on new sma build modal`, null, { smaBuildModal });
  });

  test(`Direct equity select and deselect`, async ({ When, And, Then, smaBuildModal }) => {
    await When(`I select "strategy design" tab`, null, { smaBuildModal });
    await And(`I add direct equity`, null, { smaBuildModal });

Isolated demo
It's very helpful if you try to reproduce your behavior on the reference repo playwright-bdd-example. Feel free to make a pull request and post link here.

Environment
Please run npx bddgen env and post output here:

> [email protected] watch:bdd
> nodemon -w ./features -w ./steps -w ./step-utils -e feature,js,ts --exec 'npx bddgen'

[nodemon] 3.0.1
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): features/**/* steps/**/* step-utils/**/*
[nodemon] watching extensions: feature,js,ts
[nodemon] starting `npx bddgen`
setting on env:development
public url is: http://localhost:3006
ERROR: Several step definitions found for text: I add direct equity (features/sma-build/funding-transition.feature)
- I add direct equity
- I add direct equity
[nodemon] app crashed - waiting for file changes before starting...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.