component-driven / cypress-axe Goto Github PK

With Cypress 5.0 released, could this be added now to peerDependencies? I don't know if other changes would be needed.
"peerDependencies": { "cypress": "^3.1.1 || ^4.0.2" },

Is there another way to see the a11y failures than through the console? I'm finding it somewhat time consuming to identify the violating DOM node, it'd be neat if you could use regular Cypress functionality like highlighting the DOM node through the UI, etc.

Issues with the current API

• Multiple optional positional arguments in the checkA11y() method make it awkward to use and very difficult to extend. We have several open pull requests that are adding a new argument at the end.
• Requires too much setup to have violations summary either in the terminal or JSON that could be supplied to another too.
• Doesn't support Cypress chaining in the checkA11y() method, and uses a custom context argument. This makes the usage less idiomatic and doesn't allow integration with tools like Cypress Testing Library.
• No way to define defaults for the checkA11y() method, we have to pass things like violationCallback in every call or create a custom command.

Related issues: #40, #49, #62, #67, #68

New API proposal

1. Remove the context argument of the checkA11y() method, and rely on the Cypress chaining instead:

cy.checkA11y() // Check the whole document
cy.getByTestId('my-modal').checkA11y() // Check part of the document

2. Replace the rest of the arguments with an optional object of the following shape:

interface CypressAxeOptions {
  axeOptions?: axe.RunOptions, // This used to be `options` argument
  shouldFail?: (violations: RunResults[], results: RunResults[]) => boolean,  // Replaces `includedImpacts` and `skipFailures`
  reporters?: Reporter[]
}

type Reporter = (results: RunResults[]) => void

interface RunResults {
  filename: string, // Test file
  label?: string, // Label passed to checkA11y
  results: axe.Result[]
}

The defaults are going to be:

{
  axeOptions: undefined,
  shouldFail: (violations) => violations.length > 0,  // Fail on any number of violations
  reporters: [
   require('cypress-axe/cli-reporter')
  ]
}

Reporters are replacing the violationCallback: they are more flexible, have access to both passes and violations, and you could have global reporters that would run after all tests, or pass a reporter for a separate checkA11y() method call. The default reporter is printing an overall summary after all tests.

3. Add a new optional label to the checkA11y() method:

cy.checkA11y(options?: CypressAxeOptions, label?: string)

Similar to labels describe() or it() methods, to identify results of a particular call.

4. Add a new method to define defaults:

cy.configureCypressAxe(options: CypressAxeOptions)

It should accept the same object as the checkA11y() method, and set default values that could be overwritten in checkA11y().

I hope this covers all the use cases we have. I'm not sure that all the names are clear, so if you have any feedback, leave a comment ;-)

I'm not sure if anyone else if having this issue. However, when running cy.checkA11y three times, with the following params:

cy.checkA11y(null, null, null, {skipFailures: true});

it's causing the Cypress test to freeze all together with no failures

Getting this error when using 0.9.1 with axe-core 4.0.2 as a peer dependencies

ReferenceError: exports is not defined
      at axeFunction (eval at <anonymous> (http://172.25.25.197:3000/__cypress/tests?p=cypress/support/index.js), <anonymous>:24:14)
      at eval (eval at <anonymous> (http://172.25.25.197:3000/__cypress/tests?p=cypress/support/index.js), <anonymous>:27630:3)
      at eval (<anonymous>)

We are trying to use a custom matches for some of the axe rules to remove a few false positives that are currently being thrown. The documentation for axe-core states that a string with the relative path to the JS file containing the custom matches function must be used (https://github.com/dequelabs/axe-core/blob/develop/doc/developer-guide.md#matches-function).
The problem we are having is what is the path relative too as nothing we have tried has worked.

Also is there any way of importing the original matches function used by axe core ie (https://github.com/dequelabs/axe-core/blob/develop/lib/rules/color-contrast-matches.js)

Would it be possible to update the axe-core package to the latest version in order to include the larger rule set that came in with 4.1? Thank you.

Some webpack configurations make the result of require('axe-core') contain axe only under a default member variable, so window.axe becomes undefined here:

https://github.com/avanslaars/cypress-axe/pull/18/files#diff-168726dbe96b3ce427e7fedce31bb0bcR4

Also, @avanslaars, is there a reason you went back to window.eval(axe) in cf6d885#diff-168726dbe96b3ce427e7fedce31bb0bcR4?

Whenever I try to use cy.injectAxe() in a test, Cypress gets stuck at the step immediately prior. The Cypress test runner itself is locked and must be force-quit.

Here is code to reproduce the problem.

context('Problem with cy.injectAxe()', function () {

  beforeEach (function () {
    // These are not real credentials.
    // Please sign up (for free) at www.vidyard.com, verify your email address, then update these accordingly before running.
    const email = '[email protected]'
    const password = 'password123!'

    cy.visit('https://secure.vidyard.com/user/sign_in')
    cy.get('#username').type(email)
    cy.get('#password').type(password)
    cy.get('#sign-in').click()
    cy.wait(10000) // Just to make sure the page is fully loaded.
  })

  it('Accessibility Test', function () {
    // When the lines below are commented out, there is no issue.
    // When the lines are let to run, the Cypress Test Runner gets stuck on the cy.wait step.
    // Cypress will be completely frozen and must be force quit.
    cy.injectAxe()
    cy.checkA11y()
  })
})

I am using Cypress version 4.12.1 and cypress-axe version 0.8.1 on macOS 10.14.6 with Chrome 84.0.4147.105 and Firefox 79.0.

I need to disables assertions based on violations and only logs violations to a report, so was trying to pass 4 arguments to cy.checka11y() as described by @avanslaars , but got an error

0-2 arguments expected, got 4

package.json
"devDependencies": {
"@type/cypress-axe":"0.4.0"
"cypress-axe": "^0.8.1"
}

#https://github.com/avanslaars/cypress-axe

As mentioned here, the latest version of axe-core fixes a CSP issue where injectAxe() doesn't work with a strict content security policy that doesn't allow unsafe-eval for scripts.

I'll submit a PR for this.

Hi, is it possible to disable a rule via ID when running cy.checkA11y() on a page?

We're already aware of the color-contrast issue we have on our pages and until our site rebrand this will remain an a11y issue.

I've read the provided documentation and tried a few different variations (passing just an array through, removing [] brackets etc) of the below but the rule still appears to be checked:

        cy.injectAxe()
        cy.configureAxe({
            rules: [{ 'aria-hidden-focus': { enabled: false } }]
        })
        cy.checkA11y()

I have this in my support/index.ts

my tsconfig.json looks like this:

{
  "compilerOptions": {
    "lib": [
      "dom",
      "es2015",
      "es5",
      "es6",
      "es2015"
    ],
    "target": "es5",
    "module": "commonjs",
    "noImplicitAny": false,
    "strictNullChecks": true,
    "noImplicitThis": true,
    "alwaysStrict": true,
  },
  "files": [
    "./support/index.d.ts"
  ]
}

I have this test:

describe("TypeScript", () => {
  it("checks shape of an object", () => {
    cy.visit('http://localhost:6001')
    cy.injectAxe();
  });
})

It fails with

ReferenceError: exports is not defined
at axeFunction (eval at cy.window.then.window (http://localhost:6001/__cypress/tests?p=cypress/support/index.ts-538:111:12), :16:14)
at eval (eval at cy.window.then.window (http://localhost:6001/__cypress/tests?p=cypress/support/index.ts-538:111:12), :15287:3)
at eval ()

Why would this be?

I'm using cypress-axe to do some accessibility testing, but the errors I'm getting are not clear enough for me to act on... Is there a way to get more detailed information here?

  6 passing (23s)
  22 pending
  2 failing

  1) Accessibility Tests Global Component Assessments Desktop Assessments Petition Petition form should be accessible.:
     AssertionError: 1 accessibility violation was detected: expected 1 to equal 0
      at Function.fn.(anonymous function) [as equal] (http://localhost:8000/__cypress/runner/cypress_runner.js:86181:34)
      at getRet (http://localhost:8000/__cypress/runner/cypress_runner.js:89187:16)
      at tryCatcher (http://localhost:8000/__cypress/runner/cypress_runner.js:139407:23)
      at Function.Promise.attempt.Promise.try (http://localhost:8000/__cypress/runner/cypress_runner.js:136682:29)
      at Context.thenFn (http://localhost:8000/__cypress/runner/cypress_runner.js:89201:23)
      at Context.then (http://localhost:8000/__cypress/runner/cypress_runner.js:89525:21)
      at Context.<anonymous> (http://localhost:8000/__cypress/runner/cypress_runner.js:100860:21)
      at http://localhost:8000/__cypress/runner/cypress_runner.js:100381:33
      at tryCatcher (http://localhost:8000/__cypress/runner/cypress_runner.js:139407:23)
      at Promise._settlePromiseFromHandler (http://localhost:8000/__cypress/runner/cypress_runner.js:137343:31)
      at Promise._settlePromise (http://localhost:8000/__cypress/runner/cypress_runner.js:137400:18)
      at Promise._settlePromiseCtx (http://localhost:8000/__cypress/runner/cypress_runner.js:137437:10)
      at Async../node_modules/bluebird/js/release/async.js.Async._drainQueue (http://localhost:8000/__cypress/runner/cypress_runner.js:134137:12)
      at Async../node_modules/bluebird/js/release/async.js.Async._drainQueues (http://localhost:8000/__cypress/runner/cypress_runner.js:134142:10)
      at Async.drainQueues (http://localhost:8000/__cypress/runner/cypress_runner.js:134016:14)

  2) Accessibility Tests Global Component Assessments Desktop Assessments Petition Petition form should be accessible.:
     AssertionError: 1 accessibility violation was detected: expected 1 to equal 0
      at Function.fn.(anonymous function) [as equal] (http://localhost:8000/__cypress/runner/cypress_runner.js:86181:34)
      at getRet (http://localhost:8000/__cypress/runner/cypress_runner.js:89187:16)
      at tryCatcher (http://localhost:8000/__cypress/runner/cypress_runner.js:139407:23)
      at Function.Promise.attempt.Promise.try (http://localhost:8000/__cypress/runner/cypress_runner.js:136682:29)
      at Context.thenFn (http://localhost:8000/__cypress/runner/cypress_runner.js:89201:23)
      at Context.then (http://localhost:8000/__cypress/runner/cypress_runner.js:89525:21)
      at Context.<anonymous> (http://localhost:8000/__cypress/runner/cypress_runner.js:100860:21)
      at http://localhost:8000/__cypress/runner/cypress_runner.js:100381:33
      at tryCatcher (http://localhost:8000/__cypress/runner/cypress_runner.js:139407:23)
      at Promise._settlePromiseFromHandler (http://localhost:8000/__cypress/runner/cypress_runner.js:137343:31)
      at Promise._settlePromise (http://localhost:8000/__cypress/runner/cypress_runner.js:137400:18)
      at Promise._settlePromiseCtx (http://localhost:8000/__cypress/runner/cypress_runner.js:137437:10)
      at Async../node_modules/bluebird/js/release/async.js.Async._drainQueue (http://localhost:8000/__cypress/runner/cypress_runner.js:134137:12)
      at Async../node_modules/bluebird/js/release/async.js.Async._drainQueues (http://localhost:8000/__cypress/runner/cypress_runner.js:134142:10)
      at Async.drainQueues (http://localhost:8000/__cypress/runner/cypress_runner.js:134016:14)

The test code is just a checka11y() call:

[...]

      context("Petition", () => {
        before(() => {
          cy.viewport('macbook-13');
          cy.visit(`/campaigns/single-page/`);
          cy.wait(500);
          cy.injectAxe().configureAxe({
            reporter: "v2"
          });
          cy.get(".sign-petition");
        });

        it("Petition content should be accessible.", () => {
          cy.checkA11y(".sign-petition .petition-content"); // passes
        });

        it("Petition form should be accessible.", () => {
          cy.checkA11y(".sign-petition form"); // generates those 2 errorrs
        });

[...]

And I updated cypress' /plugins/index.js to include the task logging:

module.exports = (on, config) => {
  on("task", {
    log(message) {
      console.log(message);
      return null;
    },
    table(message) {
      console.table(message);
      return null;
    }
  });
};

But I don't see anything in the console that lets me figure out what is actually failing and how to fix it =S

Hello there,

Im using cypress with typescript and when i set the variable:

const A11yConfig: Options = {
  includedImpacts: ["critical"],
  runOnly: {
    type: "tag",
    values: ["section508"]
  }
}; 

Then, i use the cy.checkA11y(A11yConfig), when i run the tests an error happens:

axe.run arguments are invalid

Am i doing something wrong?

Thanks!

Versions:
"cypress": "^4.12.1"
"@types/cypress-axe": "^0.8.0"
"axe-core": "^3.5.5"

axe-core has rules provided at https://github.com/dequelabs/axe-core/tree/develop/lib/rules
but could not find an example custom json to append custom rules. What I want to do is to manually test a site using Trusted Tester Test Process, and add the test to the custom list of tests with expected / actual type assertions, so as the custom tests grow in size the amount of time needed to test against trusted tester tests, manually is reduced.

I'm seeing the following error when running cy.injectAxe in my Cypress tests:

Timed out retrying after 4000ms: cy.readFile("0764") failed because the file does not exist at the following path:

/home/megan/voicethread/code/frontend/wizard/0764

Because this error occurred during a before each hook we are skipping the remaining tests in the current suite

node_modules/cypress-axe/dist/index.js:16:8
  14 | exports.configureAxe = exports.injectAxe = void 0;
  15 | exports.injectAxe = function () {
> 16 |     cy.readFile(require.resolve('axe-core/axe.min.js')).then(function (source) {
     |        ^
  17 |         return cy.window({ log: false }).then(function (window) {
  18 |             window.eval(source);
  19 |         }); 

require.resolve('axe-core/axe.min.js') seems to evaluate to "0764" instead of the intended JS.

This issue seems to be connected to Cypress's Webpack Preprocessor, which I'm using to handle importing files with absolute path aliases.

const webpack = require('@cypress/webpack-preprocessor')

module.exports = (on, config) => {
  on('file:preprocessor', webpack({
    webpackOptions: require('@vue/cli-service/webpack.config'),
    watchOptions: {},

    // Add the ability to use aliases like @shared
    resolve: {
      alias: require('../../../../aliases.config').webpack
    }
  }))

  return Object.assign({}, config, {
    fixturesFolder: 'tests/e2e/fixtures',
    integrationFolder: 'tests/e2e/specs',
    screenshotsFolder: 'tests/e2e/screenshots',
    videosFolder: 'tests/e2e/videos',
    supportFile: 'tests/e2e/support/index.js'
  })
}

When I remove the on('file:preprocessor') listener from my plugins.js, file, the cy.injectAxe command succeeds.

Node version: 12.13.1
Cypress version: 6.3.0
@cypress/webpack-preprocessor version: 5.5.0

With Cypress 4.0 released, could this be added now to peerDeps? I don't know if other changes would be needed. Thanks for allowing us to test accessibility with cypress!

Causes eslint error "Unable to resolve path to module cypress-axe."

workaround is to use 0.8.1

Build script seems to work, but what comes from npm -i to node_modules is indeed missing the whole dist directory.

suddenly getting this error with no changes to my code. Using version 0.10.0:

have tried deleting dist and node_modules folder, running npm run update and built new dist

Module not found: Error: Can't resolve 'axe-core/axe.min.js' in /path/node_modules/cypress-axe/dist

I have a modal that overlays the screen:

(note backdrop darkens)

I'm testing for a11y like so: cy.checkA11y(".modal", null, terminalLog);

But always get this contrast error:

failureSummary: "Fix any of the following:
  Element has insufficient color contrast of 2.92 (foreground color: #979797, background color: #ffffff, font size: 13.5pt (18px), font weight: normal). Expected contrast ratio of 4.5:1"

The font size is 18px, the text is #000 and the background is #FFF. I'm not sure how it's getting this #979797 value. This bug only happens when I am checking my modal element for a11y, so I think something is going wrong when I pass in the element I want to check.

My test file:

context('app', () => {
  beforeEach(() => {
    cy.server();
    cy.route('GET', '/api/exercises/core/exercises/*', 'fixture:exercises');
    cy.visit('/exercises/3bc94905-4b8a-4d67-815e-0652afaf22c2');
  });

  it('has no a11y violations', () => {
    cy.injectAxe();
    cy.checkA11y();
  });
});

the output (note the input field shown in element explorer)

A11Y ERROR! landmark-one-main on 1 Node
A11Y ERROR! page-has-heading-one on 1 Node

shows nothing about missing label on input, am i doing something wrong? ive tried adding arguments to the checkA11y call to make it more specific but then i get no errors at all

Related: #22, vercel/next.js#7945

Full repo: here (note: some of the packages there are not yet published, but the test can run without them)

I have a simple test:

import formatAxeLog from "../helpers/formatAxeLog";

describe("Home page", () => {
  beforeEach(() => {
    cy.visit("/");
  });
  it("Has no detectable a11y violations on load", () => {
    cy.injectAxe();
    // cy.wait(500);
    cy.checkA11y(null, null, formatAxeLog);
  });
});

When I run the tests multiple times they sometimes fail (due to vercel/next.js#7945, as they should), sometimes pass. The only "reliable" way is just to wait. Am I doing something wrong, or this is a bug?

Firstly, cypress-axe is brilliant! Previously I was using axe-core with webdriverio, and it was painfully slow! However, one frustration with cypress-axe is the use of node assert. Elsewhere in our tests I'm using the chai library bundled into Cypress, which I have also extended to provide soft assertions (ie, continue on assertion fail) based on code found here:https://stackoverflow.com/questions/55868107/does-cypress-support-soft-assertion

I have modified cypress-axe locally to use chai.softExpect, and it behaves as I'd like. this allows me to run multiple a11y checks against different pages / page sections in the same test.

Might it be possible to have some sort of configuration option that allowed the user to specify which assertion library / method to use? Or something similar?

Hi Guys,

Is it possible to report/log accessibility violations only and not making our tests to fail?

Patrick

I see a couple of issues opened to upgrade axe core version, which is bundled in this library. (#36, #58)

To avoid this type of request in the future, I think we can make this awesome library more flexible by allowing users to use any axe core version.

I'm proposing to add a parameter to the injectAxe method, so users can specify their axe core source/version to overwrite the default version.

Sample scenario: Assuming the user wants to use the latest axe core

1. Install the latest axe-core

npm install -D axe-core@latest

2. Load axe.min.js and pass it into the injectAxe method

cy.readFile('node_modules/axe-core/axe.min.js', 'utf-8').then((content) => {
    cy.injectAxe(content);
    cy.checkA11y();
})

Hi All,
I am trying to setup Ally in my project but I am getting below error running an initial/simple test:
Do you know where is the issue? is due to Cypress 5? it seems that it is supported based on the documentation.Do you know which with versions runs properly?

Tere you have my configuration and script:

I'm working on a component library so my focus in a11y testing of the components rather than the test harness they're loaded in. In protractor we can do something like:

browser.runAxeTest('Breadcrumbs', 'lib-breadcrumbs');

Looking at some of the discussion here, it seems like axe-core allows specificity.

It looks like it might be a simple passing of a selector as a parameter to the cy.checkA11y(); to propagate to axe API.

Does this seem like it would be a good approach? Or would using axe directly be preferable?

Hey there. I am experience what I think is a strange issue. I am testing out the accessibility on my website built with Gatsby (React.js). I have Cypress set up as per their docs with this library and axe-core. When I run the the following I get no errors on a page that has a footer which I know is rendering links that will fail the color contrast test. I know this because I have react-axe installed as well and it is reporting the color contrast problem in the console.
However because react-axe does not lead to a nice automation testing work flow I would like to use cypress to find these color contrast issues.

What could be causing cypress-axe / axe-core not picking it up with testing but react-axe picks it up as expected?

Sample test

describe('Accessibility tests', () => {
    beforeEach(() => {
      cy.visit('http://localhost:8000')
      cy.injectAxe()
    })
    it('Has no detectable accessibility violations on load', () => {
      cy.wait(1000)
      cy.checkA11y()
    })
  })

Hey!
In some cases, the full axe-core results object is needed (for logging / tracking or other uses)
It would be helpful to have another callback that returns the full results object (similar to how violationsCallback returns the violations object).

I submitted a PR for this - please consider it :)
#68

I am creating a new issue for #7 as the original owner closed it despite the issue not being resolved. If you are having issues getting cypress-axe to work with Typescript/Webpack, please look at that issue for solutions.

I am using these versions:

    "axe-core": "4.0.1",
    "cypress": "4.11.0",
    "cypress-axe": "0.8.1",

Using this in my cypress test:

    cy.injectAxe();
    cy.checkA11y({
      exclude: [
        [['#onetrust-consent-sdk']]
      ]
    });

But getting these error:

SyntaxError
Failed to execute 'querySelectorAll' on 'Document': '[object Object]' is not a valid selector.

It seems like the first param for checkA11y should not be an object, yet looking at the documentation and examples here it should work:

• https://www.deque.com/axe/core-documentation/api-documentation/#context-parameter
• https://github.com/dequelabs/axe-core/blob/19cf29f5c59d79819a39cc51502b33c439111953/test/integration/full/context/context.js#L17

Any ideas? My goal is to ignore a11y checks for the div specified by an ID.

Hi,

Great project, thanks!

Just been trying to get this running with a Cypress project in TypeScript but came across some issues, so I am proposing some readme updates for typescript to state that typings are available here

https://www.npmjs.com/package/@types/cypress-axe

https://github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/cypress-axe

And that it won't work with webpack, but will work with browserify pre-processors

Related issues

#6

dequelabs/axe-core#1406

dequelabs/axe-core#1427

Cypress Typescript users can switch from webpack, to browerify

https://github.com/cypress-io/cypress-browserify-preprocessor

Happy to make the readme changes if required 👍

cy.checkA11y( ".example-class" ) runs the rule on whole page. I want to do a11y testing only one specific portion of the page. Do I need to pass any other configuration?

With 0.8.1 even after you add the import command to support/index.js the axe commands are not detected.
cy.injectAxe()
cy.injectAxe()
cy.checkA11y()

Those commands are not detected by cypress..

Hi,

I noticed that checkA11y failed when called after using the cy.clock(...) function to set a date

I get the following error when trying to call checkA11y :

cy.then() timed out after waiting 4000ms.

Your callback function returned a promise that never resolved.

The callback function was:

function(a){i(e)&&(e=void 0),i(t)&&(t=void 0),i(n)&&(n=void 0);var o=t||{},s=o.includedImpacts,u=r(o,["includedImpacts"]);return a.axe.run(e||a.document,u).then((function(e){var t=e.violations;return s&&Array.isArray(s)&&Boolean(s.length)?t.filter((function(e){return s.includes(e.impact)})):t}))}

Running cypress-axe 0.8.1 and cypress 4.12.1

Hi,

I am trying to follow the instructions as documented.
When doing cypress open, I have the following error:

ReferenceError: t is not defined

This is happening when trying to eval(axe.source).
It appears that axe is not defined inside the eval.

I am not sure why.

Update the axe-core package to version <= 3.5.0 to include the 3.5 ruleset

Hi,

is it possible to configure cypress-axe, so it fails tests only if there, for example, critical violations?

I started using cypress-axe to catch a11y issues when running end to end tests automatically, but I am finding it challenging to get the tests to pass. I get two errors which I believe are false positives:

• page-has-heading-one
• landmark-one-main

I use react-axe, which prints a11y issues to browser console in development mode, but I don't see these two issues reported by that package. I tried auditing using Axe Chrome extension, but I do not see these.

I would be happy to provide more information, but at this point, I believe it to be a bug with cypress-axe.

Hi, I am currently using cypress-axe and I'm experiencing the same issues that were identified in this issue that's been closed:
#22

I left a comment there but didn't get a response and I tried implementing the other attempts on that issue board. I was just hoping to get a clear understanding of what the fix was or if there was a fix/workaround at all. Below are the versions I'm using for Cypress:

Cypress version: 6.1
cypress-axe version: 0.12.0

I am also using cypress-axe in conjunction with Storybook: 5.3.21.

In https://github.com/avanslaars/cypress-axe/blob/master/src/index.js#L3, I see that the file is directly loaded from node_modules using fs
I see that axe-core exposes a source parameter. Can that be used instead?
i.e.

import axe from 'axe-core';
export const injectAxe = () => {
  cy.window({ log: false }).then(window => {
    window.eval(axe.source)
  })
}

Is it a concern if it is unminified?

Hello,

Love the product! This is a great idea and way to manage a11y development, creating a much tighter feedback loop.

I wanted to request support for configuring axe rule enforcement - some rules are inappropriately enforced in certain scenarios, leading to false positives. Ideally in cypress/support/index.js a config file could be passed in, leveraging axe.configure().

This may already be possible! If so, just some simple docs with an example would suffice.

Still after reading axe-core docs cannot for the life of me figure out how to run a single rule, for example how to run just "color-contrast" rule against text on the page...

cy.readFile() must be passed a non-empty string as its 1st argument. You passed: 146.Learn more

This happens after finishing setup and trying to run a sample test.

This is in my command that I use to login:

    .then((url) => {
      cy.intercept('api/users/user/profile/').as('login');
      cy.visit(url);
      cy.wait('@login');
      cy.injectAxe();
    }),

Error happens on the cy.injectAxe() line.

Thanks!

Hi,

Is there a way to disable certain axe rules? We're looking to run it through CI and gradually update accessibility errors as we go then enable the rules once they're updated.

Is there a way to do this?

Thanks

Thank you very much for contributing cypress-axe to the community!

I noticed that around 0.7.0 it was decided to bundle axe-core and I guess I'm wondering if would it might be better to keep axe-core as a peerDependency so that consumers can exert some level of control over their version? I notice that there is already a request (#36) to upgrade the bundled version.

Reasons why I ask:

1. Maybe it's just a recent phenomenon, but I notice that Deque has had a few minor/patch releases to fine-tune the aggressiveness of the axe ruleset. In my application I've had to upgrade a few times (to 3.4, to 3.5) to get around "overly aggressive" (my words, not theirs) errors that were not failing in other a11y checkers.

2. I could be doing it wrong ^TM (and therefore it may not be a valuable use case), but in our application we currently also use axe-core directly in some of our faster JSDOM tests that are not covered by our slightly slower full-browser cypress tests. While not maybe technically necessary, it would be nice to have the a single axe-core version/ruleset apply.

Any thoughts are appreciated. Thank you again for creating this useful library.

Having trouble running tests and get the following error:

cypress_runner.js:157775 TypeError: Cannot read property 'run' of undefined
    at Context.<anonymous> (https://qa1.domain.com/__cypress/tests?p=cypress\support\index.js-350:18962:22)
From previous event:
    at Context.thenFn (https://qa1.domain.com/__cypress/runner/cypress_runner.js:71909:23)
    at Context.then (https://qa1.domain.com/__cypress/runner/cypress_runner.js:72229:21)
    at Context.<anonymous> (https://qa1.domain.com/__cypress/runner/cypress_runner.js:80256:21)
    at https://qa1.domain.com/__cypress/runner/cypress_runner.js:79783:33
From previous event:
    at runCommand (https://qa1.domain.com/__cypress/runner/cypress_runner.js:79765:14)
    at next (https://qa1.domain.com/__cypress/runner/cypress_runner.js:79900:14)
From previous event:
    at next (https://qa1.domain.com/__cypress/runner/cypress_runner.js:79900:34)
From previous event:
    at https://qa1.domain.com/__cypress/runner/cypress_runner.js:79937:37
From previous event:
    at run (https://qa1.domain.com/__cypress/runner/cypress_runner.js:79929:15)
    at Object.cy.<computed> [as percySnapshot] (https://qa1.domain.com/__cypress/runner/cypress_runner.js:80290:11)
    at Context.runnable.fn (https://qa1.domain.com/__cypress/runner/cypress_runner.js:80477:20)
    at callFn (https://qa1.domain.com/__cypress/runner/cypress_runner.js:30713:21)
    at Test.../driver/node_modules/mocha/lib/runnable.js.Runnable.run (https://qa1.domain.com/__cypress/runner/cypress_runner.js:30706:7)
    at https://qa1.domain.com/__cypress/runner/cypress_runner.js:84151:28
From previous event:
    at Object.onRunnableRun (https://qa1.domain.com/__cypress/runner/cypress_runner.js:84140:17)
    at $Cypress.action (https://qa1.domain.com/__cypress/runner/cypress_runner.js:78413:30)
    at Test.Runnable.run (https://qa1.domain.com/__cypress/runner/cypress_runner.js:82943:20)
    at Runner.../driver/node_modules/mocha/lib/runner.js.Runner.runTest (https://qa1.domain.com/__cypress/runner/cypress_runner.js:31180:10)
    at https://qa1.domain.com/__cypress/runner/cypress_runner.js:31286:12
    at next (https://qa1.domain.com/__cypress/runner/cypress_runner.js:31100:14)
    at https://qa1.domain.com/__cypress/runner/cypress_runner.js:31110:7
    at next (https://qa1.domain.com/__cypress/runner/cypress_runner.js:31042:14)
    at https://qa1.domain.com/__cypress/runner/cypress_runner.js:31073:7
    at next (https://qa1.domain.com/__cypress/runner/cypress_runner.js:84079:16)
    at https://qa1.domain.com/__cypress/runner/cypress_runner.js:84097:11
From previous event:
    at onNext (https://qa1.domain.com/__cypress/runner/cypress_runner.js:84094:57)
    at done (https://qa1.domain.com/__cypress/runner/cypress_runner.js:30674:5)
    at https://qa1.domain.com/__cypress/runner/cypress_runner.js:30718:11
From previous event:
    at callFn (https://qa1.domain.com/__cypress/runner/cypress_runner.js:30717:10)
    at Hook.../driver/node_modules/mocha/lib/runnable.js.Runnable.run (https://qa1.domain.com/__cypress/runner/cypress_runner.js:30706:7)
    at https://qa1.domain.com/__cypress/runner/cypress_runner.js:84151:28
From previous event:
    at Object.onRunnableRun (https://qa1.domain.com/__cypress/runner/cypress_runner.js:84140:17)
    at $Cypress.action (https://qa1.domain.com/__cypress/runner/cypress_runner.js:78413:30)
    at Hook.Runnable.run (https://qa1.domain.com/__cypress/runner/cypress_runner.js:82943:20)
    at next (https://qa1.domain.com/__cypress/runner/cypress_runner.js:31056:10)
    at https://qa1.domain.com/__cypress/runner/cypress_runner.js:31078:5
    at timeslice (https://qa1.domain.com/__cypress/runner/cypress_runner.js:26146:27)

My package:

{
	"devDependencies": {
		"@cypress/code-coverage": "^1.8.0",
		"@percy/cypress": "^2.0.0",
		"axe-core": "^3.3.1",
		"concurrently": "^4.1.1",
		"cypress": "^3.4.1",
		"cypress-axe": "^0.5.1",
		"fs-extra": "^8.1.0",
		"istanbul-lib-coverage": "^2.0.5",
		"mocha": "^5.2.0",
		"mocha-multi-reporters": "^1.1.7",
		"mochawesome": "^4.1.0",
		"mochawesome-merge": "^2.0.1",
		"mochawesome-report-generator": "^4.0.1",
		"nyc": "^14.1.1"
	},
	"dependencies": {
		"ls": "^0.2.1",
		"rimraf": "^2.6.3",
		"yargs": "^13.3.0"
	}
}

My test:

describe('APPB', function () {
	context('[Login] Login into account', function () {
		beforeEach(function(){
		  cy.visit('/login')
		  cy.injectAxe()
		})
		it('[Login] Redirects to /dashboard on success', function () {
			cy.percySnapshot('testing before')
			
			cy.get('input[name=username]').should('have.attr', 'placeholder', 'Username or Email address')
				.type('[email protected]', { delay: 50 }).should('have.value', '[email protected]')
			cy.get('input[name=password]').should('have.attr', 'placeholder', 'Password')
				.type('12345678', { delay: 50 }).should('have.value', '12345678')
			
			cy.percySnapshot('testing after')
			
			cy.get('button.button.primary').contains('Sign in').click() 
			cy.checkA11y()
		})
	})
})

Any ideas or suggestions on what could be wrong with my setup and or test?

First of all: Nice project, great job ;-)

Is there a reason why there is no license?