paypal / aatt Goto Github PK
View Code? Open in Web Editor NEWAutomated Accessibility Testing Tool
License: BSD 3-Clause "New" or "Revised" License
Automated Accessibility Testing Tool
License: BSD 3-Clause "New" or "Revised" License
I get the mentioned error.
The selenium test runs for a long time about 5 mins and then it returns.
java.net.SocketException: Unexpected end of file from server.
Also the terminal log shows the ENGINE statement twice.
Can you please look into it and lmk what I am missing?
code:
String request = "http://localhost:3000/evaluate";
String source = URLEncoder.encode(driver().findElement(By.cssSelector(".formPanel")).getAttribute("outerHTML"), "UTF-8");
String urlParameters = "source=" + source + "&engine=axe&output=html&errLevel=1,2,3&level=WCAG2AA";
After reading through the AATT ReadMe.md it looks like the tool supports just the WCAG2.0 and WCAG2.1 hasn't been mentioned in it. So, can I consider AATT doesn't support WCAG2.1. If so, is there any plan for upgrading the libraries?
Q2. I've tried out the AATT integration API and it returned success status. The API consumes just the HTML content for the source
parameter and not CSS. my question is: isn't it supposed to validate both HTML and CSS in order to test with the WC guidelines?
Thank you
When using the axe engine to examine an internal webpage a problem with one of the lists is identified. When the output string is rendered in the HTML the help text is not escaped so it is not displayed correctly. The text should read <ul> and <ol> must only directly contain <li>, <script> or <template> elements
Attached is a text file containing both the JSON and HTML versions of the output.
error file.txt
I'm not sure if this is you guys or the axe project.
I am trying to test a web page, but I am getting an error.
When I run Page URL testing with Html Code Sniffer it returns no results:
When I run Page URL testing with Axe it returns an error:
When I try and example page like google everything works.
Please help.
EDIT: Tried other Angular web pages.....
Edited:2/14/2018
Hi,
I am using evaluate end for automate testing. While a dozens of other pages like google, etc works. All our pages received the error likes below. It seems to be a AATT error, but I was unable to catch it in the app.js.
Thanks
Error: spawn E2BIG
at exports._errnoException (util.js:1026:11)
at ChildProcess.spawn (internal/child_process.js:317:11)
at exports.spawn (child_process.js:471:9)
at Object.exports.execFile (child_process.js:190:15)
at /AATT/app.js:220:17
at Layer.handle [as handle_request] (/AATT/node_modules/express/lib/router/layer.js:82:5)
at next (/AATT/node_modules/express/lib/router/route.js: 100:13)
at Route.dispatch (/AATT/node_modules/express/lib/router/route.js:81:3)
at Layer.handle [as handle_request] (/AATT/node_modules/express/lib/router/layer.js:82:5)
at /AATT/node_modules/express/lib/router/index.js:233:24
Would be great to update the version of axe. There have been a lot of improvements.
Can this tool also perform crawl-based accessibility analysis, analysis of authenticated flows (sign-in), and be integrated into the CI CD pipeline?
When running AATT test tool, in Section508 mode, against the application I am getting, the following HTML code snippet is throwing an error:
Snippet:
<a href="/application/some.action.do" accesskey="q" aria-label="Do Something Quickly" role="menuitem"> ...</a>
Message:
Anchor element found with a valid href attribute, but no link content has been supplied.
Technique: H91
Rule: Section,1194.22 (l)
According to our development team, this is a "false positive", as the aria-label is supplying the necessary accessible name.
That leads me to ask: Is the above error (in this condition) a true Section 508 failure, or is it a false positive?
For the purpose of the tests that I am running, I can "bypass" this failure by modifying the code locally to exclude the failure condition (at least when aria-label is either not present or does not have a value).
Thoughts?
Hi, I've been using AATT via the web API, it is very useful! I think it would be great if you could also use this as a standard node module instead, e.g.:
import { evaluate } from '@paypal/aatt';
evaluate({ source: '<html />', output: 'json' }).then(results => {
console.log(results);
});
Is this on the road map? I'm happy to put in a PR for this if not
Elements with role="presentation" should not be, in my opinion, checked for contrast ratio issues. By definition they are completely presentational, should be omitted by screen readers, don't contain any valuable content.
Hi,
I tried to use aatt node module like below:
const content = await testPage.content();
const result = await evaluate({
source: content,
output: "html",
engine: "axe",
level: "WCAG2A"
});
await fs.writeFile('./aatt-report.html', result,{}, (err)=>{
if (err) throw err;
});
testPage is object Page from Playwright framework.
Current results for each engine:
Could please tell what is wrong?
Versions:
"playwright": "^1.12.3",
"aatt": "^1.0.1",
Thanks!
This is more towards the suggestion and not an issue, are there any plans to upgrade to WCAG 2.1
starting at revision 031d856 Various fixes for being module based rather than running in same folder
when test from http://localhost:3000/test/AATT_API.html
OR posting with python
requests.post('http://localhost:3000/evaluate', stream=True, verify=False, json={"source": htmlC1})
I got error: Unable to load the address!
Is there anything I have to do differently ?? thanks !!
I did a test install on a Ubuntu Gnome 16.04 VM and all worked fine
Now the same install on a headless Ubuntu 16.04 server looked to be working well but now with any test, the loading overlay pops up then disappears and no results are displayed.
Output from AATT
AATT:log Express started on port 3000 +0ms
AATT:log E N G I N E htmlcs +11s
AATT:log +15ms
AATT:log E N G I N E chrome +3s
AATT:log +4ms
AATT:log E N G I N E chrome +4s
AATT:log +3ms
AATT:log E N G I N E chrome +4s
AATT:log +3ms
AATT:log E N G I N E axe +6s
AATT:log +4ms
I am trying to automate accessibility testing. I tried using AATT web application and also AATT with nemo-accessibility but I am getting different results from both. Is it expected to get different results? If yes, can you explain why?
I saw they are hitting different api end points /sniffUrl and /evaluate but I am not able to understand why I am getting different results because both are used to test accessibility.
Hi Team,
I saw this tool and I am impressed on this. Just few questions
Thanks in advance.
Looking to create a docker file from this and would like to link to a specific release rather than master branch.
Can you begin to create releases in GitHub to mark when things have changed in order for us to track them within the dockerfile?
Thanks.
When running DEBUG=AATT* http_port=3000 node app.js
I get the following error:
fs.unlink(tempFilename, (err) => {
^^
SyntaxError: Unexpected token =>
at exports.runInThisContext (vm.js:73:16)
at Module._compile (module.js:443:25)
at Object.Module._extensions..js (module.js:478:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Function.Module.runMain (module.js:501:10)
at startup (node.js:129:16)
at node.js:814:3
Hi,
when i run $ DEBUG=AATT* http_port=3000 node app.js
i get this error
Error: Cannot find module 'express'
at Function.Module._resolveFilename (module.js:469:15)
at Function.Module._load (module.js:417:25)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at Object.<anonymous> (/home/user/AATT/app.js:2:15)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)
I have set up an AATT instance in the intranet. Due to security reason, we have to use proxy to access internet pages and I cannot find a way to provide the proxy to AATT.
I noticed a proxy setting in config.json, but failed to find an example for this config. What is more, I also need to enable exclusion in proxy setting for internal URLs.
Hello.
In using the tool in its default form, I have run into an issue when evaluating webpages using the "Section508" level.
If I submit a simple webpage using Source as shown below,, HTML code as shown, AATT endpoint evaluate never responds:
<html> <head> <body></body> </html>
However, if a table is added to the body (as shown), I do get a response:
<html> <head> <body><table></table></body> </html>
Is there a reason I'm not getting a response for the HTML that has a lack of a table? If I change level to something else (like WCAG2AA), the call works. Why is this?
Parameters being used (from body in postman)
source: (what is shown above)
engine and output: htmlcs
errLevel: 1,2,3 (although I have also tried 1)
level: Section508
(from Headers in postman)
Content-Type: application/x-www-form-urlencoded
engine: htmlcs
I am doing POC with AATT for my organization's accessibility needs. I tested bunch of websites and most of the time Chrome is not working on the sites. Also Axe is not working for chase.com, but when I tried manually Axe add-on on the site (not from the AATT tool), please suggest what should be done.
Axe, Chrome and html Code Sniffer works fine.
Axe and html Code Sniffer works fine.
Chrome doesn’t work, it gets stuck on “Loading Please Wait..” the experience is same in Chrome and firefox browser
Axe - Error “Mismatched anonymous define() module: function () {"use strict";return axe;} http://requirejs.org/docs/errors.html#mismatch”
Chrome - doesn’t work, it gets stuck on “Loading Please Wait..” the experience is same in Chrome and firefox browser
Html - doesn’t show any error
Axe - worked fine
Chrome - doesn’t work, it gets stuck on “Loading Please Wait..” the experience is same in Chrome and firefox browser
Html - doesn’t show any error
Axe - worked fine
Chrome - doesn’t work, it gets stuck on “Loading Please Wait..” the experience is same in Chrome and firefox browser
Html - doesn’t show any error
Axe - worked fine
Html - doesn’t show any error
Chrome - TypeError: JSON.stringify cannot serialize cyclic structures. https://www.apple.com/metrics/ac-analytics/2.2/scripts/ac-analytics.js:870 in stringify
We are trying to understand if there is an advantage to using axe over htmlcs but are having issues. The axe results are not returning code snippets and in some cases we're getting "undefined" where htmlcs generates results. Is there additional config we should be doing in order to use axe?
Is there documentation about how to use AATT to crawl a domain?
I tested on Mac and Debian. Did I forgot to setup anything?
AATT:log +9s
AATT:log E N G I N E axe +6s
AATT:log undefined
AATT:log +754ms
AATT:log E N G I N E axe +39m
AATT:log undefined
AATT:log +783ms
AATT:log E N G I N E axe +30m
AATT:log undefined
AATT:log +716ms
When doing git submodule update, I get the following error:
...
fatal: reference is not a tree: 02ea409a140269b32d6a8c0466304cf812e5e209
Unable to checkout '02ea409a140269b32d6a8c0466304cf812e5e209' in submodule path
'src/htmlcs'
...
This occurs on both linux and windows using node 4.2.2,
The app runs and I get the initial "express log started" message,. The html page comes up on localhost: but when I submit a url for testing, I get the "if this message appears for more than 50 seconds click here" and nothing else; the test never completes and nothing shows up in the log.
Thanx for any info you can provide.
Line 222 of app.js
This change was made on July 31. Is there a reason why this is being forced?
Not an issue, just a minor typo: Run a search for the word reqults
in the README. It would be nice to fix that typo in your next push maybe :)
Usecase: Those who (We currently) don't embed CSS within the HTML(angular js app), with that the page won't properly render with just HTML - while it shows as a plain text without CSS. It looks like the API is validating that plain text. So, if there's a way to supply the endpoint with the capability of reading CSS within. that would be amazing.
Thanks
Hi,
Could you please add a parameter to API request to retrieve warning messages? We can currently get warning from web UI, but not from API
Thanks for great this work.
Actually i have installed Node js and git utility, what is next action for start testing?
I am trying to research on this tool and I am facing an issue where running the htmlcs through the UI shows me 7 errors but when I use the API and pass the same html code, it only shows me 2 errors
below is my html
<script type="text/javascript"> function updateProfile(){ var xmlhttp = new XMLHttpRequest(); xmlhttp.open("POST","http://localhost:3000/evaluate", true); xmlhttp.setRequestHeader("Content-type","application/x-www-form-urlencoded"); var generatedSource = new XMLSerializer().serializeToString(this.document); xmlhttp.send("source=" + generatedSource + "&output="+"json"); } </script> First Name;<label>Last Name;</label><br />
<input type='text' name='lastName' />
<label>Email</label><br />
<input type='text' name='email' />
<input type='submit' onClick='updateProfile()' />
</form>
What am I doing wrong?
The license for this project still references the eBay Software Foundation. As part of the split earlier this year, copyright for this project was assigned from the eBay Software Foundation to PayPal.
The application fails to respond if you send it some simple HTML source with no accessibility issues, and ask for json
as the output type.
Example - validating this HTML:
<html lang="en"><head><title>Hello</title></head><body><h1>Heading</h1></body></html>
Using this command:
curl --data 'source=%3Chtml%20lang%3D%22en%22%3E%3Chead%3E%3Ctitle%3EHello%3C%2Ftitle%3E%3C%2Fhead%3E%3Cbody%3E%3Ch1%3EHeading%3C%2Fh1%3E%3C%2Fbody%3E%3C%2Fhtml%3E&output=json' http://localhost:3000/evaluate
No response is returned.
I've made a simple proposed fix here: https://github.com/petenattress/AATT/tree/json-bugfix - I can raise a pull request if you like.
Hello,
Currently, Checkbox for Screenshots, options like Show Failures, warnings, notices are shown below the Test page button. It would be nice to have them before Test page / code button so that while submitting URL / Code itself user can select required options. This would be useful to keyboard users who rely on screen readers.
Getting (node:1496) DeprecationWarning: Calling an asynchronous function without callback is deprecated, when using AATT API for accessibility evaluation. I am using node v7.4.0.
I had originally posted about this issue over on the nemo-accessibility issue list, but as @nikulkarni pointed out on that thread, it actually seems to be an AATT issue, so I am reposting over here.
My team would like to use AATT to automate our accessibility testing. I am trying to evaluate some markup using the 'Test HTML' tab on AATT. From what I understand, nemo-accessibility sends the source code to the API this way (according to @mpnkhan on paypal/nemo-accessibility#13 (comment)), so I'll need to resolve this issue before I can move on to using nemo-accessibility to automate the testing. I've pulled down the latest (which seemed to include an upgrade to HTMLCS), but I'm unfortunately still having issues.
If I copy the source of a short page such as https://github.com/about and paste it in the Test HTML tab, it works fine with all of the engines (Axe, Chrome, HTML Code Sniffer). If I try to test source code from a slightly longer page such as https://github.com/features it returns the following error:
Error: spawn ENAMETOOLONG
at exports._errnoException (util.js:1026:11)
at ChildProcess.spawn (internal/child_process.js:313:11)
at exports.spawn (child_process.js:380:9)
at Object.exports.execFile (child_process.js:143:15)
at C:\Users\mswithen\Documents\AATT\app.js:149:17
at Layer.handle [as handle_request] (C:\Users\mswithen\Documents\AATT\node_m
odules\express\lib\router\layer.js:82:5)
at next (C:\Users\mswithen\Documents\AATT\node_modules\express\lib\router\ro
ute.js:100:13)
at Route.dispatch (C:\Users\mswithen\Documents\AATT\node_modules\express\lib
\router\route.js:81:3)
at Layer.handle [as handle_request] (C:\Users\mswithen\Documents\AATT\node_m
odules\express\lib\router\layer.js:82:5)
at C:\Users\mswithen\Documents\AATT\node_modules\express\lib\router\index.js
:233:24
This error displays with long pages for all engines I have tried on the 'Test HTML' tab -- Axe, Chrome, and HTML Code Sniffer. On the 'Page URL' tab, using the HTML Code Sniffer, longer pages process fine (however, Axe and Chrome return "undefined" with the same page on the 'Page URL' tab).
I would greatly appreciate your assistance. Thank you.
When I try to perform npm install, the prompt reports of the following,
found 10 vulnerabilities (1 low, 8 moderate, 1 high)
Could you help me here with whats wrong?
using npm 6.1.0
Everything installed correctly and is running locally without issues. However I never receive a result from the web-application, only "Loading.... Please wait...." dialog.
Wasn't running into any node issues, but shut down the apache server for good measure, removed local directory, fresh clone/dependency install and re-init. results in the same outcome.
In Readme.md, the default value for accessibility level mentioned is 'WCAG2A'.
But in module, the default level is 'WCAG2AA'.
Line 14 in 70a152d
I've successfully dockerized the AATT with Docckerfile
FROM node:8.9.4
RUN apt-get update &&
apt-get install -y openssh-client git &&
rm -rf /var/lib/apt/lists/*
COPY . /usr/src/app
WORKDIR /usr/src/appRUN npm install
RUN git submodule init
RUN git submodule updateEXPOSE 3000
ENTRYPOINT DEBUG=AATT* http_port=3000 node app.js
and .dockerignore
Dockerfile
./node_modules
It would be nice if the maintainer can include these files into master tree to help people deploying AATT much easier.
please add a param that allows starting the server on another port than 80
I am trying to use the HTMLCS option, using both the API and website, both hang when it tries to get the response from the server and no results are available. Eventually I get a timeout error on the API. But no reason for the hanging.
Get this info from the webserver.
E N G I N E undefined [ '--config=config/config.json',
'/home/mint/aatt/src/HTMLCS_Run.js',
'tmp/1468921496350.html',
'WCAG2AAA',
'1,2,3',
'json' ]
AATT:log sniffHTML, E N G I N E +9m htmlcs
$ DEBUG=AATT* http_port=80 node app.js giving the following error:
/var/www/html/tourplanner/AATT/app.js:165
fs.unlink(tempFilename, (err) => {
^
SyntaxError: Unexpected token >
at Module._compile (module.js:439:25)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
at startup (node.js:119:16)
at node.js:935:3
OS: Ubuntu 14.04
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.