percy / percy-puppeteer Goto Github PK
View Code? Open in Web Editor NEWVisual testing with Puppeteer and Percy
Home Page: https://percy.io
License: MIT License
Visual testing with Puppeteer and Percy
Home Page: https://percy.io
License: MIT License
ReferenceError: window is not defined
is thrown after upgrading to Jest 28.
Same setup works fine with Jest 27 though.
@percy/cli
version: 1.6.0"dependencies": {
"@percy/cli": "^1.6.0",
"@percy/puppeteer": "^2.0.2",
"expect-puppeteer": "^6.1.1",
"jest": "^28.1.2",
"jest-puppeteer": "^6.1.1",
"puppeteer": "^15.3.2"
},
ReferenceError: window is not defined
error occurs when referencing percySnapshot
method after upgrading to Jest 28.
Installing JSDOM environment explicitly does not help either. https://jestjs.io/docs/upgrading-to-jest28#jsdom
Test suite failed to run
The error below may be caused by using the wrong test environment, see https://jestjs.io/docs/configuration#testenvironment-string.
Consider using the "jsdom" test environment.
ReferenceError: window is not defined
at Object.<anonymous> (node_modules/@percy/sdk-utils/dist/bundle.js:347:9)
at Object.<anonymous> (node_modules/@percy/puppeteer/index.js:1:35)
Install Jest 28 and configure latest jest-puppeteer
as per documentation https://jestjs.io/docs/28.0/puppeteer#use-jest-puppeteer-preset
Run simple test:
import percySnapshot from '@percy/puppeteer';
describe('E2E test', () => {
it('renders', async () => {
await page.goto(`https://github.com`);
await percySnapshot(page, 'homepage');
});
});
When a DOM is between 10mb and 15mb an unclear error message appears. Add better handling around this case by outputting a clear error message.
Hi,
Faced an issue with local fonts failing to be rendered on percy rendering environment.
It works fine locally, it was verified by additional listener for this font loading.
Example of expected appearance:
Example of received screenshot from percy:
Tried:
Issue is tricky for me and is not reproducible for 100%.
Please advise how to fix or workaround it on my side, or will it be fixed on percy side(if so, when?).
Thank you in advance.
Unfortunately, spent around 15% of monthly screenshots trying to fix this issue until realised it cannot be fixed on my side.
The request is to expand API to do percySnapshot(frame: Frame)
, not just a Page
. It's possible that the current API works even now, since Frame
and Page
APIs are interchangeable for the most part. But would be good to have a typed and confirmed support.
When using vw
or vh
units, it's messing up with Chrome's screenshots.
Expected Result (Firefox):
What we've got (Chrome):
Seems related to puppeteer/puppeteer#703
Is there any way of fixing that issue in percy?
Hey folks!
We're evaluating how to best do Visual Regression tests for a range of problems. The first thing we're trying to tackle is our design system. Now, we don't want to use Storybook (we're using docz), but I thought I would be able to relatively easily use puppeteer to create snapshots of marked elements on our page. I have everything working so far, except I don't have a way to target sub-elements on a page instead of the whole page (the percySnapshot
function takes a page
as argument, which won't even run because typescript fails the test). Is this something you could help me with? I haven't dug into either puppeteer or these bindings enough yet, but think I could contribute it. I guess my first question would be; is this an option, an extra argument / overload, or an entirely new method?
Thanks!
Apologies if this isn’t the best place to report this! I couldn’t find information about Jest support anywhere in official resources so I’m not sure whether this would be considered a bug or not.
In #411, the import of SDK utils has been switched to use an ES modules dynamic import. This is problematic for users of Jest, because Jest uses Node’s VM API to execute code. The VM modules API for ES modules support is experimental (only available behind the --experimental-vm-modules
feature flag).
For the Puppeteer SDK – this means v2.0.0 works fine in Jest, and v2.0.1 doesn’t work, with the following error when running percySnapshot
:
TypeError: _vm(...).SyntheticModule is not a constructor
@percy/cli
version: v1.1.2@percy/[email protected]
Jest has experimental support for ES modules, so this can be resolved by adding the correct feature flag for Node to expose its vm.Module
API. I believe there are a few things worth addressing in the SDK nonetheless (assuming the SDK attempts to support Jest):
The only relevant bit is the stack trace.
[…]
[percy:cli:exec] Running "jest" (1ms)
● Groups › loads
TypeError: _vm(...).SyntheticModule is not a constructor
39 | beforeAll(async () => {
40 | await page.goto(`${TEST_ORIGIN}/admin/groups/2/`);
> 41 | await percySnapshot(page, 'test name')
| ^
42 | }, 10000);
at Runtime.loadCjsAsEsm (../../../node_modules/jest-runtime/build/index.js:657:20)
at percySnapshot (node_modules/@percy/puppeteer/index.js:9:15)
at Object.<anonymous> (groups.test.js:41:11)
A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --detectOpenHandles to find leaks.
[…]
Ran all test suites.
[percy:core:browser] Closing browser (25951ms)
[percy:core:browser] Browser closed (29ms)
[percy:core] Build not created (0ms)
I can provide a small reproduction case once confirmed Jest is indeed supported.
For people running into this, there are three possible workarounds:
It works.
Following Jest’s documentation. Here is what it looks like:
NODE_OPTIONS=--experimental-vm-modules percy exec -- jest
This will output a lot of warning messages.
@percy/sdk-utils
directlyThe code of v2.0.0 of the SDK can be copy-pasted and tweaked as needed to improve Jest support.
Whilst trying to add Percy in https://github.com/buildkite/site/compare/add-percy I hit the following error:
(node:72) UnhandledPromiseRejectionWarning: AssertionError [ERR_ASSERTION]: console.error not allowed in production: Refused to connect to 'http://localhost:5338/percy/healthcheck' because it violates the following Content Security Policy directive: "connect-src 'self' https://www.google-analytics.com".
The CSP being used is:
default-src 'self' 'unsafe-inline'; script-src 'self' 'unsafe-inline' https://www.google-analytics.com; img-src 'self' 'unsafe-inline' data: https://www.google-analytics.com; connect-src 'self' https://www.google-analytics.com; media-src 'self' https://d3lj8s78qytm30.cloudfront.net
I could use Puppeteer’s page.setBypassCSP and disable CSP entirely, but because it's black-box testing of the prod Docker image (and we want to test the CSP doesn't break anything in the Percy screenshots) I'd prefer to leave it as-is.
Is there any known workaround for this, with the new percy/agent setup? I searched the Percy docs for CSP and Content-Security-Policy, but I couldn't find anything.
I started to integrate Percy into my puppeteer testing flow and came across a failure I can't seem to work around. Percy successfully gets a Dom snapshot but has a final snapshot step which invokes page.get
reloading test harness.
When the test harness reloads it aborts the current test and Percy fails.
I looked through the code and I can't figure out why we need to reload the page given it was already loaded. Perhaps to reset the injected Percy code?
One solution that just came to me is stubbing the page object to ignore the extra reload but curious if there are other options.
Hey there!
I have a really long list of tests (~1000) and I wouldn't want to run them synchronously, it would take a really long time.
That's why I'm creating multiple pages on the same browser instance (this is different from the example repo in that the browser object is recreated for each test, it probably doesn't matter because I tried creating a new browser instance for each async operation).
Unfortunately, whenever I try to run the snapshots for multiple pages I get this kind of error:
UnhandledPromiseRejectionWarning: Error: Request is already handled!
Im having problems getting the latest version of percy to work with puppeteer. Before this new CLI update it worked very well.
The problem seems to be that i run 23 x 3 of these tests:
it('should not have visual regressions in new dark theme (percy)', async () => {
const page = await browser.newPage();
await page.setBypassCSP(true);
await page.goto(url, { waitUntil: 'load' });
await page.evaluate(() => {
document.querySelector('ids-theme-switcher').setAttribute('mode', 'dark');
});
await percySnapshot(page, 'ids-textarea-new-dark');
});
And at a certain point it will stop and error with
: Timeout - Async callback was not invoked within the 25000 ms timeout specified by jest.setTimeout.Timeout - Async callback was not invoked within the 25000 ms timeout specified by jest.setTimeout.Error:
21 | });
22 |
> 23 | it('should not have visual regressions in new contrast theme (percy)', async () => {
| ^
24 | const page = await browser.newPage();
25 | await page.setBypassCSP(true);
26 | await page.goto(url, { waitUntil: 'load' });
at new Spec (node_modules/jest-jasmine2/build/jasmine/Spec.js:116:22)
at Suite.<anonymous> (test/ids-text/ids-text-percy-test.js:23:3)
(node:20849) UnhandledPromiseRejectionWarning: Error: Caught error after test environment was torn down
Protocol error (Page.navigate): Target closed.
at /Users/tmcconechy/Dev/enterprise-wc/node_modules/puppeteer/src/common/Connection.ts:270:57
at new Promise (<anonymous>)
at CDPSession.send (/Users/tmcconechy/Dev/enterprise-wc/node_modules/puppeteer/src/common/Connection.ts:269:12)
Seems like either my test is wrong but i cant see anything else i can do? Or there is some issue with percy that makes it "crash out" after a certain number of screenshots.
This is the repo/build check if this helps https://github.com/infor-design/enterprise-wc/pull/59/checks?check_run_id=2308847368#step:8:1512
Any suggestions?
Using Percy with Puppeteer, for example with simple script:
await page.goto(url); await percySnapshot(page, 'name');
I receive sometimes a screenshot of the full page and sometimes only part of it.
It happens pretty often and was not able to find a consistent solution.
Not sure is it a Percy's issue or Puppeteer itself.
PNG images are not loaded in screenshot results (on this page SVG - displayed, JPG - random (sometime show), PNG - only alt text): https://percy.io/10ecbde0/Lition/builds/8869567/changed/503202803?browser=firefox&browser_ids=9%2C14&subcategories=approved&viewLayout=side-by-side&viewMode=new&width=1920&widths=375%2C1920
On puppeteer screenshot displayed all images.
This happened in period from 2021-02-08 - 2021-02-15, before - everything works fine
Hi im having an issue were percy service, it throws an error if I try to snapshot a URL that is a file I opened from the system.
The currently flow is:
I had to do a work around this simple scenario and create a ExpressJS server to provide that generated HTML as http://localhost/...
My code looks like
const filePath = resolve(fileName);
await page.goto(`file://${filePath}`, {
waitUntil: "networkidle0"
});
but when I run my tests/snapshot on that code I get
Is there a way to handle this?
It would be really helpful to have better error logging here:
Lines 28 to 37 in ace51ad
I get the CSP error pretty regularly when setting up stuff with Percyscript, and CSP usually isn't the problem. Right now, I have a setup that works locally but fails in CI, and it would be extremely helpful to know what is actually throwing the error here.
I also got this error earlier in the setup, but again it wasn't a CSP issue. I simply didn't have my Percy token loaded into the env properly. It probably took me 5x as long to debug because of the misleading and unhelpful message.
I am working on a common library for our 12 projects setup.
There is an error compiling TypeScript due to missing @percy/core
dependency.
@percy/cli
version: N/A@percy/puppeteer
has @percy/core
listed as a development dependency, but should be also an optional peer dependency for TS users.
$ tsc --build
node_modules/@percy/puppeteer/types/index.d.ts:2:33 - error TS2307: Cannot find module '@percy/core' or its corresponding type declarations.
import { SnapshotOptions } from '@percy/core';
Package manifest:
{
"name": "@xxx/e2e-test-utils",
"description": "XXX generic end-to-end test utilities.",
...
"dependencies": {
"@percy/puppeteer": "^2.0.2"
},
"devDependencies": {},
"peerDependencies": {
"puppeteer": "^14.4.1"
},
...
}
Helper function:
import percySnapshot from '@percy/puppeteer';
export const takeSnapshot = (name: string) => percySnapshot(page, name);
Hi, I'm running into an issue using @percy/cli
with @percy/puppeteer
and some quick googling didn't seem to turn up anything particularly relevant. Specifically I'm trying to take snapshots at various points during my company's initial onboarding flow. At one point this was working as expected but I now see the following error for every screenshot in my build:
When I run the test without headless mode it steps through each page and they look like they should. I don't see anything particularly suspicious in the debug logs either. Anyone have any suggestions?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.