Giter VIP home page Giter VIP logo

agent-cypress's Introduction

Native integration for Cypress with Visual Regression Tracker

Codacy Badge

Npm: https://www.npmjs.com/package/@visual-regression-tracker/agent-cypress

Installation

Please note that the minimum supported Cypress version is 12.17.1, released in 10th July 2023.

Add package

npm install @visual-regression-tracker/agent-cypress

Add commands

<rootDir>/cypress/support/e2e.js or <rootDir>/cypress/support/component.js

All available commands:

import {
  addVrtCommands,
} from "@visual-regression-tracker/agent-cypress";

addVrtCommands();

or one by one:

import {
  addVrtTrackCommand,
  addVrtStartCommand,
  addVrtStopCommand,
  addVrtTrackBufferCommand,
  addVrtTrackBase64Command,
} from "@visual-regression-tracker/agent-cypress";

addVrtStartCommand();
addVrtStopCommand();
addVrtTrackCommand();
addVrtTrackBufferCommand();
addVrtTrackBase64Command();

Add plugin

<rootDir>/cypress.config.js

import { defineConfig } from 'cypress';
import { addVisualRegressionTrackerPlugin } from "@visual-regression-tracker/agent-cypress";

export default defineConfig({
  // e2e or component, depending of testing style
  e2e: {
    setupNodeEvents (on, config) => {
      addVisualRegressionTrackerPlugin(on, config);
    }
  }
});

Configuration

Update Cypress config

<rootDir>/cypress.config.js

export default defineConfig({

  env: {
    "visualRegressionTracker": {
      // URL where backend is running
      // Required
      "apiUrl": "http://localhost:4200",

      // Project name or ID
      // Required
      "project": "Default project",

      // User apiKey
      // Required
      "apiKey": "tXZVHX0EA4YQM1MGDD",

      // Current git branch
      // Required
      "branchName": "develop",

      // Branch with baseline
      // Optional - when not set, main branch from project settings is used
      "baselineBranchName": "release",

      // Log errors instead of throwing exceptions
      // Optional - default false
      "enableSoftAssert": true,

      // Unique ID related to one CI build
      // Optional - default null
      "ciBuildId": "SOME_UNIQUE_ID",
    }
  }
});

Or, as JSON config file vrt.json

Used only if not explicit config provided Is overriden if ENV variables are present

{
  "apiUrl": "http://localhost:4200",
  "project": "Default project",
  "apiKey": "tXZVHX0EA4YQM1MGDD",
  "ciBuildId": "commit_sha",
  "branchName": "develop",
  "enableSoftAssert": false
}

Or, as environment variables

Used only if not explicit config provided

VRT_APIURL="http://localhost:4200"
VRT_PROJECT="Default project"
VRT_APIKEY="tXZVHX0EA4YQM1MGDD"
VRT_CIBUILDID="commit_sha"
VRT_BRANCHNAME="develop"
VRT_BASELINEBRANCHNAME="release"
VRT_ENABLESOFTASSERT=true

Usage

Setup

cy.vrtStart();

Assert

cy.vrtTrack("Whole page with default params");

cy.get("#navbar").vrtTrack("Separate element with default params");

cy.vrtTrack(
  "Whole page with additional options",
  {
    viewport: "1920x1080",
    os: "MacOS",
    device: "Cloud agent",
    customTags: "Cloud, DarkTheme, Auth",
    diffTollerancePercent: 1,
    ignoreAreas: [{ x: 1, y: 2, width: 100, height: 200 }],
    retryLimit: 2,
    keepScreenshot: false, // Keep screenshot local copy, false by default
  },
  (err) => {
    console.log("Screenshot has diff with baseline", err);
    return true; // Skip failing test
  }
);
options (optional)

Allows to set options for taking screenshot. All options from screenshot command are also supported more details

Viewport is taken from Cypress.config(), if option is not set

Browser is taken from Cypress.browser.name

errorCallbak (optional)

Allows you to define a callback that receives the error for custom side-effects.

Also allows to override assertion policy. When callback returns true this acts similar to enableSoftAssertions option in config, but allows to enable soft assertion only for one specific screenshot.

Teadown

cy.vrtStop();

Examples

https://github.com/Visual-Regression-Tracker/examples-js

agent-cypress's People

Contributors

dekaravanhoc avatar dependabot-preview[bot] avatar dependabot[bot] avatar dimitriharding avatar eduard-ich avatar paazmaya avatar pashidlos avatar wsbaser avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

agent-cypress's Issues

Example test

No response from server

Because this error occurred during a before all hook we are skipping all of the remaining tests.
node_modules/@visual-regression-tracker/agent-cypress/dist/commands.js:23:1
  21 |             .then(function (err) {
  22 |             if (err) {
> 23 |                 throw new Error(err);
     | ^
  24 |             }
  25 |         })
  26 |             .then(function () {

I followed the readme, but I always receive this error message

How to disable this in local environment?

Hi!

We use this plugin on our CI to make use of the tests and keep the library updated.

But on local env the tests don't run without this plugin configured (and we don't want to configure it locally).
Is there a workaround?

cannot use VRT + Cypress agent to compare images for projects that use odiff or look-same libraries

Trying to use VRT + Cypress agent to compare images for projects that use odiff or look-same image compare libraries will result in the following error:

  CypressError: `cy.task('VRT_TRACK_IMAGE_MULTIPART')` failed with the following error:

  > {
        "path": "/test-runs/multipart",
        "name": "Error",
        "message": "",
        "exception": {},
        "stack": "Error\n    at ChildProcess.<anonymous> (/node_modules/odiff-bin/odiff.js:147:13)\n    at ChildProcess.emit (events.js:327:22)\n    at maybeClose (internal/child_process.js:1021:16)\n    at Socket.<anonymous> (internal/child_process.js:443:11)\n    at Socket.emit (events.js:315:20)\n    at Pipe.<anonymous> (net.js:674:12)"
  }

  https://on.cypress.io/api/task

Could you please let me know if

  • this might come from something that I am not doing correctly
    • e.g. an error in the installation or the implementation, or
  • if it's an expected/known issue
    • e.g. cannot use odiff or look-same libraries with Cypress agent

I tried to find some documentation on this or troubleshoot myself, but unfortunately could not find anything.

Thank you ๐Ÿ™‚

Stopping the test when no baseline exists prevents all screenshots from being taken

Hi, I'm currently implementing this and have come across an issue.

We have a cypress command that modifies the viewport and takes 5 screenshots at different screen sizes. The issue is that when the first screenshot has no baseline, the test stops and doesn't take the next 4 screenshots.

Setting enableSoftAssert: true let's the test run all the way through, but then causes another problem - the tests never fail when there are differences.

It would be great to have an auto-approve if no baseline exists setting in VRT, or alternatively a way to take all the screenshots before the test fails.

Thanks.

error adding commands from documentation

I'm using the latest version of VRT Agent Cypress.
After adding the command settings from the documentation, I get this error when running the tests.

Error

image

support/commands.js

image

"enableSoftAssert": true does not allow cy.vrtTrack() function to run in async and allow tests to run independently

Tested the below use case with the Cypress-Agent:

  • in the vrt.json file in the root of the project add "enableSoftAssert": true
  • for a simple test that runs in 1 sec create 10 vrtTracks at the end of the test
  • the test now takes 70-80 seconds to finish

Is there any way to allow the tests to run async and independent of the engine of VRT, at least when the enableSoftAssert is set to true?

If we don't want to fail the tests because of the results of the visual test, then at least we could leverage the speed of the tests.

Let me know if this issue should actually be moved to the js-sdk - was not 100% sure, but that might be best ๐Ÿ™‚

Adding VRT plugin conflicts with other plugins

Adding the VRT plugin, as per the example in the readme: https://github.com/Visual-Regression-Tracker/agent-cypress#add-plugin

Will conflict with other plugins required for some specific cases, for example a plugin created to allow the manipulation of the screen size in run mode: https://docs.cypress.io/api/plugins/browser-launch-api#Set-screen-size-when-running-headless

Due to this matter, the results of the vrtTrack command are different screen sizes between Open Mode and Run Mode, resulting in an error from VRT as the different screen sizes cannot be compared.

There is a simple workaround to this, that is to add the screen-size-manipulation plugin in the same export as the VRT plugin, but would prefer to be able to keep them separately.

This issue might be coming from the fact that the VRT plugin is exported by and then ran by plugins file as a function, instead a module export, as per here:

export function addVisualRegressionTrackerPlugin(on, config) {

Would be glad to know your thoughts and if you might have a quick-win solution for this one ๐Ÿ™‚

How does vrtTrackBase64 command work?

I see that the vrtTrackBase64 command requires an image to be added as an argument for execution.

Could you please share some example of how this function can be used?
I found an example here: https://github.com/Visual-Regression-Tracker/examples-js/blob/17575d8621e4974b24b462a009250a281d79616e/agent-cypress/cypress/integration/vrt.spec.js#L18

But is the tool restricted to using an image that exists locally, or can it take as an argument the cy.screenshot command?

Would be interesting to better understand how that works ๐Ÿ™‚

Also, maybe it'd be interesting for admin to open up a Discussions section in the VRT Gitlab projects, where we can ask such questions and discuss topics with project administrators and the community.

Error on command add

Hi, it's pretty similar issue to #217 but the topic was closed.
I am having the following error:

X [ERROR] Could not resolve "fs"

    node_modules/@visual-regression-tracker/agent-cypress/dist/plugin.js:13:21:
      13 โ”‚ const fs_1 = require("fs");
         โ•ต                      ~~~~

  The package "fs" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error.

X [ERROR] Could not resolve "fs"

    node_modules/@visual-regression-tracker/sdk-js/dist/helpers/config.helper.js:4:21:
      4 โ”‚ const fs_1 = require("fs");
        โ•ต                      ~~~~

  The package "fs" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error.

X [ERROR] Could not resolve "fs"

    node_modules/@visual-regression-tracker/sdk-js/dist/helpers/dto.helper.js:8:37:
      8 โ”‚ const fs_1 = __importDefault(require("fs"));
        โ•ต                                      ~~~~

  The package "fs" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error.


Oops...we found an error preparing this test file:

  > cypress\support\index.ts

The error was:

Error: Build failed with 3 errors:
node_modules/@visual-regression-tracker/agent-cypress/dist/plugin.js:13:21: ERROR: Could not resolve "fs"
node_modules/@visual-regression-tracker/sdk-js/dist/helpers/config.helper.js:4:21: ERROR: Could not resolve "fs"
node_modules/@visual-regression-tracker/sdk-js/dist/helpers/dto.helper.js:8:37: ERROR: Could not resolve "fs"
    at failureErrorWithLog (C:\Users\HASAZEN\GitHub\testproject-cbportal-user-interface\node_modules\esbuild\lib\main.js:1650:15)
    at C:\Users\HASAZEN\GitHub\testproject-cbportal-user-interface\node_modules\esbuild\lib\main.js:1059:25
    at runOnEndCallbacks (C:\Users\HASAZEN\GitHub\testproject-cbportal-user-interface\node_modules\esbuild\lib\main.js:1485:45)
    at buildResponseToResult (C:\Users\HASAZEN\GitHub\testproject-cbportal-user-interface\node_modules\esbuild\lib\main.js:1057:7)
    at C:\Users\HASAZEN\GitHub\testproject-cbportal-user-interface\node_modules\esbuild\lib\main.js:1086:16
    at responseCallbacks.<computed> (C:\Users\HASAZEN\GitHub\testproject-cbportal-user-interface\node_modules\esbuild\lib\main.js:703:9)
    at handleIncomingPacket (C:\Users\HASAZEN\GitHub\testproject-cbportal-user-interface\node_modules\esbuild\lib\main.js:763:9)
    at Socket.readFromStdout (C:\Users\HASAZEN\GitHub\testproject-cbportal-user-interface\node_modules\esbuild\lib\main.js:679:7)
    at Socket.emit (node:events:513:28)
    at Socket.emit (node:domain:489:12)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Socket.Readable.push (node:internal/streams/readable:234:10)

when I try to add commands to my e2e project.

Cypress version is: 13.7.3

Is this problem fixed completely with the new version ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.