cypress-io / circleci-orb Goto Github PK

• All the way to command.
• Prepare executors matching our Docker images

On our regular install job for other CI steps, we set:

PUPPETEER_SKIP_CHROMIUM_DOWNLOAD: 'true'

to avoid installing chromium for jobs which do not use it, as the install is slow. We instead use the circleci base docker image with Chromium pre-installed.

It'd be nice to be able to set this variable, and potentially other variables in order to skip this in the cypress install step as well.

to automatically update dependencies

Circle is not picking up my dependency changes and I see no way to clear the cache from the config file or the UI. Thanks!

Semantic convention, https://github.com/bahmutov/semantic-action

Could probably be different based on the branch: develop branch here could publish cypress-io/cypress@dev:x.y.z while master branch could publish cypress-io/[email protected]

Generate Markdown documenting the main cypress/install and cypress/run commands and their parameters

version: 2.1
orbs:
  # import Cypress orb by specifying an exact version
  cypress: cypress-io/[email protected]

workflows:
  version: 2.1
  build_and_deploy_staging:
    jobs:
      - build
      - deploy-staging:
          requires:
            - build
          filters:
            branches:
              only: master

      - cypress/run:
          record: true
          parallel: false
          requires:
            - deploy-staging
          filters:
            branches:
              only: master

This is weird, the CI has stopped working using the yml in circleCI. Do i need to change where to store the cache?

We can simplify the code for optional build and starts steps I think. See https://github.com/CircleCI-Public/config-preview-sdk/blob/master/docs/parameters.md#steps

To save 20 seconds?

For simple projects without parallel builds this step is unnecessary and takes 20 seconds

In project like https://github.com/cypress-io/circleci-orb-test I see expanded config file using npm run process

$ npm run process

> [email protected] process /Users/gleb/git/circleci-orb-test
> circleci config process .circleci/config.yml

version: 2
jobs:
  cypress/run:
    docker:
    - image: cypress/base:10
    steps:
    - checkout
    - restore_cache:
        keys:
        - cache-{{ .Branch }}-{{ checksum "package.json" }}
    - run:
        name: Npm CI
        command: npm ci
    - run:
        command: npx cypress verify
    - save_cache:
        key: cache-{{ .Branch }}-{{ checksum "package.json" }}
        paths:
        - ~/.npm
        - ~/.cache
    - persist_to_workspace:
        root: ~/
        paths:
        - project
        - .cache/Cypress
    - attach_workspace:
        at: ~/
    - run:
        name: Run Cypress tests
        command: 'npx cypress run  --record --group ''all tests''  '
workflows:
  build:
    jobs:
    - cypress/run
  version: 2

# Original config.yml file:
# version: 2.1
# orbs:
#   cypress: cypress/cypress@dev:0.0.1
# 
# workflows:
#   build:
#     jobs:
#       - cypress/run:
#           group: \"all tests\"
#           record: true

Because the readme keeps growing.

Also add link to our CircleCI Orb page into README itself.

For our test setup we rely on docker commands, so we need to run those in circleci (using the setup_remote_docker step, see here). Sadly this option is only available when basing on the official circleci node images circleci/node:xx. The cypress images though are based on the node:xx images which do not support this.
There might be even more capabilities the official circleci images are adding that won't be supported when using node:xx.

I know this is kind of a stretch and that'd mean more images/dockerfiles to maintain for you but as this is a circleci specific orb, I thought I'd kindly ask. Currently we're working around this by rolling our own docker image: https://github.com/JoinColony/colony-docker-images/blob/master/circleci-cypress/Dockerfile, but I see that a lot of work went into this orb and I'd really like to use it :)

Of this orb for testing

Currently, when the cypress/install job is using npm, the command npm ci is used to install packages. The npm-ci docs note that

If dependencies in the package lock do not match those in package.json, npm ci will exit with an error, instead of updating the package lock.

which makes sense in a CI environment.

By contrast, when yarn is being used, the command yarn install is used to install packages, which will happily update yarn.lock if it doesn't match with package.json. Per the yarn install docs, the --frozen-lockfile flag is needed to get similar behavior to npm ci, in the sense of requiring that the package.json and the lockfile are in agreement.

Could --frozen-lockfile be added to the yarn install command or made configurable with a parameter so that the build will fail if package.json and the lockfile disagree, like when npm is used?

https://docs.cypress.io/guides/guides/command-line.html#cypress-run-spec-lt-spec-gt

Using store_artifacts option after run

- store_artifacts:
      path: cypress/videos
- store_artifacts:
      path: cypress/screenshots

To publish Cypress orb on Circle CI first I need to create organization namespace. Seems it is tied to GitHub permissions because when I run the command I get the following error

$ circleci namespace create cypress github cypress-io
Error: AUTHORIZATION_FAILURE

where create <namespace> github <github organization name>

Can someone who is an admin for Cypress create the namespace before I publish this orb there?

Need to install Circle CLI tool and login:

• https://github.com/CircleCI-Public/circleci-cli#install-script
• https://github.com/CircleCI-Public/circleci-cli#configure-the-cli

Our general circle workflow is to do an install using npm ci, and then to fan out to the jobs we need to run in parallel, attaching the initial workspace to each job.

This is not possible when using: cypress/run and results in duplicate install logic/time, using up an additional container.

Additionally, we sometimes still use other install steps. One such example is bower install (or front-end component library http://origami.ft.com/ still uses Bower) - adding a ./node_modules.bin/bower install build parameter increases duplication, skips the existing circle bower cache, and requires an additional --allow-root flag due to the container used.

It'd be a nice to have to be able to skip the cypress/install step by passing a flag, but still attach the workspace at the correct location.

With appropriate documentation this may remove the need for modification for #28

I am getting the following error on CircleCi

#!/bin/sh -eo pipefail
# Error loading cypress-io/[email protected]: only certified orbs are permitted in this project.
# 
# -------
# Warning: This configuration was auto-generated to show you the message above.
# Don't rerun this job. Rerunning will have no effect.

note: the same happens with cypress-io/[email protected] as well

my config.yml looks similar to this:

version: 2.1

orbs:
  cypress: cypress-io/[email protected]

jobs:
  ...
workflows:
  version: 2
  build-deploy:
    jobs:
      ...
      - cypress/run
      ...

To match discussion in cypress-io/cypress-example-circleci-orb#4 (comment)

Orb should be cypress-io/cypress

somehow need to resolve examples using orb.yml file

Tried validating and processing inline orb examples, but it does not detect bad ones, hmm

Including 180-200 characters for Circle promotional materials

Examples of orbs:

• https://github.com/CircleCI-Public/circleci-orbs/blob/master/src/rollbar/orb.yml
• https://github.com/CircleCI-Public/circleci-orbs/blob/master/src/codecov/orb.yml
• https://github.com/CircleCI-Public/circleci-orbs/blob/master/src/heroku/orb.yml

Example descriptions:

• Commands for calling the Rollbar API
• Provides basic functionality for deploying to Heroku
• longer one:

Provides a command for reporting code coverage to Codecov.
The token will be read from $CODECOV_TOKEN by default, but this
can be overridden using the `token` parameter.

Hi there.

Sorry if I'm missing something obvious here. I have enabled support for new CircleCI config 2.1 features within my CircleCI project, but when running CircleCI I get:

#!/bin/sh -eo pipefail
# Warning: Don't rerun. Rerunning will *not* recompile the config.
# Error loading cypress-io/[email protected]: only certified orbs are permitted in this project.
false
Exited with code 1

Any ideas?

I'm new to CircleCI, and I'm trying to use the Cypress CircleCI Orb. To build my application, I need to install private NPM packages (using Yarn). So I need to be able to run a command like this somewhere in config.yml:

echo "//registry.npmjs.org/:_authToken=${NPM_TOKEN}" > ~/.npmrc

I don't know where to put it though. The install job's build parameter runs after yarn install. Is it possible to set the NPM auth token somewhere? Maybe @eddiewebb would have a suggestion? I think one approach would be to add a parameter called pre to the install job.

Alternatively, I'd imagine I could write out my config.yml with all the settings I need (except for the NPM token). Then, I could "eject" using circleci config process config.yml and figure out where to add the NPM auth token.

Side Note

This Orb seems really cool! We've been using the Cypress Dashboard and Travis CI to run our Cypress tests in CI for the past several months. We never found a way to avoid installing Cypress on every machine when running the tests in parallel. That introduces a huge bottleneck. We could run our test suite so much more efficiently with this orb!

Here is our circle ci set up. The error I am getting using a starter plan is about grouping. I thought that was fixed in previous PRs, but it is still showing up.

workflows:
  version: 2.1
  build_and_deploy_staging:
    jobs:
      - build
      - deploy-staging:
          requires:
            - build
          filters:
            branches:
              only: master

      - cypress/run:
          record: true
          parallel: false
          requires:
            - deploy-staging
          filters:
            branches:
              only: master

#!/bin/bash -eo pipefail
npx cypress run \
   \
   \
   --record \
    --group 'default group' \
     \
Grouping is not included under your current billing plan.

To run your tests with groups, please visit your billing and upgrade to another plan with grouping.
...

We don't have a group set, is it possible to do this on the free tier?

Links to our project ci server and github repository are below:

https://circleci.com/gh/build-canaries/nevergreen/1224

https://github.com/build-canaries/nevergreen/blob/51e67aca6bb52d2f89cf3e2da820fa6d074476b4/.circleci/config.yml#L184

Originally posted by @cowley05 in #43 (comment)

Sometimes we need to build application (once) before running tests. Need to expose parameter or steps to do this

To avoid mixing dependencies from different OS. Then we could use same orb to build on Linux and Mac for example, like I am trying in cypress-io/cypress-example-todomvc#80

it would be nice to do this

version: 2.1
orbs:
  cypress: cypress-io/[email protected]

executors:
  mac:
    macos:
      xcode: "10.1.0"

workflows:
  build:
    jobs:
      - cypress/run:
          name: Linux test
      - cypress/run:
          name: Mac test
          executor: mac

and get both builds using their own caches. Right now it tries to use Linux cache on Mac https://circleci.com/gh/cypress-io/cypress-example-todomvc/313

Found a cache from build 307 at cache-mac-build-79-ipAJGERZllQPwdX7i0NoFA5DdFdL7ktmv7d1IrOvf7w=
Size: 146 MB
Cached paths:
  * /root/.npm
  * /root/.cache

Downloading cache archive...
Validating cache...

Unarchiving cache...
root/.npm/: Failed to create dir 'root'Can't create 'root/.npm'
root/.npm/_cacache/: Failed to create dir 'root'Can't create 'root/.npm/_cacache'

How do you control browsers docker container?

By version, use https://github.com/bahmutov/next-ver#readme

You can call circleci orb process command to give you fully parsed and translated config YML text that replaces all commands. This could be the way to go from using the Cypress orb to normal config and allow you to do configuration that Cypress orb jobs do not allow.

Need to document this

Because CircleCI gives 4 boxes by default, not 10

To simplify parallel jobs and commands

CircleCI-Public/config-preview-sdk#121

I have a project where I have a server folder which has its own package.json file. The orb does not seem to install any of these dependencies, despite my root package.json having a postinstall step in it which works fine in development.

my-app/
    |_ package.json
    |_ package-lock.json
    server/  <-- dependencies don't get installed
      |_ package.json
      |_ package-lock.json

I assume this is because npm ci doesn't use package.json? If so, how can I install a nested folder's dependencies?

I have tried many things, including just running npm run postinstall or cd server && npm i in the build step of cypress/install but nothing seems to work.

Here is my current circle.yml file:

version: 2.1
orbs:
  cypress: cypress-io/[email protected]
workflows:
  build:
    jobs:
      - cypress/install:
          build: 'npm run build && npm run postinstall'
      - cypress/run:
          requires:
            - cypress/install
          group: portal
          start: npm run server
          wait-on: 'http://localhost:3000'

I'd very much appreciate some guidance, I'm excited to get Cypress working with my CI so I can start writing more tests!

Thanks!

Hi!
I created a basic project with Vue-CLI 3 and I added two tests:

• One E2E test
• One unit test with cypress-vue-unit-test.

Using the simple circle.yml config, the unit test passes, but the E2E test fails:

https://circleci.com/gh/mornir/tdd-monster-slayer/9

I guess I need to start the Vue app so Cypress can visit it?
I've tried all sorts of npm commands in the circle.yml, but none worked.

I am new to continuous integration and my mistake must be obvious to more experienced developers...
I was kind of expecting for everything to be working magically out the box 😅

Using the example from the readme on CircleCI, I get the following issue:

#!/bin/sh -eo pipefail
# Error calling workflow: 'build'
# Error calling job: 'cypress/install'
# Cannot find orb 'cypress' looking for executor named 'cypress/base-10'
# 
# -------
# Warning: This configuration was auto-generated to show you the message above.
# Don't rerun this job. Rerunning will have no effect.
false
Exited with code 1

Seems like if you pass executor: cypress/base-8 to both cypress/install and cypress/run things work.

Currently, the install step fails with NPM's cryptic message if there is no package lock file. We could check for this file ourselves and give a much better error message.

[[ ! -e ./package-lock.json ]] && echo 'the cypress orb requires you to have a package-lock.json' \ && [[ ! -z $(cat .gitignore | grep package-lock) ]] && echo 'you should remove package-lock.json from your .gitignore'

But only do it on NPM install, not yarn install

i would say a different message like

> the cypress orb uses `npm ci` to install `node_modules`, which requires a `package-lock.json`.
> a `package-lock.json` file was not found. please run `npm install` in your project, and commit `package-lock.json` to your repo

cc @brian-mann @bkucera

Since now we understand how to program the orb, no need for "hello world" example. This could be a patch release, since this job was "internal" demo job.

Need to expose command / parameter to start the server before running tests.

Related problem: build the app

For wait-on command and maybe for Cypress command?

Using CircleCI no_output_timeout option, probably do not even expose it to the user, just 5 minutes without stdout output and it is done

Need to clearly state what needs to happen

• for organization to enable 3rd party orbs
• for project to enable beta features

for #34

What if the user wants to use yarn to install dependencies instead of npm ci?

My environment requires two servers to run before testing is possible.

In the docs you provide the start param -

    - cypress/run:
        start: 'npm start'    # start server before running tests

Which runs the command in the background.
But how can i run two or more servers?

If we use npm ci to install NPM dependencies and Cypress binary we need to cache folders

• ~/.npm
• ~/.cache

If we allow yarn command to install NPM dependencies we need to only cache folder ~/.cache

We cannot control the cached folders using a parameter, unfortunately. I would love to say something like this

when <parameters.command> eq "npm ci"
  - save_cache:
        key: cache-npm-ci-{{ .Branch }}-{{ checksum "package.json" }}
        paths:
        - ~/.npm
        - ~/.cache
when <parameters.command> eq "yarn"
  - save_cache:
        key: cache-yarn-{{ .Branch }}-{{ checksum "package.json" }}
        paths:
        - ~/.cache  

and the right folder would be cached

In the docs for the docker version of running tests with cypress you had an example of storing the test result in Circle CI, i can't find any similar documentation for the orb version? Am i missing something?

If the server takes a while to start, we need to "wait" for the url to respond. Similar to https://github.com/bahmutov/start-server-and-test we could use npx wait-on ... parameter if passed to wait for the url to respond before continuing with tests

use cypress-io/cypress@1 and not cypress-io/[email protected] in orb readme