solidusio / circleci-orbs-extensions Goto Github PK

Since we use capybara-screenshot the path to store screenshots as CircleCi artifacts should be

spec/dummy/tmp/capybara

#29 introduced some code aimed at migrating the previous vendor/bundle paths. That code should be removed after we're reasonably sure most extensions picked it up.

The set date for the removal is after December 1st 2020.

We can speed up extensions builds by caching gems on CircleCI. I think we can do something very similar to what has been done in Solidus core.

Simplify the default test results path

We're using dynamic test results paths because for extensions using an
older setup different solidus versions are tested within the same job.

However, we don't need to mimic the entire cache key as path. CircleCI
doesn't interpolate everything within environment variable values, so
breaking this coupling will help in the future.

Currently all extension specs run with Ruby 2.7.

Is it possible to run specs with Ruby 3.0 as well?

The orbit uses three hardcoded cache keys:

https://github.com/solidusio/circleci-orbs-extensions/blame/fec1bff9ef447b2c8bf78b8da7f87aa03524570d/src/commands/test-branch.yml#L34-L36

We need a way to invalidate cache from individual extensions, probably adding an environment variable into the key names.

When creating a pull request on solidus_importer (see solidusio-contrib/solidus_importer#79), which uses this repository's CircleCI orbs, my team and I ran into an issue where specs fail due to encoding errors. The spec suite is expecting test data to be ASCII, but there are UTF-8 characters in some test CSV data files.

While we could fix the CSV data to use only ASCII characters, we think it would be more realistic to ensure that the CircleCI orb executors have their LANG configured to C.UTF-8. I think this is just a matter of ensuring that the Docker images being used have a UTF-8 locale available and that their LANG environment variable is being set.

Since we're using CircleCI-provided images I'm a little surprised this isn't already being done.

We need to add failures in the test summary of CircleCI. More info here:

https://circleci.com/docs/2.0/configuration-reference/#store_test_results

This is especially needed since PRs are not generating dev versions since we are not passing keys to fork for security concerns.

At the moment if an extension specs run fails at one specific Solidus version (let's say 2.5), all the other steps are not performed. This makes hard to have a complete matrix in every scenario.

This can be solved in several ways, for example, parallelizing jobs or using when: always option on test commands.

We've started seeing some failures on runs against Solidus master due to the fact that we are using a base CI image with Ruby 2.5. We've seen this issue appear when pry-byebug and pry-stack_explorer are referenced in an extension's Gemfile, but it is possible that other gems rely on this as well.

The error in question is

NameError: uninitialized constant DidYouMean::PlainFormatter

You can see some examples of this error on the Solidus Taxjar and Subscriptions extensions

https://app.circleci.com/pipelines/github/SuperGoodSoft/solidus_taxjar/309/workflows/062db350-ce0e-4916-855b-cc5a7b10fa8a/jobs/326

https://app.circleci.com/pipelines/github/solidusio-contrib/solidus_subscriptions?branch=master&filter=all

If there is interest in moving the base CI image to a more recent and supported version of Ruby, say 2.7, I can open a pull request for that, but first I wanted to see if that is something we would want to do. Let me know if I can provide any more details here.

Due to this issue iynere/compare-url#25 with a dependency of the current version of the orb-tools, the first commit in a PR will always have a CI step failing (see this workflow as example).

We can try to update the orb tooling to the latest version and see if it works well for us.