Not sure the impact of this, currently tests are passing.

Looking through all the project mocks, some are:

• specifying an ancient runtime: 'nodejs:10'
• declaring dependency of "@adobe/asset-compute-sdk": "^2.10.6" which is 2 majors outdated

Also, the test-projects directory, after tests are run, consumes ~1.5 GB of disk space. It would be nice if this was cleaned up, or mocked so all the dependencies did not need to be installed.

Runtime is deprecating the nodejs:10 image on Jan. 4th, 2023.

Remove the image from the list of available kinds here and mark it as deprecated.

First of all, not sure if the right repo to raise that issue but in my opinion it fits the best. Anyway, when implementing workers to uses some 3rd party service you usually need to provide some secrets/credentials to use that service via the action parameters (via manifest.yml). Apparently, if such params are defined, if running the test-worker tests via aio app test those parameters simply does not exists - so unable to test properly any worker.

Expected Behaviour

• Any parameter defined in manifest.yml is passed to the worker action when running test-worker

Actual Behaviour

• Parameters from manifest.yml are simply ignored

Steps to reproduce

You can use my sample repo to reproduce the issue: https://github.com/marcinczeczko/aio-cli-test-worker-reproducer

• It's basically a vanilla worker code generate via aio app init
• I added myParam to the manifest.yml and params argument to the worker code, as follows and then printed out to console log.

exports.main = worker(async (source, rendition, params) {}

1. Run aio app test
2. Open the generate log file and fine the line (will start with All the params: that prints out all the action params). It will contains all the params without but no myParam

2020-10-06T09:39:41.197858382Z All the params:  {
  13   │ 2020-10-06T09:39:41.197882664Z   source: { url: 'file.jpg' },
  14   │ 2020-10-06T09:39:41.197888364Z   renditions: [ { fmt: 'jpg', qlt: 10, target: undefined } ],
  15   │ 2020-10-06T09:39:41.197892623Z   requestId: 'Test: corrupt-input -----------------------------------------------',
  16   │ 2020-10-06T09:39:41.197896726Z   metrics: AssetComputeMetrics {
  17   │ 2020-10-06T09:39:41.197923345Z     startDate: 2020-10-06T09:39:41.073Z,
  18   │ 2020-10-06T09:39:41.197928861Z     params: [Circular],
  19   │ 2020-10-06T09:39:41.197933175Z     newRelic: NewRelic { defaultMetrics: [Object], canSendMetrics: false }
  20   │ 2020-10-06T09:39:41.197956590Z   },
  21   │ 2020-10-06T09:39:41.197962640Z   times: {}
  22   │ 2020-10-06T09:39:41.197967078Z }

Platform and Version

System:
    OS: macOS 10.15.6
    CPU: (4) x64 Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz
    Memory: 30.11 MB / 16.00 GB
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 10.22.1 - ~/.asdf/installs/nodejs/10.22.1/bin/node
    Yarn: 1.22.5 - /usr/local/bin/yarn
    npm: 6.14.6 - ~/.asdf/installs/nodejs/10.22.1/bin/npm
  Virtualization:
    Docker: 19.03.12 - /usr/local/bin/docker
  npmGlobalPackages:
    @adobe/aio-cli: 4.0.0
  CLI plugins:
    @adobe/aio-cli 4.0.0 (core)
    @adobe/aio-cli-plugin-aem 1.0.4 (user)
    @adobe/aio-cli-plugin-aep 1.0.5 (user)
    @adobe/aio-cli-plugin-app 4.0.2 (core)
    @adobe/aio-cli-plugin-asset-compute 1.4.3 (user)
    @adobe/aio-cli-plugin-auth 2.4.0 (core)
    @adobe/aio-cli-plugin-certificate 0.2.0 (core)
    @adobe/aio-cli-plugin-cloudmanager 0.14.0 (user)
    @adobe/aio-cli-plugin-cna 0.2.1 (user)
    @adobe/aio-cli-plugin-config 2.3.0 (core)
    @adobe/aio-cli-plugin-console 3.2.0 (core)
    @adobe/aio-cli-plugin-events 1.1.0 (core)
    @adobe/aio-cli-plugin-info 1.1.1 (core)
    @adobe/aio-cli-plugin-runtime 2.0.0 (core)
    @oclif/plugin-autocomplete 0.2.0 (core)
    @oclif/plugin-not-found 1.2.4 (core)
    @oclif/plugin-plugins 1.9.0 (core)
    @oclif/plugin-warn-if-update-available 1.7.0 (core)

Sample Code that illustrates the problem

See my github repo: https://github.com/marcinczeczko/aio-cli-test-worker-reproducer

Logs taken while reproducing problem

See above.

Expected Behaviour

When testing a shellscript worker, it may be necessary to check for the presence or content of the out/errors/error.json and out/errors/type.txt output files defined by the ShellScriptWorker.prepareMetadata() function. A test script should have the ability to open these files for reading, in the same way that the out/rendition0.png file can be read.

Actual Behaviour

However, when running npm test in a CI environment under a different user than the docker daemon, the asset-compute plugin fails to ensure that the out/errors directory is pre-created with the execute bit allowed for everyone, which makes it possible for the docker user to create it with a more restrictive umask, leading to a situation like this:

$ sudo ls -laF /home/circleci/repo/.nui/2022_04_21T17_51_03_056Z/out
total 208
drwxrwxrwx 3 circleci       circleci         4096 Apr 21 17:51 ./
drwxrwxr-x 6 circleci       circleci         4096 Apr 21 17:51 ../
drwxrw-rw- 2 circleci-admin circleci-admin   4096 Apr 21 17:51 errors/
-rwxrw-rw- 1 circleci-admin circleci-admin 198804 Apr 21 17:51 rendition0.ai*

In the above example, even though the errors/ directory and its children are readable by everyone, the lack of the execute bit makes it impossible for the circleci user to open the directory to read the contents of those nested files.

Reproduce Scenario (including but not limited to)

1. Set up a circleci pipeline using the default machine (linux) runner, where the docker agent is running as circleci-admin and the shell is running as circleci.
2. Run npm test on a shellscript worker project where worker.sh writes content to either $errorfile or $typefile, for example:

mkdir -p "$(dirname "$typefile")"
echo "image/png" > "$typefile"

3. Attempt to read the contents of the out/errors/type.txt file in the validate script

expect_typefile="$(dirname "${BASH_SOURCE[0]}")/type.txt"
typefile="$(dirname "$2")/errors/type.txt"
cmp -s "${expect_typefile}" "${typefile}"

4. Confirm

Platform and Version

• CircleCI Machine Linux Medium
• @adobe/aio-cli-plugin-asset-compute 1.5.0

Links

• util.prepareInOutDir()
• ShellScriptWorker.prepareMetadata()

I have latest version of node, it failed when I execute the aio commad.

warning @adobe/aio-cli-plugin-asset-compute > @adobe/asset-compute-devtool > [email protected]: Please upgrade to latest, formidable@v2 or formidable@v3! Check these notes: https://bit.ly/2ZEqIau
warning @adobe/aio-cli-plugin-asset-compute > @adobe/asset-compute-devtool > express-formidable > [email protected]: Please upgrade to latest, formidable@v2 or formidable@v3! Check these notes: https://bit.ly/2ZEqIau
error @adobe/[email protected]: The engine "node" is incompatible with this module. Expected version "^10 || ^12 || ^14". Got "17.2.0"
error Found incompatible module.
Installing plugin @adobe/aio-cli-plugin-asset-compute... failed
Error: yarn add @adobe/aio-cli-plugin-asset-compute@latest
--non-interactive
--mutex=file:/Users/mohammedahmed/.local/share/@adobe/aio-cli/yarn.lock
--preferred-cache-folder=/Users/mohammedahmed/Library/Caches/@adobe/aio-cl
i/yarn --check-files exited with code 1

There is hacky code in the published init hook which modifies oclif's internal config plugin list.
Plugins should interact with the system through the apis of the system and not go and modify things, especially with hooks as they run for every command, not just asset-compute commands.

This is similar to previous practices where JS libraries would rewrite prototypes for other objects.

There is a devDep on {"@adobe/aio-cli-plugin-app": "^8.0.0"} however the current version is 9.2.0

I tried updating this, but immediately broke all the tests.

The way this is used it appears it could actually be a peer dependency. I think this need to be thought out and re-architected to work with published apis.

Calls to @adobe/aio-cli-plugin-app/src/commands/app/deploy are bound to break at any time, without even a major version bump, this is internal to the aio-cli-plugin-app and should not be directly depended on.