It's tricky to do in a plugin, using hooks.
Here is the farther i got, the extensions.json file is created as an asset, bur require('extensions.json') does not work...

const fs = require('fs');
const download = require('download');

// Plugin downloading the supported extensions from production backend
const pluginName = 'ExtensionsPlugin';
class ExtensionsPlugin {
  apply(compiler) {
    compiler.hooks.compilation.tap(pluginName, compilation => {
      compilation.hooks.additionalAssets.tapAsync(pluginName, callback => {

        console.debug('Downloading coverage extensions...');
        download('https://coverage.moz.tools/v2/extensions').then(data => {

          // Setup downloaded file as asset
          let extensions = data.toString('utf-8');
          compilation.assets['extensions.json'] = {
            source: () => {
              return extensions;
            },
            size: () => {
              return extensions.length;
            }
          };

          callback();
        });
      });
    });
  }
}

After looking at the chunk_mapping.sqlite database in PR mozilla/release-services#1401, I found that the count for the file_to_test table is exactly 360000. I think the limits in the ActiveData queries from chunk_mapping.py are too low.

Depends on mozilla/release-services#1401

Some tools can only send smaller hashes (12 first characters) : DXR with code coverage addon does that.

Right now, we always show mozilla-central. It would be nice to allow other options (e.g. try).

We can use pytest-responses to do this.

https://github.com/mozilla/coverage-crawler/blob/31c8b1841a931e3a4d15013f8463be9475b34f13/coverage_crawler/latest_cov_build.py#L107 - I wanted to run a crawler from the scratch and downloaded artifacts. However, this line resulted in tests_report.json weighing 157 bytes. Not sure whether this is relevant to grcov, or here.

This would be needed to make sure we successfully run tasks.

Let's say we receive a message for a review, we trigger a static analysis task and ack the pulse message removing it from the queue. Then pulse listener dies for some reason, the static analysis task fails. When the pulse listener is back, the failed task will not be retriggered as the pulse message is no longer in the queue.

This was originally filed as #800, which was fixed in #1812. Unfortunately the assertion failed, so I made it a warning in #1819.
I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1522304 to track the LCOV rewriter bug. Once it is fixed, we'll be able to turn the warning back into an assertion.

We temporarily switched to daily to ease development, but we should switch back to weekly.

According to this, https://github.com/marco-c/firefox-code-coverage/blob/cc386da20f389fd5159746d56729e149029de6ae/firefox_code_coverage/codecoverage.py#L81, if there is a downloaded artifact, then we don't download a new one. However, there could be a wrong one: for example, because in the previous try there was a bad Internet connection, and the download was not finished but the file exists. Can this be checked too (for example, to check whether the size of those are equal)? Or won't this condition occur because of this: https://github.com/marco-c/firefox-code-coverage/blob/cc386da20f389fd5159746d56729e149029de6ae/firefox_code_coverage/codecoverage.py#L88?
I had the same problem, and I have an artifact that weights 147 bytes and has an empty .info file inside, what seems to be wrong.

For example, as fixed here https://hg.mozilla.org/mozilla-central/rev/6aec9dc32d1c
there are files which are platform specific but built for others.

We should have a way to extract files with 0% code codecoverage on a platform but > 0% on others.

From Marco:

For JavaScript files, the top-level code is always covered if the file is loaded, so we came up with a heuristic: a "zero coverage" file is a file which has at least one function and whose functions are all not covered.
For C/C++ it's easier, as there's no top-level code and the code is always in functions. In this case, a "zero coverage" file is a file which has no covered lines.

It would great to put this in the UI somewhere for new users. Maybe add a "?" icon next to the uncovered files option that can show this explanation on hover.

As discussed with @marco-c on #1977, we can now migrate the code coverage function use Harbormaster:

• upload_coverage_results
• upload_lint_results

Since we are using webpack now, addons.mozilla.org requests us to also upload a zip file containing the source code of the addon.
It'd be nice if it was generated automatically.

We'll need to retrieve the covdir files and upload them on GCP.

The coverage from fuzzing should only be available to staff and NDA'd Mozillians, so we'll have to implement some kind of auth.

We will create a GitHub repository where we will store the reports. Enabling gh-pages will allow people to navigate the reports easily.

At the moment, the APIs are all retrieving the overall coverage.
We should add filtering capabilities so the frontend can require specific things from the backend.

Filters we should support:

• Platform (Linux, Windows, Mac, etc.);
• Test suite;
• Language?;
• Third-party code?

This can be very useful for some use cases.

Asynchronous workers are better suited for I/O bound tasks like ours.

The last chunk mapping database has an empty chunk_to_test table.

Last task: https://tools.taskcluster.net/groups/KmeRq8B1QxiOLQdkexEmRg/tasks/KmeRq8B1QxiOLQdkexEmRg/details
Chunk mapping: https://queue.taskcluster.net/v1/task/KmeRq8B1QxiOLQdkexEmRg/runs/0/artifacts/public/chunk_mapping.tar.xz

I cannot reproduce the issue because ActiveData seems to be down.
This might be because of previous ActiveData outage when the task ran.

The basic idea is to write a clang plugin which is able to detect if a function in an uncovered file is called (directly or via a call to another function) by an other function in a covered file.
That would avoid manual inspection and give the files with a high probability to be dead.

STR:

1. Go to https://mozilla.github.io/code-coverage-reports/zero_coverage_report.html#servo/components/style
2. Click on the checkbox "Show completely uncovered files only"
3. Copy the url and open a new page with this url => the checked checkbox is lost

It could be useful when sharing urls.

A coverage request on the changeset at tip (or any changeset between the last push & tip) will give a 404.

Right now the coverage backend only looks for coverage data after the requested changeset. But that does not work in the case of tip, when no coverage data is yet available.

For that case (when changeset's push is above last changeset) the backend should fall back to the previous coverage data

Frontend part is at https://github.com/marco-c/code-coverage-reports/issues/23.

Right now, for example on hg.mozilla.org, if there is no coverage information yet, the button shows up red. It should show up as some other color than red in this case.

codetribute is still using the URLs of the old repos.

Today, we analyze per-push mozilla-central code coverage reports to identify new code that isn't covered by any tests, and automatically send an email report to a few people, with links to the relevant commits.

This allows manually filing bugs when new code is obviously lacking tests, and ask the developers to add tests.

But the process is very manual and tedious, and the reports (email and coverage view) are lacking important information to make pertinent decisions. It would be nice to improve these reports with more useful information, and to make them actionable (e.g. with a button to auto-file a blocking bug).

Code locations:

• The reports are generated in shipit_code_coverage/notifier.py
• The coverage viewer is hosted at firefox-code-coverage.herokuapp.com and its source code is under mozilla/firefox-code-coverage-frontend

Information to add:

• Patch author in the changeset view (see mozilla/firefox-code-coverage-frontend#170)
• Modified files paths in the email (without making the email unreadable)

Actions to automate or streamline:

• Pre-fill a blocking bug that asks the developer to add tests (similar to Talos perf regression blocking bugs)

Tests should be written in src/codecoverage/bot/tests/test_suite_reports.py for the generate function from the suite_reports module (src/codecoverage/bot/code_coverage_bot/suite_reports.py).

Currently, we're downloading grcov from GitHub releases. This means it won't run on one-click loaners from mozilla-central build/test machines, as they have an old version of libc.

Right now we have one hook which is then doing different things according to the input. We should instead have separate hooks.

Dependabot couldn't fetch one or more of your project's path-based Python dependencies. The affected dependencies were bot/[email protected]:mozilla/code-coverage.git@85f62f11cd712e41e8a53668976af73d400cee11/setup.py.

To use path-based dependencies with Dependabot the paths must be relative and resolve to a directory in this project's source code.

You can mention @dependabot in the comments below to contact the Dependabot team.

Dependabot couldn't fetch one or more of your project's path-based Python dependencies. The affected dependencies were bot/[email protected]:mozilla/code-coverage.git@85f62f11cd712e41e8a53668976af73d400cee11/setup.py.

To use path-based dependencies with Dependabot the paths must be relative and resolve to a directory in this project's source code.

You can mention @dependabot in the comments below to contact the Dependabot team.

We could copy (with minor modifications) the one from bugbug (like I'm doing for autonag).

Add android as a platform here and write queries to get the chunk mapping for the A(test-ccov) job.

Most of the failures are due to transient issues, we should implement some kind of retry mechanism.

Does it apply to static analysis too, @La0 @jankeromnes?

Coverage for generated files can still be interesting. We were dropping it because codecov/coveralls wouldn't show them anyway, but now that we have our own backend we could keep it.

They are no longer up to date.
the backend is using https://shipit.mozilla-releng.net.

This way on the frontend we don't have to check the type.

The zero coverage reports generated on the bot use the coveralls+ format, which should not be used anymore.

Dependabot couldn't fetch one or more of your project's path-based Python dependencies. The affected dependencies were bot/[email protected]:mozilla/code-coverage.git@85f62f11cd712e41e8a53668976af73d400cee11/setup.py.

To use path-based dependencies with Dependabot the paths must be relative and resolve to a directory in this project's source code.

You can mention @dependabot in the comments below to contact the Dependabot team.

We should allow filtering by OS (Windows, Linux, Android, Mac).

The Mercurial commits take a while to get synced to gecko-dev, so we should trigger coverage jobs after a timeout of ~30 minutes (instead of waiting in the coverage task).

E.g. it puts package.json and package-lock.json in the zip. There's probably some way to select files to ignore.

https://bugzilla.mozilla.org/show_bug.cgi?id=1428470

The suite and chunk info are already in the task data, we don't need to parse them from the task name:

   {
      "status": {
        "taskId": "CkICVuBvRr--6rratJoG1A",
        "provisionerId": "aws-provisioner-v1",
        "workerType": "gecko-t-linux-large",
        "schedulerId": "gecko-level-3",
        "taskGroupId": "e4wR3JiKRmCqFXMG1sZoqw",
        "deadline": "2018-01-16T14:08:30.100Z",
        "expires": "2019-01-15T14:08:30.100Z",
        "retriesLeft": 5,
        "state": "completed",
        "runs": [
          {
            "runId": 0,
            "state": "completed",
            "reasonCreated": "scheduled",
            "reasonResolved": "completed",
            "workerGroup": "us-west-2",
            "workerId": "i-02f880a62021594ec",
            "takenUntil": "2018-01-15T15:18:41.003Z",
            "scheduled": "2018-01-15T14:58:40.472Z",
            "started": "2018-01-15T14:58:41.080Z",
            "resolved": "2018-01-15T15:04:13.243Z"
          }
        ]
      },
      "task": {
        "provisionerId": "aws-provisioner-v1",
        "workerType": "gecko-t-linux-large",
        "schedulerId": "gecko-level-3",
        "taskGroupId": "e4wR3JiKRmCqFXMG1sZoqw",
        "dependencies": [
          "HyAoNcNPRvyV6mgzFFFWSg",
          "XUDNfri0ReazT-7gyTTAYg"
        ],
        "requires": "all-completed",
        "routes": [
          "tc-treeherder.v2.mozilla-central.a00e3a3dbad43759ee9474d6c7c908fa16deaee8.33255",
          "coalesce.v1.mozilla-central.71fe57dadaef770d25ba"
        ],
        "priority": "medium",
        "retries": 5,
        "created": "2018-01-15T14:08:30.100Z",
        "deadline": "2018-01-16T14:08:30.100Z",
        "expires": "2019-01-15T14:08:30.100Z",
        "scopes": [
          "secrets:get:project/taskcluster/gecko/hgfingerprint",
          "docker-worker:feature:allowPtrace",
          "docker-worker:cache:level-3-mozilla-central-test-workspace-bc7e1a7ad01a345394f1",
          "docker-worker:cache:level-3-checkouts-bc7e1a7ad01a345394f1"
        ],
        "payload": {
          "supersederUrl": "https://coalesce.mozilla-releng.net/v1/list/3600/5/mozilla-central.71fe57dadaef770d25ba",
          "onExitStatus": {
            "retry": [
              4
            ]
          },
          "maxRunTime": 3600,
          "image": {
            "path": "public/image.tar.zst",
            "type": "task-image",
            "taskId": "XUDNfri0ReazT-7gyTTAYg"
          },
          "cache": {
            "level-3-mozilla-central-test-workspace-bc7e1a7ad01a345394f1": "/builds/worker/workspace",
            "level-3-checkouts-bc7e1a7ad01a345394f1": "/builds/worker/checkouts"
          },
          "artifacts": {
            "public/logs/": {
              "path": "/builds/worker/workspace/build/upload/logs/",
              "expires": "2019-01-15T14:08:30.100Z",
              "type": "directory"
            },
            "public/test": {
              "path": "/builds/worker/artifacts/",
              "expires": "2019-01-15T14:08:30.100Z",
              "type": "directory"
            },
            "public/test_info/": {
              "path": "/builds/worker/workspace/build/blobber_upload_dir/",
              "expires": "2019-01-15T14:08:30.100Z",
              "type": "directory"
            }
          },
          "command": [
            "/builds/worker/bin/run-task",
            "--",
            "/builds/worker/bin/test-linux.sh",
            "--no-read-buildbot-config",
            "--installer-url=https://queue.taskcluster.net/v1/task/HyAoNcNPRvyV6mgzFFFWSg/artifacts/public/build/target.tar.bz2",
            "--test-packages-url=https://queue.taskcluster.net/v1/task/HyAoNcNPRvyV6mgzFFFWSg/artifacts/public/build/target.test_packages.json",
            "--reftest-suite=reftest-no-accel",
            "--e10s",
            "--allow-software-gl-layers",
            "--total-chunk=8",
            "--this-chunk=7",
            "--download-symbols=ondemand"
          ],
          "env": {
            "SCCACHE_DISABLE": "1",
            "MOZ_NODE_PATH": "/usr/local/bin/node",
            "TASKCLUSTER_CACHES": "/builds/worker/checkouts;/builds/worker/workspace",
            "HG_STORE_PATH": "/builds/worker/checkouts/hg-store",
            "GECKO_HEAD_REV": "a00e3a3dbad43759ee9474d6c7c908fa16deaee8",
            "GECKO_HEAD_REPOSITORY": "https://hg.mozilla.org/mozilla-central",
            "TASKCLUSTER_VOLUMES": "/builds/worker/.cache;/builds/worker/checkouts;/builds/worker/tooltool-cache;/builds/worker/workspace",
            "MOZHARNESS_URL": "https://queue.taskcluster.net/v1/task/HyAoNcNPRvyV6mgzFFFWSg/artifacts/public/build/mozharness.zip",
            "NEED_PULSEAUDIO": "true",
            "MOZ_AUTOMATION": "1",
            "NEED_WINDOW_MANAGER": "true",
            "MOZHARNESS_CONFIG": "unittests/linux_unittest.py remove_executables.py",
            "ENABLE_E10S": "true",
            "MOZHARNESS_SCRIPT": "desktop_unittest.py",
            "GECKO_BASE_REPOSITORY": "https://hg.mozilla.org/mozilla-unified",
            "MOZILLA_BUILD_URL": "https://queue.taskcluster.net/v1/task/HyAoNcNPRvyV6mgzFFFWSg/artifacts/public/build/target.tar.bz2"
          },
          "features": {
            "taskclusterProxy": true,
            "allowPtrace": true
          }
        },
        "metadata": {
          "owner": "[email protected]",
          "source": "https://hg.mozilla.org/mozilla-central/file/a00e3a3dbad43759ee9474d6c7c908fa16deaee8/taskcluster/ci/test",
          "description": "Reftest not accelerated run ([Treeherder push](https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=a00e3a3dbad43759ee9474d6c7c908fa16deaee8))",
          "name": "test-linux32/opt-reftest-no-accel-e10s-7"
        },
        "tags": {
          "kind": "test",
          "worker-implementation": "docker-worker",
          "createdForUser": "[email protected]",
          "label": "test-linux32/opt-reftest-no-accel-e10s-7",
          "test-type": "reftest",
          "os": "linux"
        },
        "extra": {
          "chunks": {
            "current": 7,
            "total": 8
          },
          "suite": {
            "flavor": "reftest-no-accel",
            "name": "reftest"
          },
          "treeherder": {
            "jobKind": "test",
            "groupSymbol": "tc-R-e10s",
            "collection": {
              "opt": true
            },
            "machine": {
              "platform": "linux32"
            },
            "groupName": "Reftests executed by TaskCluster with e10s",
            "tier": 1,
            "symbol": "Ru7"
          },
          "parent": "e4wR3JiKRmCqFXMG1sZoqw",
          "index": {
            "rank": 1516025190
          }
        }
      }
    },

Right now the bot fails when it encounters a secure revision:

  File "/nix/store/c94zc39953i838di2kppybjrg37k2yyn-python3.7-mozilla-codecoverage-bot-1.0.0/lib/python3.7/site-packages/code_coverage_bot/codecov.py", line 150, in go_from_trigger_mozilla_central
    phabricatorUploader.upload(json.loads(output))
  File "/nix/store/c94zc39953i838di2kppybjrg37k2yyn-python3.7-mozilla-codecoverage-bot-1.0.0/lib/python3.7/site-packages/code_coverage_bot/phabricator.py", line 137, in upload
    rev_data = phabricator.load_revision(rev_id=rev_id)
  File "/nix/store/4g87vc35ilp7ax0rn8c1zdc6d0an7fab-python3.7-python3.7-mozilla-cli-common-1.0.0/lib/python3.7/site-packages/cli_common/phabricator.py", line 152, in load_revision
    'Revision not found'
AssertionError: Revision not found