FYI: The following changes were made to this repository's wiki:

• defacing spam has been removed

• Restricting write access to contributors is strongly encouraged. Please make that change (documentation).

These were made as the result of a recent automated defacement of publically writeable wikis.

There's a persistent crash with signature xul.dll@0x******* | _PR_NativeRunThread | pr_root", where the address changes each build. It would be nice if we could avoid considering each of them as a new one (at least in this case where foo.dll@0xdeadbeef is not the whole signature).

Now that fennec is replaced by fenix, we should update this to monitor the new product in crash-stats.

cc @pascalchevrel @rvandermeulen

Some signatures such as:

IPCError-browser | ShutDownKill
EMPTY: no crashing thread identified; ERROR_NO_THREAD_LIST
EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER

Are signatures that are not actionable and probably shouldn't be included in the report when they spike. The ShutDownKill signature, for example, can have 33733 crashes in two days, and seems to be on the report quite a bit.

While looking at this email, I noticed a few signatures such as https://bit.ly/2RLFPEY that appears to be crashes on older versions of nightly, dating back to Firefox 29? Is there a way to filter nightly crashes so we only see the most recent nightly.

There seems to be a few other signatures in that email that also seem to be crashes on older versions of nightly.

Right now, it looks like the spike report gets mailed out shortly after 5pm PST, which is the end of the standard working day in California. If the spike report went out a little earlier, say at 4:30pm or 4:45pm, I (and other people working on California time) could file bugs as appropriate right after the report comes up, and presumably the amount of data that comes in during that period isn't much so the quality would not be affected.

As of January 1 2019, Mozilla requires that all GitHub projects include this CODE_OF_CONDUCT.md file in the project root. The file has two parts:

1. Required Text - All text under the headings Community Participation Guidelines and How to Report, are required, and should not be altered.
2. Optional Text - The Project Specific Etiquette heading provides a space to speak more specifically about ways people can work effectively and inclusively together. Some examples of those can be found on the Firefox Debugger project, and Common Voice. (The optional part is commented out in the raw template file, and will not be visible until you modify and uncomment that part.)

If you have any questions about this file, or Code of Conduct policies and procedures, please see Mozilla-GitHub-Standards or email [email protected].

(Message COC001)

The release-mgmt lists hasn't see Spikes in signatures in nightly the YYYY-MM-DD emails after the one for 2021-03-18.

@calixteman Looks like you are already investigating based on your commits after it broke.

cc @rvandermeulen @jcristau @pascalchevrel

May not be a way to do this, but thought I would mention it in light of the mac spike we had: http://bit.ly/2t4SIm5

For spikes in nightly, could we have a threshold for spikes that we think might need immediate attention? Sometimes we may want to spin a new Nightly as fast as possible (including over weekends). We could change the subject line of the email to include something like "URGENT".

For a threshold, I know it's complicated, but to start with, maybe have an increase in crashes by install (by signature ) more than 300 trigger the "urgent" status. If that's too low, we can fine tune it later.

It's so useful to have a link to the code repo actually in the emails (otherwise I so easily forget which tool sends them).

We're opening this issue because your project has used Travis CI within the last 6 months. If you have already migrated off it, you can close and ignore this issue.

Travis CI is ending free builds on public repositories. travis-ci.com stopped providingthem in early November, and travis-ci.org will stop after December 31, 2020. To avoid disruptions to your workflows, you must migrate to another CI service.

For production use cases, we recommend switching to CircleCI. This service is already widely used within Mozilla. There is a guide to migrating from Travis CI to CircleCI available here.

For non production use cases, we recommend either CircleCI or Github Actions. There is a guide to migrating from Travis CI to Github Actions available here. Github Actions usage within Mozilla is new, and you will have to work with our github administrators to enable specific actions following this process.

If you have any questions, reach out in #github-admin:mozilla.org on matrix.