Upgrading to NX 19 causes CI to fail with `The Nx Cloud heartbeat process failed to report its status in time` about nx HOT 3 OPEN

@eamon0989 I just encountered this issue today. However, I upgraded my Angular application to Angular 18 / NX 19.3.0 about 3 weeks ago and didn't have any issues deploying until today (no changes to the pipeline or anything).

Adding the --require-explicit-completion to our pipeline seems to have fixed the issue. However, I do wonder why it started happening out of the blue (after 50+ successful builds on Angular 18 / NX 19.3.0).

from nx.

Hi @eamon0989,

While this might not be directly related to your issue, your nx report shows duplicate packages - you have both @nx/* in version 19 and @nrwl/* is version 18 installed. This might be a sign of manual package update being done sometime in the past.

Can you remove all the @nrwl/* packages and leave just nx and @nx/* packages? You should be using yarn nx migrate latest whenever your need to migrate to latest version. This not only bumps the versions of packages but also automatically runs any necessary migrations scripts.

from nx.

An update for anyone that comes across this issue in the future.

First, the duplicate nx packages: we have always updated nx using npx nx migrate latest, never manually.

I've taken a look into the duplicate nx/nrwl dependencies, and it seems they come from devDependencies of some of our dependencies. Our package.json only contains the latest versions of the packages of nx, the older and duplicate versions come from dependencies. I was not aware of this, but apparently it is the default behaviour of yarn, which we use as a package manager - see https://stackoverflow.com/questions/49530678/why-does-yarn-install-dev-dependencies-but-i-just-need-the-builds.

As for the heartbeat issue, I contacted nx support and they recommended adding the flag --require-explicit-completion to nx-cloud start-ci-run. The ci still failed, and I was given this suggestion:

Could you please verify that in the full pipeline, you don't run any Nx targets before the npx nx-cloud start-ci-run --require-explicit-completion?

For example, I see that the main job needs [init, api-specs, regenerate]. If any of those other jobs are running Nx commands first, then they will create the nx-cloud run automatically first, without the --require-explicit-completion option. The fix for this is to make sure the npx nx-cloud start-ci-run --require-explicit-completion command happens before all Nx targets are run in the pipeline.

I commented out the jobs that ran before our main job and the ci passed, it turns out that one of our earlier jobs was using an nx target, causing the nx-cloud to be created. I'm guessing it was nx format:write.

TLDR: the issue was caused by calling an nx target in a previous job which was starting nx-cloud before we expected it to be started.

Thanks to nx support for helping me debug the issue!

from nx.