roachtest: loqrecovery/workload=movr/rangeSize=default failed about cockroach HOT 6 CLOSED

cc @cockroachdb/test-eng

from cockroach.

Strange issue. I think last time we saw something similar, we fixed it with #122158. But this might be something else... 🤔

from cockroach.

cc @cockroachdb/test-eng

from cockroach.

It was able to still find all the other clusters so i think this is a different issue than i.e. #121489 or any of the Azure issues.

It was a cluster that was reused multiple times just fine, so shouldn't be anything to do with eventual consistency?

I also notice that in the gce and test runner logs, the cluster was destroyed just fine one minute after the test failed by the test runner. In the previous issues, since the cluster isn't found in the cache, it's left behind and GC needs to clean it up. So the entry was only missing for a very brief moment.

The file locking for the cache seems sound to me so not sure if there's anything else I can look into without a repro 🤷‍♂️

from cockroach.

Might be a ghost... should we create a tracking issue and see when and if this comes up again?

from cockroach.

Created #125714 to keep track of this and to group similar issues if they happen in the future. Closing.

from cockroach.