roachtest.rebalance/by-load/leases/mixed-version <a href="https://teamcity.cockroachdb

<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clip

This is a similar issue to <a class="issue-link js-issue-link" data-error-text="Failed

roachtest: rebalance/by-load/leases/mixed-version failed about cockroach HOT 4 CLOSED

cockroach-teamcity commented on August 26, 2024

roachtest: rebalance/by-load/leases/mixed-version failed

from cockroach.

Comments (4)

kvoli commented on August 26, 2024

mean=69.0 tolerance=20.0% (±13.8) bounds=[55.2, 82.8]
stores=[s1: 66 (-4.1%), s2: 78 (+14.4%), s3: 61 (-10.3%)]

It appears all stores are within the bounds, might be a testing issue.

from cockroach.

kvoli commented on August 26, 2024

The test is unable to gather the CPU usage information, causing the test to timeout before balancing is achieved (or noticed to be achieved).

06:34:15 cluster.go:2429: running cmd `./cockroach auth-session lo...` on nodes [:4]; details in run_063415.317963092_n4_cockroach-authsessio.log
06:34:15 httpclient.go:210: ./cockroach auth session login failed on node 4: failed to authenticate: COMMAND_PROBLEM: exit status 1

This may be an issue with mixed version secure clusters and the roachtest framework?

run_063413.087691684_n1_cockroach-authsessio: 06:34:13 cluster.go:2432: > ./cockroach auth-session login root --port={pgport:1} --certs-dir ./certs --format raw
teamcity-14772140-1712728228-03-n4cpu4:[1]: ./cockroach auth-session lo...
ERROR: connection lost.

EOF
Failed running "auth-session login"
run_063413.087691684_n1_cockroach-authsessio: 06:34:13 cluster.go:2457: > result: Error for Node 1: COMMAND_PROBLEM: exit status 1
(1) Node 1. Command with error:
  | ```
  | ./cockroach auth-session login root --port={pgport:1} --certs-dir ./certs --format raw
  | ```
  | stdout: <empty>
  | stderr:ERROR: connection lost.
  |
  | EOF
  | Failed running "auth-session login"
Wraps: (2) COMMAND_PROBLEM
Wraps: (3) exit status 1
Error types: (1) *hintdetail.withDetail (2) errors.Cmd (3) *exec.ExitError

from cockroach.

kvoli commented on August 26, 2024

This is a similar issue to #119064. The fix there was to run the test in insecure mode. Should we do the same here @DarrylWong?

from cockroach.