azure / cyclecloud-nodehealth Goto Github PK

According to nhc logging /var/log/nhc.log, there is an error but hcheck/send_log is not sending it to the cyclecloud portal.

tail /var/log/nhc.log
Running check: "check_hw_physmem 928858388kB 928858388kB 3%"
Running check: "check_hw_swap 0kB 0kB 3%"
Running check: "check_hw_ib 40 mlx5_an0:1"
ERROR: nhc: Health check failed: check_hw_ib: No IB port mlx5_an0:1 is ACTIVE (LinkUp 40 Gb/sec).

In NHC DETACHED_MODE, hcheck creates /var/log/healthcheck.json, but it reports no errors (exit-code is 0)?
{
"metadata": {
"azEnvironment": "AzurePublicCloud",
"isHostCompatibilityLayerVm": "false",
"location": "southcentralus",
"name": "xxxxx",
"offer": "ubuntu-hpc",
"osType": "Linux",
"platformFaultDomain": "0",
"platformUpdateDomain": "1",
"provider": "Microsoft.Compute",
"publisher": "microsoft-dsvm",
"resourceGroupName": "slm27u2004hc-xxxx",
"resourceId": "/subscriptions/xxxxx",
"subscriptionId": "xxxx",
"version": "20.04.2022121201",
"vmId": "xxxxx",
"vmScaleSetName": "xxxxx",
"vmSize": "Standard_ND96asr_v4",
"zone": ""
},
"testresults": {
"/opt/cycle/nodehealth/nhc-runner.sh": {
"exit-code": 0,
"test-time": 155.6628,
"extra-info": "None",
"message": "Node Health Check detached parent completed successfully (0s).\n"
}
}
}

I think the problem is

hpcadmin@slm27u2004hc-hpc-pg0-1:/opt/cycle/nodehealth$ sudo ./nhc-runner.sh

• getopts L x
• OPTIND=0
• nhc NHC_CHECK_ALL=0 LOGFILE=- MARK_OFFLINE=0
  Node Health Check detached parent completed successfully (0s).
  + exit 0

Also the hcheck tool does not seem to work when nhc is running in DETACHED_MODE=0 (not detached mode), I get this error message "There was an error in launching the script: Script ./nhc-runner.sh timed out.". ?