Comments (7)
处理中
from matrixone.
repro:https://github.com/matrixorigin/matrixone/actions/runs/8986398280/job/24682403725?pr=15880
from matrixone.
https://github.com/matrixorigin/matrixone/actions/runs/9050329457/job/24865567531?pr=16040
from matrixone.
repor: https://github.com/matrixorigin/matrixone/actions/runs/9057918105/job/24882671908?pr=16046
2024-05-13T06:24:01.2595852Z === RUN TestLockSuccWithKeepBindTimeout
2024-05-13T06:24:01.2597219Z 2024/05/13 06:02:41.239813 +0000 DEBUG cn-service.lockservice.lock-server application accept loop started {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/service-237485116-s1.sock]"}
2024-05-13T06:24:01.2599013Z 2024/05/13 06:02:41.240189 +0000 DEBUG cn-service.lockservice.lock-server application accept loop stopped {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/service-237485116-s1.sock]"}
2024-05-13T06:24:01.2600722Z 2024/05/13 06:02:41.240303 +0000 DEBUG cn-service.lockservice.lock-server application started {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/service-237485116-s1.sock]"}
2024-05-13T06:24:01.2601969Z 2024/05/13 06:02:41.239927 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-13T06:24:01.2603053Z 2024/05/13 06:02:41.239931 +0000 INFO cn-service.lockservice.lock-client.lock-client gc idle backends task started {"uuid": ""}
2024-05-13T06:24:01.2604143Z 2024/05/13 06:02:41.239977 +0000 INFO cn-service.lockservice keep lock table bind task {"uuid": ""}
2024-05-13T06:24:01.2605038Z 2024/05/13 06:02:41.240013 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-13T06:24:01.2605935Z 2024/05/13 06:02:41.240045 +0000 INFO cn-service.lockservice keep remote locks task {"uuid": ""}
2024-05-13T06:24:01.2606819Z 2024/05/13 06:02:41.240065 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-13T06:24:01.2607671Z 2024/05/13 06:02:41.240101 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-13T06:24:01.2608767Z 2024/05/13 06:02:41.240094 +0000 DEBUG cn-service.lockservice.lock-client.lock-client gc inactive backends task started {"uuid": ""}
2024-05-13T06:24:01.2610077Z 2024/05/13 06:02:41.240771 +0000 INFO cn-service.lockservice lock service start successfully {"uuid": "", "serviceID": "1715580161237546550s1"}
2024-05-13T06:24:01.2611381Z 2024/05/13 06:02:41.241038 +0000 DEBUG cn-service.lockservice.lock-client.lock-client gc inactive backends task started {"uuid": ""}
2024-05-13T06:24:01.2612793Z 2024/05/13 06:02:41.241046 +0000 INFO cn-service.lockservice.lock-client.lock-client gc idle backends task started {"uuid": ""}
2024-05-13T06:24:01.2613950Z 2024/05/13 06:02:41.241212 +0000 INFO cn-service.lockservice.allocator clean cannot commit task {"uuid": ""}
2024-05-13T06:24:01.2615345Z 2024/05/13 06:02:41.246064 +0000 DEBUG cn-service.lockservice.lock-server application accept loop started {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/237385731.sock]"}
2024-05-13T06:24:01.2617017Z 2024/05/13 06:02:41.246158 +0000 DEBUG cn-service.lockservice.lock-server application accept loop stopped {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/237385731.sock]"}
2024-05-13T06:24:01.2618626Z 2024/05/13 06:02:41.246203 +0000 DEBUG cn-service.lockservice.lock-server application started {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/237385731.sock]"}
2024-05-13T06:24:01.2619890Z 2024/05/13 06:02:41.246296 +0000 INFO cn-service.lockservice lock allocator start successfully {"uuid": ""}
2024-05-13T06:24:01.2620903Z 2024/05/13 06:02:41.246379 +0000 DEBUG cn-service.lockservice txn created {"uuid": "", "txn": "74786e31()"}
2024-05-13T06:24:01.2622350Z 2024/05/13 06:02:41.247118 +0000 DEBUG cn-service.lockservice.lock-client start connect to remote {"uuid": "", "remote": "unix:///tmp/237385731.sock", "backend-id": "018f708b-ccde-77e6-98bb-a9a6202332f0"}
2024-05-13T06:24:01.2623927Z 2024/05/13 06:02:41.247372 +0000 DEBUG cn-service.lockservice.lock-server session init completed {"uuid": "", "name": "lock-server", "session-id": 1}
2024-05-13T06:24:01.2625622Z 2024/05/13 06:02:41.247483 +0000 DEBUG cn-service.lockservice.lock-server session connected {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/237385731.sock]", "session-id": 1, "addr": "@"}
2024-05-13T06:24:01.2627489Z 2024/05/13 06:02:41.247606 +0000 DEBUG cn-service.lockservice.lock-client session init completed {"uuid": "", "remote": "unix:///tmp/237385731.sock", "backend-id": "018f708b-ccde-77e6-98bb-a9a6202332f0", "session-id": 0}
2024-05-13T06:24:01.2629444Z 2024/05/13 06:02:41.247688 +0000 DEBUG cn-service.lockservice.lock-client connect to remote succeed {"uuid": "", "remote": "unix:///tmp/237385731.sock", "backend-id": "018f708b-ccde-77e6-98bb-a9a6202332f0"}
2024-05-13T06:24:01.2631221Z 2024/05/13 06:02:41.247781 +0000 DEBUG cn-service.lockservice.lock-client read loop started {"uuid": "", "remote": "unix:///tmp/237385731.sock", "backend-id": "018f708b-ccde-77e6-98bb-a9a6202332f0"}
2024-05-13T06:24:01.2632960Z 2024/05/13 06:02:41.247919 +0000 DEBUG cn-service.lockservice.lock-client write loop started {"uuid": "", "remote": "unix:///tmp/237385731.sock", "backend-id": "018f708b-ccde-77e6-98bb-a9a6202332f0"}
2024-05-13T06:24:01.2634988Z 2024/05/13 06:02:41.248054 +0000 DEBUG cn-service.lockservice.lock-client write request {"uuid": "", "remote": "unix:///tmp/237385731.sock", "backend-id": "018f708b-ccde-77e6-98bb-a9a6202332f0", "request-id": 1, "request": "1: GetBind/1715580161237546550s1-0"}
2024-05-13T06:24:01.2637180Z 2024/05/13 06:02:41.248219 +0000 DEBUG cn-service.lockservice.lock-server session read message {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/237385731.sock]", "session-id": 1, "addr": "@", "seqence": 1}
2024-05-13T06:24:01.2639113Z 2024/05/13 06:02:41.248295 +0000 DEBUG cn-service.lockservice.lock-server received request {"uuid": "", "name": "lock-server", "sequence": 1, "client": "@", "request-id": 1, "request": "1: GetBind/1715580161237546550s1-0"}
2024-05-13T06:24:01.2640644Z 2024/05/13 06:02:41.248398 +0000 DEBUG cn-service.lockservice received a request {"uuid": "", "request": "1: GetBind/1715580161237546550s1-0"}
2024-05-13T06:24:01.2642125Z 2024/05/13 06:02:41.248444 +0000 DEBUG cn-service.lockservice.lock-server handle request completed {"uuid": "", "name": "lock-server", "sequence": 1, "client": "@", "request-id": 1}
2024-05-13T06:24:01.2643674Z 2024/05/13 06:02:41.249297 +0000 INFO cn-service.lockservice.allocator bind created {"uuid": "", "table": 0, "service": "1715580161237546550s1"}
2024-05-13T06:24:01.2645080Z 2024/05/13 06:02:41.249362 +0000 DEBUG cn-service.lockservice handle request completed {"uuid": "", "response": "1: GetBind/0-1715580161237546550s1-1"}
2024-05-13T06:24:01.2646661Z 2024/05/13 06:02:41.249440 +0000 DEBUG cn-service.lockservice.lock-server write responses {"uuid": "", "name": "lock-server", "client": "@", "request-id": 1, "response": "1: GetBind/0-1715580161237546550s1-1"}
2024-05-13T06:24:01.2648708Z 2024/05/13 06:02:41.249595 +0000 DEBUG cn-service.lockservice.lock-client read response {"uuid": "", "remote": "unix:///tmp/237385731.sock", "backend-id": "018f708b-ccde-77e6-98bb-a9a6202332f0", "request-id": 1, "response": "1: GetBind/0-1715580161237546550s1-1"}
2024-05-13T06:24:01.2650561Z 2024/05/13 06:02:41.249749 +0000 INFO cn-service.lockservice bind created {"uuid": "", "service": "1715580161237546550s1", "remote": false, "bind": "0-1715580161237546550s1-1"}
2024-05-13T06:24:01.2652051Z 2024/05/13 06:02:41.249846 +0000 DEBUG cn-service.lockservice try to lock on local {"uuid": "", "txn": "74786e31()", "table": 0, "rows": "[01]", "opts": "Exclusive-Row-Wait"}
2024-05-13T06:24:01.2653744Z 2024/05/13 06:02:41.249947 +0000 DEBUG cn-service.lockservice holder added {"uuid": "", "txn": "74786e31()", "table": 0, "rows": "[01]", "opts": "Exclusive-Row-Wait", "holders": "[74786e31(1715580161237546550s1)]", "waiters": "[]"}
2024-05-13T06:24:01.2655213Z 2024/05/13 06:02:41.250040 +0000 DEBUG cn-service.lockservice lock added to txn {"uuid": "", "txn": "74786e31()", "rows": "[01]"}
2024-05-13T06:24:01.2656635Z 2024/05/13 06:02:41.250119 +0000 DEBUG cn-service.lockservice lock added to local {"uuid": "", "txn": "74786e31()", "table": 0, "rows": "[01]", "opts": "Exclusive-Row-Wait"}
2024-05-13T06:24:01.2657922Z 2024/05/13 06:02:41.250234 +0000 INFO cn-service.lockservice keep lock table bind task {"uuid": "", "cost": "9.735208ms"}
2024-05-13T06:24:01.2659123Z 2024/05/13 06:02:41.250317 +0000 INFO cn-service.lockservice keep remote locks task {"uuid": "", "cost": "9.791783ms"}
2024-05-13T06:24:01.2660027Z panic: missing free for type: *lockservice.waiter, 02-0xc000000900(1)
2024-05-13T06:24:01.2660547Z create by: <<<>>>
2024-05-13T06:24:01.2660721Z
2024-05-13T06:24:01.2660726Z
2024-05-13T06:24:01.2660862Z goroutine 5 [running]:
2024-05-13T06:24:01.2661470Z github.com/matrixorigin/matrixone/pkg/common/reuse.(*checker[...]).gc(0x1215c40, 0xc000000900)
2024-05-13T06:24:01.2662357Z /home/runner/work/matrixone/matrixone/pkg/common/reuse/checker.go:133 +0x394
2024-05-13T06:24:01.2663177Z github.com/matrixorigin/matrixone/pkg/common/reuse.newSyncPoolBased[...].func1.1()
2024-05-13T06:24:01.2664054Z /home/runner/work/matrixone/matrixone/pkg/common/reuse/sync_pool_based.go:46 +0xc5
2024-05-13T06:24:01.2664831Z FAIL github.com/matrixorigin/matrixone/pkg/lockservice 184.230s
from matrixone.
new case: TestReLockSuccWithLockTableBindChanged, same error.
https://github.com/matrixorigin/matrixone/actions/runs/9059936309/job/24888593773?pr=16067
2024-05-13T09:15:25.7200138Z === RUN TestReLockSuccWithLockTableBindChanged
2024-05-13T09:15:25.7200622Z 2024/05/13 08:54:18.119309 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-13T09:15:25.7201020Z 2024/05/13 08:54:18.119440 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-13T09:15:25.7201453Z 2024/05/13 08:54:18.119600 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-13T09:15:25.7201851Z 2024/05/13 08:54:18.119674 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-13T09:15:25.7202465Z 2024/05/13 08:54:18.119811 +0000 INFO cn-service.lockservice.lock-client.lock-client gc idle backends task started {"uuid": ""}
2024-05-13T09:15:25.7203098Z 2024/05/13 08:54:18.119968 +0000 DEBUG cn-service.lockservice.lock-client.lock-client gc inactive backends task started {"uuid": ""}
2024-05-13T09:15:25.7203545Z 2024/05/13 08:54:18.120739 +0000 INFO cn-service.lockservice keep lock table bind task {"uuid": ""}
2024-05-13T09:15:25.7204036Z 2024/05/13 08:54:18.120927 +0000 INFO cn-service.lockservice keep remote locks task {"uuid": ""}
2024-05-13T09:15:25.7204945Z 2024/05/13 08:54:18.121109 +0000 DEBUG cn-service.lockservice.lock-server application accept loop started {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/service-105352713-s1.sock]"}
2024-05-13T09:15:25.7205905Z 2024/05/13 08:54:18.126031 +0000 DEBUG cn-service.lockservice.lock-server application accept loop stopped {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/service-105352713-s1.sock]"}
2024-05-13T09:15:25.7206740Z 2024/05/13 08:54:18.126084 +0000 DEBUG cn-service.lockservice.lock-server application started {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/service-105352713-s1.sock]"}
2024-05-13T09:15:25.7207396Z 2024/05/13 08:54:18.126409 +0000 INFO cn-service.lockservice lock service start successfully {"uuid": "", "serviceID": "1715590458105442711s1"}
2024-05-13T09:15:25.7207791Z 2024/05/13 08:54:18.127983 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-13T09:15:25.7208195Z 2024/05/13 08:54:18.134072 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-13T09:15:25.7208588Z 2024/05/13 08:54:18.134173 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-13T09:15:25.7208997Z 2024/05/13 08:54:18.134329 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-13T09:15:25.7209603Z 2024/05/13 08:54:18.134520 +0000 INFO cn-service.lockservice.lock-client.lock-client gc idle backends task started {"uuid": ""}
2024-05-13T09:15:25.7210247Z 2024/05/13 08:54:18.134683 +0000 DEBUG cn-service.lockservice.lock-client.lock-client gc inactive backends task started {"uuid": ""}
2024-05-13T09:15:25.7210677Z 2024/05/13 08:54:18.135473 +0000 INFO cn-service.lockservice keep remote locks task {"uuid": ""}
2024-05-13T09:15:25.7211116Z 2024/05/13 08:54:18.135687 +0000 INFO cn-service.lockservice keep lock table bind task {"uuid": ""}
2024-05-13T09:15:25.7212080Z 2024/05/13 08:54:18.135894 +0000 DEBUG cn-service.lockservice.lock-server application accept loop started {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/service-105390654-s2.sock]"}
2024-05-13T09:15:25.7212979Z 2024/05/13 08:54:18.135976 +0000 DEBUG cn-service.lockservice.lock-server application accept loop stopped {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/service-105390654-s2.sock]"}
2024-05-13T09:15:25.7213815Z 2024/05/13 08:54:18.136044 +0000 DEBUG cn-service.lockservice.lock-server application started {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/service-105390654-s2.sock]"}
2024-05-13T09:15:25.7214465Z 2024/05/13 08:54:18.136389 +0000 INFO cn-service.lockservice lock service start successfully {"uuid": "", "serviceID": "1715590458126536645s2"}
2024-05-13T09:15:25.7215074Z 2024/05/13 08:54:18.136651 +0000 INFO cn-service.lockservice.lock-client.lock-client gc idle backends task started {"uuid": ""}
2024-05-13T09:15:25.7215710Z 2024/05/13 08:54:18.136807 +0000 DEBUG cn-service.lockservice.lock-client.lock-client gc inactive backends task started {"uuid": ""}
2024-05-13T09:15:25.7216266Z 2024/05/13 08:54:18.136964 +0000 INFO cn-service.lockservice.allocator clean cannot commit task {"uuid": ""}
2024-05-13T09:15:25.7217108Z 2024/05/13 08:54:18.137837 +0000 DEBUG cn-service.lockservice.lock-server application accept loop started {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/105227950.sock]"}
2024-05-13T09:15:25.7217945Z 2024/05/13 08:54:18.137952 +0000 DEBUG cn-service.lockservice.lock-server application accept loop stopped {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/105227950.sock]"}
2024-05-13T09:15:25.7218726Z 2024/05/13 08:54:18.138015 +0000 DEBUG cn-service.lockservice.lock-server application started {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/105227950.sock]"}
2024-05-13T09:15:25.7219208Z 2024/05/13 08:54:18.138069 +0000 INFO cn-service.lockservice lock allocator start successfully {"uuid": ""}
2024-05-13T09:15:25.7219748Z 2024/05/13 08:54:18.138160 +0000 DEBUG cn-service.lockservice txn created {"uuid": "", "txn": "74786e31()"}
2024-05-13T09:15:25.7220808Z 2024/05/13 08:54:18.138947 +0000 DEBUG cn-service.lockservice.lock-client start connect to remote {"uuid": "", "remote": "unix:///tmp/105227950.sock", "backend-id": "018f7128-eb1a-7486-b433-5c9ebfc17639"}
2024-05-13T09:15:25.7221581Z 2024/05/13 08:54:18.139182 +0000 DEBUG cn-service.lockservice.lock-server session init completed {"uuid": "", "name": "lock-server", "session-id": 1}
2024-05-13T09:15:25.7222515Z 2024/05/13 08:54:18.139298 +0000 DEBUG cn-service.lockservice.lock-server session connected {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/105227950.sock]", "session-id": 1, "addr": "@"}
2024-05-13T09:15:25.7223501Z 2024/05/13 08:54:18.146061 +0000 DEBUG cn-service.lockservice.lock-client session init completed {"uuid": "", "remote": "unix:///tmp/105227950.sock", "backend-id": "018f7128-eb1a-7486-b433-5c9ebfc17639", "session-id": 0}
2024-05-13T09:15:25.7224437Z 2024/05/13 08:54:18.146118 +0000 DEBUG cn-service.lockservice.lock-client connect to remote succeed {"uuid": "", "remote": "unix:///tmp/105227950.sock", "backend-id": "018f7128-eb1a-7486-b433-5c9ebfc17639"}
2024-05-13T09:15:25.7225342Z 2024/05/13 08:54:18.146203 +0000 DEBUG cn-service.lockservice.lock-client read loop started {"uuid": "", "remote": "unix:///tmp/105227950.sock", "backend-id": "018f7128-eb1a-7486-b433-5c9ebfc17639"}
2024-05-13T09:15:25.7226257Z 2024/05/13 08:54:18.146336 +0000 DEBUG cn-service.lockservice.lock-client write loop started {"uuid": "", "remote": "unix:///tmp/105227950.sock", "backend-id": "018f7128-eb1a-7486-b433-5c9ebfc17639"}
2024-05-13T09:15:25.7227426Z 2024/05/13 08:54:18.146502 +0000 DEBUG cn-service.lockservice.lock-client write request {"uuid": "", "remote": "unix:///tmp/105227950.sock", "backend-id": "018f7128-eb1a-7486-b433-5c9ebfc17639", "request-id": 1, "request": "1: GetBind/1715590458105442711s1-0"}
2024-05-13T09:15:25.7228492Z 2024/05/13 08:54:18.146706 +0000 DEBUG cn-service.lockservice.lock-server session read message {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/105227950.sock]", "session-id": 1, "addr": "@", "seqence": 1}
2024-05-13T09:15:25.7229456Z 2024/05/13 08:54:18.146810 +0000 DEBUG cn-service.lockservice.lock-server received request {"uuid": "", "name": "lock-server", "sequence": 1, "client": "@", "request-id": 1, "request": "1: GetBind/1715590458105442711s1-0"}
2024-05-13T09:15:25.7230080Z 2024/05/13 08:54:18.146929 +0000 DEBUG cn-service.lockservice received a request {"uuid": "", "request": "1: GetBind/1715590458105442711s1-0"}
2024-05-13T09:15:25.7230896Z 2024/05/13 08:54:18.146989 +0000 DEBUG cn-service.lockservice.lock-server handle request completed {"uuid": "", "name": "lock-server", "sequence": 1, "client": "@", "request-id": 1}
2024-05-13T09:15:25.7231550Z 2024/05/13 08:54:18.148335 +0000 INFO cn-service.lockservice.allocator bind created {"uuid": "", "table": 0, "service": "1715590458105442711s1"}
2024-05-13T09:15:25.7232217Z 2024/05/13 08:54:18.148430 +0000 DEBUG cn-service.lockservice handle request completed {"uuid": "", "response": "1: GetBind/0-1715590458105442711s1-1"}
2024-05-13T09:15:25.7233198Z 2024/05/13 08:54:18.154120 +0000 DEBUG cn-service.lockservice.lock-server write responses {"uuid": "", "name": "lock-server", "client": "@", "request-id": 1, "response": "1: GetBind/0-1715590458105442711s1-1"}
2024-05-13T09:15:25.7234374Z 2024/05/13 08:54:18.154329 +0000 DEBUG cn-service.lockservice.lock-client read response {"uuid": "", "remote": "unix:///tmp/105227950.sock", "backend-id": "018f7128-eb1a-7486-b433-5c9ebfc17639", "request-id": 1, "response": "1: GetBind/0-1715590458105442711s1-1"}
2024-05-13T09:15:25.7235137Z 2024/05/13 08:54:18.160351 +0000 INFO cn-service.lockservice bind created {"uuid": "", "service": "1715590458105442711s1", "remote": false, "bind": "0-1715590458105442711s1-1"}
2024-05-13T09:15:25.7235956Z 2024/05/13 08:54:18.160462 +0000 DEBUG cn-service.lockservice try to lock on local {"uuid": "", "txn": "74786e31()", "table": 0, "rows": "[01]", "opts": "Exclusive-Row-Wait"}
2024-05-13T09:15:25.7236915Z 2024/05/13 08:54:18.160565 +0000 DEBUG cn-service.lockservice holder added {"uuid": "", "txn": "74786e31()", "table": 0, "rows": "[01]", "opts": "Exclusive-Row-Wait", "holders": "[74786e31(1715590458105442711s1)]", "waiters": "[]"}
2024-05-13T09:15:25.7237549Z 2024/05/13 08:54:18.160657 +0000 DEBUG cn-service.lockservice lock added to txn {"uuid": "", "txn": "74786e31()", "rows": "[01]"}2024-05-13T09:15:25.7238277Z 2024/05/13 08:54:18.160745 +0000 DEBUG cn-service.lockservice lock added to local {"uuid": "", "txn": "74786e31()", "table": 0, "rows": "[01]", "opts": "Exclusive-Row-Wait"}
2024-05-13T09:15:25.7238757Z 2024/05/13 08:54:18.160834 +0000 DEBUG cn-service.lockservice unlock txn {"uuid": "", "txn": "74786e31()"}
2024-05-13T09:15:25.7239264Z 2024/05/13 08:54:18.160885 +0000 DEBUG cn-service.lockservice ready to unlock txn {"uuid": "", "txn": "74786e31()"}
2024-05-13T09:15:25.7239828Z 2024/05/13 08:54:18.160939 +0000 DEBUG cn-service.lockservice txn unlock table {"uuid": "", "txn": "74786e31()", "table": 0}
2024-05-13T09:15:25.7240508Z 2024/05/13 08:54:18.160995 +0000 DEBUG cn-service.lockservice txn unlock table on local {"uuid": "", "txn": "74786e31()", "bind": "0-1715590458105442711s1-1"}
2024-05-13T09:15:25.7241215Z 2024/05/13 08:54:18.161064 +0000 DEBUG cn-service.lockservice lock unlocked {"uuid": "", "txn": "74786e31()", "key": "01", "lock": "Exclusive(row): holder(), waiters()"}
2024-05-13T09:15:25.7241883Z 2024/05/13 08:54:18.161153 +0000 DEBUG cn-service.lockservice txn unlock table completed {"uuid": "", "txn": "74786e31()", "table": 0, "rows": "[01]"}
2024-05-13T09:15:25.7242440Z 2024/05/13 08:54:18.161217 +0000 DEBUG cn-service.lockservice unlock txn {"uuid": "", "txn": "74786e31()", "cost": "335.887µs"}
2024-05-13T09:15:25.7242913Z 2024/05/13 08:54:18.161299 +0000 DEBUG cn-service.lockservice txn created {"uuid": "", "txn": "74786e32()"}
2024-05-13T09:15:25.7243927Z 2024/05/13 08:54:18.161719 +0000 DEBUG cn-service.lockservice.lock-client start connect to remote {"uuid": "", "remote": "unix:///tmp/105227950.sock", "backend-id": "018f7128-eb31-7698-9a78-1b6b1c5c0193"}
2024-05-13T09:15:25.7244266Z panic: missing free for type: *lockservice.waiter, 03-0xc0000d6780(1)
2024-05-13T09:15:25.7244383Z create by: <<<>>>
2024-05-13T09:15:25.7244390Z
2024-05-13T09:15:25.7244396Z
2024-05-13T09:15:25.7244516Z goroutine 5 [running]:
2024-05-13T09:15:25.7244917Z github.com/matrixorigin/matrixone/pkg/common/reuse.(*checker[...]).gc(0x1215c40, 0xc0000d6780)
2024-05-13T09:15:25.7245289Z /home/runner/work/matrixone/matrixone/pkg/common/reuse/checker.go:133 +0x394
2024-05-13T09:15:25.7245645Z github.com/matrixorigin/matrixone/pkg/common/reuse.newSyncPoolBased[...].func1.1()
2024-05-13T09:15:25.7246020Z /home/runner/work/matrixone/matrixone/pkg/common/reuse/sync_pool_based.go:46 +0xc5
2024-05-13T09:15:25.7246283Z FAIL github.com/matrixorigin/matrixone/pkg/lockservice 190.856s
2024-05-13T09:15:25.7247079Z 2024/05/13 08:51:09.675567 +0000 INFO malloc/malloc.go:42 malloc {"max buffer size": 1073741824, "num shards": 16, "classes": 23, "min class size": 128, "max class size": 1048576, "buffer objects per class": 23}
2024-05-13T09:15:25.7247588Z 2024/05/13 08:51:09.814962 +0000 INFO logutil/internal.go:37 MO logger init, level=debug, log file=, stackLevel=
2024-05-13T09:15:25.7247735Z === RUN TestClientConfigIsValidated
from matrixone.
https://github.com/matrixorigin/matrixone/actions/runs/9094783677/job/24996592593?pr=16147
2024-05-15T11:46:30.2714693Z 2024/05/15 11:32:20.269424 +0000 DEBUG cn-service.lockservice lock added to txn {"uuid": "", "txn": "74786e31()", "rows": "[01]"}
2024-05-15T11:46:30.2715437Z 2024/05/15 11:32:20.269495 +0000 DEBUG cn-service.lockservice lock added to local {"uuid": "", "txn": "74786e31()", "table": 0, "rows": "[01]", "opts": "Exclusive-Row-Wait"}
2024-05-15T11:46:30.2715995Z 2024/05/15 11:32:20.269578 +0000 INFO cn-service.lockservice keep remote locks task {"uuid": "", "cost": "12.526836ms"}
2024-05-15T11:46:30.2716553Z 2024/05/15 11:32:20.269624 +0000 INFO cn-service.lockservice keep lock table bind task {"uuid": "", "cost": "12.591277ms"}
2024-05-15T11:46:30.2716886Z panic: missing free for type: *lockservice.waiter, 02-0xc000264300(1)
2024-05-15T11:46:30.2717007Z create by: <<<>>>
2024-05-15T11:46:30.2717015Z
2024-05-15T11:46:30.2717019Z
2024-05-15T11:46:30.2717134Z goroutine 18 [running]:
2024-05-15T11:46:30.2717544Z github.com/matrixorigin/matrixone/pkg/common/reuse.(*checker[...]).gc(0x1216e80, 0xc000264300)
2024-05-15T11:46:30.2717915Z /home/runner/work/matrixone/matrixone/pkg/common/reuse/checker.go:133 +0x394
2024-05-15T11:46:30.2718276Z github.com/matrixorigin/matrixone/pkg/common/reuse.newSyncPoolBased[...].func1.1()
2024-05-15T11:46:30.2718656Z /home/runner/work/matrixone/matrixone/pkg/common/reuse/sync_pool_based.go:46 +0xc5
2024-05-15T11:46:30.2718990Z FAIL github.com/matrixorigin/matrixone/pkg/lockservice 194.090s
2024-05-15T11:46:30.2719807Z 2024/05/15 11:29:12.761045 +0000 INFO malloc/malloc.go:42 malloc {"max buffer size": 1073741824, "num shards": 16, "classes": 23, "min class size": 128, "max class size": 1048576, "buffer objects per class": 23}
2024-05-15T11:46:30.2720258Z 2024/05/15 11:29:12.926658 +0000 INFO logutil/internal.go:37 MO logger init, level=debug, log file=, stackLevel=
from matrixone.
https://github.com/matrixorigin/matrixone/actions/runs/9248603496/attempts/1?pr=16408
2024-05-27T03:47:09.0074913Z === RUN TestLockSuccWithKeepBindTimeout
2024-05-27T03:47:09.0075605Z 2024/05/27 03:18:19.310819 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-27T03:47:09.0076464Z 2024/05/27 03:18:19.310819 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-27T03:47:09.0077305Z 2024/05/27 03:18:19.310899 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-27T03:47:09.0078156Z 2024/05/27 03:18:19.311008 +0000 INFO cn-service.lockservice dead lock checker {"uuid": ""}
2024-05-27T03:47:09.0079217Z 2024/05/27 03:18:19.311024 +0000 INFO cn-service.lockservice.lock-client.lock-client gc idle backends task started {"uuid": ""}
2024-05-27T03:47:09.0080508Z 2024/05/27 03:18:19.311123 +0000 DEBUG cn-service.lockservice.lock-client.lock-client gc inactive backends task started {"uuid": ""}
2024-05-27T03:47:09.0081600Z 2024/05/27 03:18:19.312664 +0000 INFO cn-service.lockservice keep lock table bind task {"uuid": ""}
2024-05-27T03:47:09.0082519Z 2024/05/27 03:18:19.312666 +0000 INFO cn-service.lockservice keep remote locks task {"uuid": ""}
2024-05-27T03:47:09.0083899Z 2024/05/27 03:18:19.313229 +0000 DEBUG cn-service.lockservice.lock-server application accept loop started {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/service-308974149-s1.sock]"}
2024-05-27T03:47:09.0085794Z 2024/05/27 03:18:19.313296 +0000 DEBUG cn-service.lockservice.lock-server application accept loop stopped {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/service-308974149-s1.sock]"}
2024-05-27T03:47:09.0087564Z 2024/05/27 03:18:19.313339 +0000 DEBUG cn-service.lockservice.lock-server application started {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/service-308974149-s1.sock]"}
2024-05-27T03:47:09.0089019Z 2024/05/27 03:18:19.313556 +0000 INFO cn-service.lockservice lock service start successfully {"uuid": "", "serviceID": "1716779899309059057s1"}
2024-05-27T03:47:09.0090281Z 2024/05/27 03:18:19.317861 +0000 INFO cn-service.lockservice.lock-client.lock-client gc idle backends task started {"uuid": ""}
2024-05-27T03:47:09.0091543Z 2024/05/27 03:18:19.317939 +0000 DEBUG cn-service.lockservice.lock-client.lock-client gc inactive backends task started {"uuid": ""}
2024-05-27T03:47:09.0092682Z 2024/05/27 03:18:19.318041 +0000 INFO cn-service.lockservice.allocator clean cannot commit task {"uuid": ""}
2024-05-27T03:47:09.0094063Z 2024/05/27 03:18:19.318634 +0000 DEBUG cn-service.lockservice.lock-server application accept loop started {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/308825642.sock]"}
2024-05-27T03:47:09.0095703Z 2024/05/27 03:18:19.318697 +0000 DEBUG cn-service.lockservice.lock-server application accept loop stopped {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/308825642.sock]"}
2024-05-27T03:47:09.0097296Z 2024/05/27 03:18:19.318738 +0000 DEBUG cn-service.lockservice.lock-server application started {"uuid": "", "name": "lock-server", "listen-addresses": "[/tmp/308825642.sock]"}
2024-05-27T03:47:09.0098544Z 2024/05/27 03:18:19.318779 +0000 INFO cn-service.lockservice lock allocator start successfully {"uuid": ""}
2024-05-27T03:47:09.0099280Z panic: test timed out after 30m0s
2024-05-27T03:47:09.0099628Z running tests:
2024-05-27T03:47:09.0099964Z TestLockSuccWithKeepBindTimeout (26m32s)
2024-05-27T03:47:09.0100259Z
2024-05-27T03:47:09.0100379Z goroutine 62860 [running]:
2024-05-27T03:47:09.0100708Z testing.(*M).startAlarm.func1()
2024-05-27T03:47:09.0101252Z /opt/hostedtoolcache/go/1.21.5/x64/src/testing/testing.go:2259 +0x259
2024-05-27T03:47:09.0101779Z created by time.goFunc
2024-05-27T03:47:09.0102233Z /opt/hostedtoolcache/go/1.21.5/x64/src/time/sleep.go:176 +0x45
2024-05-27T03:47:09.0102592Z
2024-05-27T03:47:09.0102737Z goroutine 1 [chan receive, 26 minutes]:
2024-05-27T03:47:09.0103209Z testing.(*T).Run(0xc00009a9c0, {0x106f1db, 0x1f}, 0x10a8850)
2024-05-27T03:47:09.0103829Z /opt/hostedtoolcache/go/1.21.5/x64/src/testing/testing.go:1649 +0x871
2024-05-27T03:47:09.0104342Z testing.runTests.func1(0x0?)
2024-05-27T03:47:09.0104836Z /opt/hostedtoolcache/go/1.21.5/x64/src/testing/testing.go:2054 +0x85
2024-05-27T03:47:09.0105477Z testing.tRunner(0xc00009a9c0, 0xc0001ffaf8)
2024-05-27T03:47:09.0106026Z /opt/hostedtoolcache/go/1.21.5/x64/src/testing/testing.go:1595 +0x262
2024-05-27T03:47:09.0106717Z testing.runTests(0xc0000ee000?, {0x17c2f80, 0xb9, 0xb9}, {0x1c?, 0x1e?, 0x17c9340?})
2024-05-27T03:47:09.0107424Z /opt/hostedtoolcache/go/1.21.5/x64/src/testing/testing.go:2052 +0x8ae
2024-05-27T03:47:09.0107923Z testing.(*M).Run(0xc0000ee000)
2024-05-27T03:47:09.0108408Z /opt/hostedtoolcache/go/1.21.5/x64/src/testing/testing.go:1925 +0xcd8
2024-05-27T03:47:09.0108879Z main.main()
2024-05-27T03:47:09.0109156Z _testmain.go:427 +0x2be
2024-05-27T03:47:09.0109360Z
2024-05-27T03:47:09.0109471Z goroutine 6 [select]:
2024-05-27T03:47:09.0110042Z github.com/panjf2000/ants/v2.(*Pool).purgeStaleWorkers(0xc000112f00, {0x1222fd0, 0xc0001507d0})
2024-05-27T03:47:09.0110866Z /home/runner/go/pkg/mod/github.com/panjf2000/ants/[email protected]/pool.go:83 +0x1eb
2024-05-27T03:47:09.0111577Z created by github.com/panjf2000/ants/v2.(*Pool).goPurge in goroutine 1
2024-05-27T03:47:09.0112308Z /home/runner/go/pkg/mod/github.com/panjf2000/ants/[email protected]/pool.go:148 +0x185
2024-05-27T03:47:09.0112742Z
2024-05-27T03:47:09.0112872Z goroutine 7 [select]:
2024-05-27T03:47:09.0113530Z github.com/panjf2000/ants/v2.(*Pool).ticktock(0xc000112f00, {0x1222fd0, 0xc000150820})
2024-05-27T03:47:09.0114355Z /home/runner/go/pkg/mod/github.com/panjf2000/ants/[email protected]/pool.go:126 +0x205
2024-05-27T03:47:09.0115161Z created by github.com/panjf2000/ants/v2.(*Pool).goTicktock in goroutine 1
2024-05-27T03:47:09.0115889Z /home/runner/go/pkg/mod/github.com/panjf2000/ants/[email protected]/pool.go:155 +0x187
2024-05-27T03:47:09.0116308Z
2024-05-27T03:47:09.0116433Z goroutine 116 [runnable]:
2024-05-27T03:47:09.0116744Z runtime.Gosched(...)
2024-05-27T03:47:09.0117165Z /opt/hostedtoolcache/go/1.21.5/x64/src/runtime/proc.go:338
2024-05-27T03:47:09.0117604Z runtime.GC()
2024-05-27T03:47:09.0118012Z /opt/hostedtoolcache/go/1.21.5/x64/src/runtime/mgc.go:477 +0x65
2024-05-27T03:47:09.0118498Z runtime/debug.freeOSMemory()
2024-05-27T03:47:09.0118984Z /opt/hostedtoolcache/go/1.21.5/x64/src/runtime/mheap.go:1672 +0x13
2024-05-27T03:47:09.0119484Z runtime/debug.FreeOSMemory(...)
2024-05-27T03:47:09.0120000Z /opt/hostedtoolcache/go/1.21.5/x64/src/runtime/debug/garbage.go:104
2024-05-27T03:47:09.0120649Z github.com/matrixorigin/matrixone/pkg/common/mpool.init.0.func1()
2024-05-27T03:47:09.0121397Z /home/runner/work/matrixone/matrixone/pkg/common/mpool/alloc_tracing.go:35 +0x4a
2024-05-27T03:47:09.0122197Z created by github.com/matrixorigin/matrixone/pkg/common/mpool.init.0 in goroutine 1
2024-05-27T03:47:09.0123015Z /home/runner/work/matrixone/matrixone/pkg/common/mpool/alloc_tracing.go:31 +0x25
2024-05-27T03:47:09.0123472Z
2024-05-27T03:47:09.0123614Z goroutine 50011 [select, 26 minutes]:
2024-05-27T03:47:09.0124366Z github.com/matrixorigin/matrixone/pkg/common/morpc.(*client).gcInactiveTask(0xc0000debb0, {0x1222f98, 0xc0005b4000})
2024-05-27T03:47:09.0125420Z /home/runner/work/matrixone/matrixone/pkg/common/morpc/client.go:406 +0x1ff
2024-05-27T03:47:09.0126260Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0127127Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0128068Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0129029Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0129482Z
2024-05-27T03:47:09.0129629Z goroutine 49999 [IO wait, 26 minutes]:
2024-05-27T03:47:09.0130081Z internal/poll.runtime_pollWait(0x7f259adc3ff8, 0x72)
2024-05-27T03:47:09.0130683Z /opt/hostedtoolcache/go/1.21.5/x64/src/runtime/netpoll.go:343 +0x85
2024-05-27T03:47:09.0131266Z internal/poll.(*pollDesc).wait(0xc00295ec20, 0x1?, 0x0)
2024-05-27T03:47:09.0131923Z /opt/hostedtoolcache/go/1.21.5/x64/src/internal/poll/fd_poll_runtime.go:84 +0xb1
2024-05-27T03:47:09.0132502Z internal/poll.(*pollDesc).waitRead(...)
2024-05-27T03:47:09.0133068Z /opt/hostedtoolcache/go/1.21.5/x64/src/internal/poll/fd_poll_runtime.go:89
2024-05-27T03:47:09.0133618Z internal/poll.(*FD).Accept(0xc00295ec00)
2024-05-27T03:47:09.0134184Z /opt/hostedtoolcache/go/1.21.5/x64/src/internal/poll/fd_unix.go:611 +0x425
2024-05-27T03:47:09.0134709Z net.(*netFD).accept(0xc00295ec00)
2024-05-27T03:47:09.0135192Z /opt/hostedtoolcache/go/1.21.5/x64/src/net/fd_unix.go:172 +0x3e
2024-05-27T03:47:09.0135690Z net.(*UnixListener).accept(0xc0003c4cc0)
2024-05-27T03:47:09.0136231Z /opt/hostedtoolcache/go/1.21.5/x64/src/net/unixsock_posix.go:172 +0x3e
2024-05-27T03:47:09.0136757Z net.(*UnixListener).Accept(0xc0003c4cc0)
2024-05-27T03:47:09.0137264Z /opt/hostedtoolcache/go/1.21.5/x64/src/net/unixsock.go:260 +0x65
2024-05-27T03:47:09.0137583Z github.com/fagongzi/goetty/v2.(*server).doStart.func2({0x1220c70, 0xc0003c4cc0})
2024-05-27T03:47:09.0138232Z /home/runner/go/pkg/mod/github.com/matrixorigin/goetty/[email protected]/application.go:309 +0xf0
2024-05-27T03:47:09.0138544Z created by github.com/fagongzi/goetty/v2.(*server).doStart in goroutine 49972
2024-05-27T03:47:09.0139371Z /home/runner/go/pkg/mod/github.com/matrixorigin/goetty/[email protected]/application.go:365 +0x187
2024-05-27T03:47:09.0139379Z
2024-05-27T03:47:09.0139530Z goroutine 50020 [select, 26 minutes]:
2024-05-27T03:47:09.0140063Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc0005b5cb0})
2024-05-27T03:47:09.0140396Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0140798Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0141151Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0141647Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0142012Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0142018Z
2024-05-27T03:47:09.0142135Z goroutine 49978 [select]:
2024-05-27T03:47:09.0142633Z github.com/matrixorigin/matrixone/pkg/common/morpc.(*client).gcIdleTask(0xc0000de9a0, {0x1222f98, 0xc0003c49c0})
2024-05-27T03:47:09.0142972Z /home/runner/work/matrixone/matrixone/pkg/common/morpc/client.go:383 +0x292
2024-05-27T03:47:09.0143372Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0143723Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0144229Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0144583Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0144651Z
2024-05-27T03:47:09.0144772Z goroutine 50001 [select]:
2024-05-27T03:47:09.0145271Z github.com/matrixorigin/matrixone/pkg/lockservice.(*waiterEvents).handle(0xc004086bd0, {0x1222f98, 0xc00010faa0})
2024-05-27T03:47:09.0145761Z /home/runner/work/matrixone/matrixone/pkg/lockservice/waiter_events.go:183 +0x2ba
2024-05-27T03:47:09.0146172Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0146523Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0147017Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0147366Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0147372Z
2024-05-27T03:47:09.0147511Z goroutine 50006 [select, 26 minutes]:
2024-05-27T03:47:09.0148052Z github.com/matrixorigin/matrixone/pkg/lockservice.(*service).handleFetchWhoWaitingMe(0xc000152780, {0x1222f98, 0xc00010fc80})
2024-05-27T03:47:09.0148431Z /home/runner/work/matrixone/matrixone/pkg/lockservice/service_remote.go:546 +0x13a
2024-05-27T03:47:09.0148827Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0149174Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0149683Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0150063Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0150069Z
2024-05-27T03:47:09.0150212Z goroutine 49986 [select, 26 minutes]:
2024-05-27T03:47:09.0150671Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0002af710})
2024-05-27T03:47:09.0150997Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0151390Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0151743Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0152247Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0152746Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0152753Z
2024-05-27T03:47:09.0152953Z goroutine 50026 [select, 26 minutes]:
2024-05-27T03:47:09.0153420Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc0005b5ef0})
2024-05-27T03:47:09.0153752Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0154142Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0154490Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0154986Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0155344Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0155350Z
2024-05-27T03:47:09.0155484Z goroutine 49989 [select, 26 minutes]:
2024-05-27T03:47:09.0155953Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0002af890})
2024-05-27T03:47:09.0156268Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0156673Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0157013Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0157513Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0157858Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0157943Z
2024-05-27T03:47:09.0158128Z goroutine 49974 [select, 26 minutes]:
2024-05-27T03:47:09.0158604Z github.com/matrixorigin/matrixone/pkg/lockservice.(*detector).doCheck(0xc004df2480, {0x1222f98, 0xc0003c46f0})
2024-05-27T03:47:09.0158951Z /home/runner/work/matrixone/matrixone/pkg/lockservice/deadlock.go:123 +0x29d
2024-05-27T03:47:09.0159345Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0159690Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0160183Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0160525Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0160530Z
2024-05-27T03:47:09.0160667Z goroutine 50015 [select, 26 minutes]:
2024-05-27T03:47:09.0161111Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc0005b5500})
2024-05-27T03:47:09.0161437Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0161824Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0162190Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0162690Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0163032Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0163037Z
2024-05-27T03:47:09.0163174Z goroutine 49993 [select, 26 minutes]:
2024-05-27T03:47:09.0163612Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0002afa40})
2024-05-27T03:47:09.0163932Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0164315Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0164663Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0165147Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0165611Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0165616Z
2024-05-27T03:47:09.0165807Z goroutine 50028 [select, 26 minutes]:
2024-05-27T03:47:09.0166264Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc0005b5fb0})
2024-05-27T03:47:09.0166581Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0166975Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0167319Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0167808Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0168158Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0168163Z
2024-05-27T03:47:09.0168293Z goroutine 49992 [select, 26 minutes]:
2024-05-27T03:47:09.0168740Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0002af9e0})
2024-05-27T03:47:09.0169052Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0169443Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0169778Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0170272Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0170610Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0170676Z
2024-05-27T03:47:09.0170803Z goroutine 50003 [select]:
2024-05-27T03:47:09.0171282Z github.com/matrixorigin/matrixone/pkg/lockservice.(*waiterEvents).handle(0xc004086bd0, {0x1222f98, 0xc00010fb60})
2024-05-27T03:47:09.0171658Z /home/runner/work/matrixone/matrixone/pkg/lockservice/waiter_events.go:183 +0x2ba
2024-05-27T03:47:09.0172070Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0172424Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0172929Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0173281Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0173286Z
2024-05-27T03:47:09.0173426Z goroutine 49979 [select, 26 minutes]:
2024-05-27T03:47:09.0173925Z github.com/matrixorigin/matrixone/pkg/common/morpc.(*client).gcInactiveTask(0xc0000de9a0, {0x1222f98, 0xc0003c4a20})
2024-05-27T03:47:09.0174266Z /home/runner/work/matrixone/matrixone/pkg/common/morpc/client.go:406 +0x1ff
2024-05-27T03:47:09.0174650Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0175000Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0175496Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0175847Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0175852Z
2024-05-27T03:47:09.0175983Z goroutine 49997 [select, 26 minutes]:
2024-05-27T03:47:09.0176429Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0002afbc0})
2024-05-27T03:47:09.0176752Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0177137Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0177488Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0177981Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0178444Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0178451Z
2024-05-27T03:47:09.0178636Z goroutine 49973 [select, 26 minutes]:
2024-05-27T03:47:09.0179108Z github.com/matrixorigin/matrixone/pkg/lockservice.(*detector).doCheck(0xc004df2480, {0x1222f98, 0xc0003c4690})
2024-05-27T03:47:09.0179451Z /home/runner/work/matrixone/matrixone/pkg/lockservice/deadlock.go:123 +0x29d
2024-05-27T03:47:09.0179843Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0180180Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0180676Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0181018Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0181029Z
2024-05-27T03:47:09.0181145Z goroutine 50002 [select]:
2024-05-27T03:47:09.0181626Z github.com/matrixorigin/matrixone/pkg/lockservice.(*waiterEvents).handle(0xc004086bd0, {0x1222f98, 0xc00010fb00})
2024-05-27T03:47:09.0181998Z /home/runner/work/matrixone/matrixone/pkg/lockservice/waiter_events.go:183 +0x2ba
2024-05-27T03:47:09.0182398Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0182744Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0183235Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0183585Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0183651Z
2024-05-27T03:47:09.0183794Z goroutine 50008 [select, 26 minutes]:
2024-05-27T03:47:09.0184334Z github.com/matrixorigin/matrixone/pkg/lockservice.(*service).handleFetchWhoWaitingMe(0xc000152780, {0x1222f98, 0xc00010fd40})
2024-05-27T03:47:09.0184712Z /home/runner/work/matrixone/matrixone/pkg/lockservice/service_remote.go:546 +0x13a
2024-05-27T03:47:09.0185096Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0185583Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0186085Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0186428Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0186433Z
2024-05-27T03:47:09.0186572Z goroutine 50009 [select, 26 minutes]:
2024-05-27T03:47:09.0187050Z github.com/matrixorigin/matrixone/pkg/common/morpc.(*client).createTask(0xc0000debb0, {0x1222f98, 0xc00010ff20})
2024-05-27T03:47:09.0187396Z /home/runner/work/matrixone/matrixone/pkg/common/morpc/client.go:458 +0x113
2024-05-27T03:47:09.0187779Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0188123Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0188607Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0188958Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0188963Z
2024-05-27T03:47:09.0189077Z goroutine 50012 [select]:
2024-05-27T03:47:09.0189652Z github.com/matrixorigin/matrixone/pkg/lockservice.(*lockTableAllocator).checkInvalidBinds(0xc0002a8be0, {0x1222f98, 0xc0005b4390})
2024-05-27T03:47:09.0190052Z /home/runner/work/matrixone/matrixone/pkg/lockservice/lock_table_allocator.go:441 +0x1cb
2024-05-27T03:47:09.0190462Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0190813Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0191473Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0191838Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0191908Z
2024-05-27T03:47:09.0192042Z goroutine 49994 [select, 26 minutes]:
2024-05-27T03:47:09.0192496Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0002afaa0})
2024-05-27T03:47:09.0192812Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0193218Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0193560Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0194061Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0194410Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0194415Z
2024-05-27T03:47:09.0194554Z goroutine 50029 [select, 26 minutes]:
2024-05-27T03:47:09.0194999Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc000192030})
2024-05-27T03:47:09.0195323Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0195723Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0196065Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0196561Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0196910Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0196987Z
2024-05-27T03:47:09.0197127Z goroutine 50016 [select, 26 minutes]:
2024-05-27T03:47:09.0197570Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc0005b55c0})
2024-05-27T03:47:09.0197889Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0198285Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0198635Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0199122Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0199479Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0199484Z
2024-05-27T03:47:09.0199620Z goroutine 49977 [select, 26 minutes]:
2024-05-27T03:47:09.0200102Z github.com/matrixorigin/matrixone/pkg/common/morpc.(*client).createTask(0xc0000de9a0, {0x1222f98, 0xc0003c4960})
2024-05-27T03:47:09.0200452Z /home/runner/work/matrixone/matrixone/pkg/common/morpc/client.go:458 +0x113
2024-05-27T03:47:09.0200838Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0201190Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0201674Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0202025Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0202031Z
2024-05-27T03:47:09.0202147Z goroutine 50010 [select]:
2024-05-27T03:47:09.0202631Z github.com/matrixorigin/matrixone/pkg/common/morpc.(*client).gcIdleTask(0xc0000debb0, {0x1222f98, 0xc00010ff80})
2024-05-27T03:47:09.0202967Z /home/runner/work/matrixone/matrixone/pkg/common/morpc/client.go:383 +0x292
2024-05-27T03:47:09.0203362Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0203699Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0204317Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0204678Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0204737Z
2024-05-27T03:47:09.0204877Z goroutine 50018 [select, 26 minutes]:
2024-05-27T03:47:09.0205337Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc0005b5bf0})
2024-05-27T03:47:09.0205654Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0206056Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0206397Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0206906Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0207251Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0207255Z
2024-05-27T03:47:09.0207380Z goroutine 50014 [select]:
2024-05-27T03:47:09.0207933Z github.com/matrixorigin/matrixone/pkg/common/morpc.(*server).closeDisconnectedSession(0xc0000dec60, {0x1222f98, 0xc0005b54a0})
2024-05-27T03:47:09.0208279Z /home/runner/work/matrixone/matrixone/pkg/common/morpc/server.go:474 +0x1e5
2024-05-27T03:47:09.0208670Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0209019Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0209515Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0209864Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0209931Z
2024-05-27T03:47:09.0210072Z goroutine 50025 [select, 26 minutes]:
2024-05-27T03:47:09.0210518Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc0005b5e90})
2024-05-27T03:47:09.0210847Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0211245Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0211598Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0212092Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0212447Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0212452Z
2024-05-27T03:47:09.0212588Z goroutine 50030 [select, 26 minutes]:
2024-05-27T03:47:09.0213038Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc000192090})
2024-05-27T03:47:09.0213351Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0213751Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0214104Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0214592Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0214944Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0214949Z
2024-05-27T03:47:09.0215062Z goroutine 49982 [select]:
2024-05-27T03:47:09.0215608Z github.com/matrixorigin/matrixone/pkg/lockservice.(*lockTableKeeper).keepRemoteLock(0xc003b8e780, {0x1222f98, 0xc0003c5e90})
2024-05-27T03:47:09.0215992Z /home/runner/work/matrixone/matrixone/pkg/lockservice/lock_table_keeper.go:98 +0x3ae
2024-05-27T03:47:09.0216392Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0216737Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0217350Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0217704Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0217761Z
2024-05-27T03:47:09.0217901Z goroutine 50027 [select, 26 minutes]:
2024-05-27T03:47:09.0218343Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc0005b5f50})
2024-05-27T03:47:09.0218666Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0219064Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0219405Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0219900Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0220245Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0220250Z
2024-05-27T03:47:09.0220394Z goroutine 49987 [select, 26 minutes]:
2024-05-27T03:47:09.0220834Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0002af7a0})
2024-05-27T03:47:09.0221157Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0221545Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0221891Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0222378Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0222793Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0222798Z
2024-05-27T03:47:09.0222934Z goroutine 49991 [select, 26 minutes]:
2024-05-27T03:47:09.0223371Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0002af980})
2024-05-27T03:47:09.0223688Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0224075Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0224424Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0224908Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0225257Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0225262Z
2024-05-27T03:47:09.0225493Z goroutine 49984 [select, 26 minutes]:
2024-05-27T03:47:09.0225940Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0003c5f80})
2024-05-27T03:47:09.0226251Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0226644Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0226986Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0227482Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0227830Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0227835Z
2024-05-27T03:47:09.0227965Z goroutine 50021 [select, 26 minutes]:
2024-05-27T03:47:09.0228401Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc0005b5d10})
2024-05-27T03:47:09.0228707Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0229101Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0229439Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0230071Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0230419Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0230488Z
2024-05-27T03:47:09.0230808Z goroutine 50004 [select]:
2024-05-27T03:47:09.0231440Z github.com/matrixorigin/matrixone/pkg/lockservice.(*waiterEvents).handle(0xc004086bd0, {0x1222f98, 0xc00010fbc0})
2024-05-27T03:47:09.0231864Z /home/runner/work/matrixone/matrixone/pkg/lockservice/waiter_events.go:183 +0x2ba
2024-05-27T03:47:09.0232258Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0232687Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0233188Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0233535Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0233540Z
2024-05-27T03:47:09.0233682Z goroutine 49995 [select, 26 minutes]:
2024-05-27T03:47:09.0234129Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0002afb00})
2024-05-27T03:47:09.0234567Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0234999Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0235347Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0235843Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0236196Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0236293Z
2024-05-27T03:47:09.0236431Z goroutine 50022 [select, 26 minutes]:
2024-05-27T03:47:09.0236886Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc0005b5d70})
2024-05-27T03:47:09.0237207Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0237605Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0237955Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0238598Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0239063Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0239068Z
2024-05-27T03:47:09.0239200Z goroutine 49998 [select, 26 minutes]:
2024-05-27T03:47:09.0239649Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0002aff20})
2024-05-27T03:47:09.0239966Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0240408Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0240752Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0241259Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0241643Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0241648Z
2024-05-27T03:47:09.0241826Z goroutine 50000 [sync.Mutex.Lock, 26 minutes]:
2024-05-27T03:47:09.0241994Z sync.runtime_SemacquireMutex(0x0?, 0x0?, 0x0?)
2024-05-27T03:47:09.0242262Z /opt/hostedtoolcache/go/1.21.5/x64/src/runtime/sema.go:77 +0x25
2024-05-27T03:47:09.0242402Z sync.(*Mutex).lockSlow(0xc0035b2e30)
2024-05-27T03:47:09.0242655Z /opt/hostedtoolcache/go/1.21.5/x64/src/sync/mutex.go:171 +0x213
2024-05-27T03:47:09.0242786Z sync.(*Mutex).Lock(0xc0035b2e30)
2024-05-27T03:47:09.0243143Z /opt/hostedtoolcache/go/1.21.5/x64/src/sync/mutex.go:90 +0x55
2024-05-27T03:47:09.0243277Z sync.(*RWMutex).Lock(0xc0035b2e30)
2024-05-27T03:47:09.0243731Z /opt/hostedtoolcache/go/1.21.5/x64/src/sync/rwmutex.go:147 +0x45
2024-05-27T03:47:09.0244397Z github.com/matrixorigin/matrixone/pkg/lockservice.(*mapBasedTxnHolder).getTimeoutRemoveTxn(0xc0035b2e00, 0xc00031e630, {0x0, 0x0?, 0x0}, 0x2540be400)
2024-05-27T03:47:09.0244890Z /home/runner/work/matrixone/matrixone/pkg/lockservice/service.go:751 +0x11d
2024-05-27T03:47:09.0245433Z github.com/matrixorigin/matrixone/pkg/lockservice.(*service).unlockTimeoutRemoteTxn(0xc000152780, {0x1222f98, 0xc00010f9e0})
2024-05-27T03:47:09.0245804Z /home/runner/work/matrixone/matrixone/pkg/lockservice/service_remote.go:490 +0x28f
2024-05-27T03:47:09.0246202Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0246550Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0247052Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0247398Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0247403Z
2024-05-27T03:47:09.0247546Z goroutine 49985 [select, 26 minutes]:
2024-05-27T03:47:09.0247994Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0001cc000})
2024-05-27T03:47:09.0248309Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0248707Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0249049Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0249549Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0249986Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0249992Z
2024-05-27T03:47:09.0250133Z goroutine 50023 [select, 26 minutes]:
2024-05-27T03:47:09.0250581Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc0005b5dd0})
2024-05-27T03:47:09.0250903Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0251291Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0251638Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0252127Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0252477Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0252484Z
2024-05-27T03:47:09.0252604Z goroutine 49980 [select]:
2024-05-27T03:47:09.0253155Z github.com/matrixorigin/matrixone/pkg/common/morpc.(*server).closeDisconnectedSession(0xc0000dea50, {0x1222f98, 0xc0003c5da0})
2024-05-27T03:47:09.0253499Z /home/runner/work/matrixone/matrixone/pkg/common/morpc/server.go:474 +0x1e5
2024-05-27T03:47:09.0253890Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0254247Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0254752Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0255107Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0255112Z
2024-05-27T03:47:09.0255244Z goroutine 49990 [select, 26 minutes]:
2024-05-27T03:47:09.0255694Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0002af920})
2024-05-27T03:47:09.0256013Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0256416Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0256876Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0257402Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0257823Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0257828Z
2024-05-27T03:47:09.0257961Z goroutine 50019 [select, 26 minutes]:
2024-05-27T03:47:09.0258414Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc0005b5c50})
2024-05-27T03:47:09.0258730Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0259130Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0259474Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0259975Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0260322Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0260326Z
2024-05-27T03:47:09.0260473Z goroutine 50031 [IO wait, 26 minutes]:
2024-05-27T03:47:09.0260665Z internal/poll.runtime_pollWait(0x7f2548ee0d08, 0x72)
2024-05-27T03:47:09.0260950Z /opt/hostedtoolcache/go/1.21.5/x64/src/runtime/netpoll.go:343 +0x85
2024-05-27T03:47:09.0261145Z internal/poll.(*pollDesc).wait(0xc000180e20, 0x1?, 0x0)
2024-05-27T03:47:09.0261490Z /opt/hostedtoolcache/go/1.21.5/x64/src/internal/poll/fd_poll_runtime.go:84 +0xb1
2024-05-27T03:47:09.0261630Z internal/poll.(*pollDesc).waitRead(...)
2024-05-27T03:47:09.0261941Z /opt/hostedtoolcache/go/1.21.5/x64/src/internal/poll/fd_poll_runtime.go:89
2024-05-27T03:47:09.0262159Z internal/poll.(*FD).Accept(0xc000180e00)
2024-05-27T03:47:09.0262465Z /opt/hostedtoolcache/go/1.21.5/x64/src/internal/poll/fd_unix.go:611 +0x425
2024-05-27T03:47:09.0262599Z net.(*netFD).accept(0xc000180e00)
2024-05-27T03:47:09.0262856Z /opt/hostedtoolcache/go/1.21.5/x64/src/net/fd_unix.go:172 +0x3e
2024-05-27T03:47:09.0263006Z net.(*UnixListener).accept(0xc0005b4780)
2024-05-27T03:47:09.0263290Z /opt/hostedtoolcache/go/1.21.5/x64/src/net/unixsock_posix.go:172 +0x3e
2024-05-27T03:47:09.0263436Z net.(*UnixListener).Accept(0xc0005b4780)
2024-05-27T03:47:09.0263691Z /opt/hostedtoolcache/go/1.21.5/x64/src/net/unixsock.go:260 +0x65
2024-05-27T03:47:09.0264018Z github.com/fagongzi/goetty/v2.(*server).doStart.func2({0x1220c70, 0xc0005b4780})
2024-05-27T03:47:09.0264671Z /home/runner/go/pkg/mod/github.com/matrixorigin/goetty/[email protected]/application.go:309 +0xf0
2024-05-27T03:47:09.0264988Z created by github.com/fagongzi/goetty/v2.(*server).doStart in goroutine 49972
2024-05-27T03:47:09.0265748Z /home/runner/go/pkg/mod/github.com/matrixorigin/goetty/[email protected]/application.go:365 +0x187
2024-05-27T03:47:09.0265755Z
2024-05-27T03:47:09.0265878Z goroutine 50013 [select]:
2024-05-27T03:47:09.0266449Z github.com/matrixorigin/matrixone/pkg/lockservice.(*lockTableAllocator).cleanCommitState(0xc0002a8be0, {0x1222f98, 0xc0005b4420})
2024-05-27T03:47:09.0266860Z /home/runner/work/matrixone/matrixone/pkg/lockservice/lock_table_allocator.go:506 +0x35c
2024-05-27T03:47:09.0267256Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0267609Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0268111Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0268460Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0268467Z
2024-05-27T03:47:09.0268608Z goroutine 50024 [select, 26 minutes]:
2024-05-27T03:47:09.0269051Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc0005b5e30})
2024-05-27T03:47:09.0269518Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0269930Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0270347Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0270842Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0271199Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0271204Z
2024-05-27T03:47:09.0271318Z goroutine 49981 [select]:
2024-05-27T03:47:09.0271873Z github.com/matrixorigin/matrixone/pkg/lockservice.(*lockTableKeeper).keepLockTableBind(0xc003b8e780, {0x1222f98, 0xc0003c5e30})
2024-05-27T03:47:09.0272261Z /home/runner/work/matrixone/matrixone/pkg/lockservice/lock_table_keeper.go:78 +0x259
2024-05-27T03:47:09.0272652Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0273003Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0273502Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0273857Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0273862Z
2024-05-27T03:47:09.0273996Z goroutine 49975 [select, 26 minutes]:
2024-05-27T03:47:09.0274461Z github.com/matrixorigin/matrixone/pkg/lockservice.(*detector).doCheck(0xc004df2480, {0x1222f98, 0xc0003c4750})
2024-05-27T03:47:09.0274800Z /home/runner/work/matrixone/matrixone/pkg/lockservice/deadlock.go:123 +0x29d
2024-05-27T03:47:09.0275276Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0275622Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0276125Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0276467Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0276474Z
2024-05-27T03:47:09.0276617Z goroutine 50005 [select, 26 minutes]:
2024-05-27T03:47:09.0277167Z github.com/matrixorigin/matrixone/pkg/lockservice.(*service).handleFetchWhoWaitingMe(0xc000152780, {0x1222f98, 0xc00010fc20})
2024-05-27T03:47:09.0277539Z /home/runner/work/matrixone/matrixone/pkg/lockservice/service_remote.go:546 +0x13a
2024-05-27T03:47:09.0277939Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0278281Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0278783Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0279129Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0279133Z
2024-05-27T03:47:09.0279310Z goroutine 49972 [sync.Mutex.Lock, 26 minutes]:
2024-05-27T03:47:09.0279541Z sync.runtime_SemacquireMutex(0xc0058f97a0?, 0x28?, 0x1024d00?)
2024-05-27T03:47:09.0279808Z /opt/hostedtoolcache/go/1.21.5/x64/src/runtime/sema.go:77 +0x25
2024-05-27T03:47:09.0279941Z sync.(*Mutex).lockSlow(0xc0001e08b8)
2024-05-27T03:47:09.0280201Z /opt/hostedtoolcache/go/1.21.5/x64/src/sync/mutex.go:171 +0x213
2024-05-27T03:47:09.0280326Z sync.(*Mutex).Lock(0xc0001e08b8)
2024-05-27T03:47:09.0280577Z /opt/hostedtoolcache/go/1.21.5/x64/src/sync/mutex.go:90 +0x55
2024-05-27T03:47:09.0280704Z sync.(*RWMutex).Lock(0xc0001e08b8)
2024-05-27T03:47:09.0280976Z /opt/hostedtoolcache/go/1.21.5/x64/src/sync/rwmutex.go:147 +0x45
2024-05-27T03:47:09.0281552Z github.com/matrixorigin/matrixone/pkg/lockservice.newActiveTxn({0xc000014f5c, 0x4, 0x4}, {0xc000014f5c, 0x4}, 0xc0003c4480, {0x0, 0x0})
2024-05-27T03:47:09.0281867Z /home/runner/work/matrixone/matrixone/pkg/lockservice/txn.go:57 +0x9e
2024-05-27T03:47:09.0282608Z github.com/matrixorigin/matrixone/pkg/lockservice.(*mapBasedTxnHolder).getActiveTxn(0xc0035b2e00, {0xc000014f5c, 0x4, 0x4}, 0x1, {0x0, 0x0})
2024-05-27T03:47:09.0283009Z /home/runner/work/matrixone/matrixone/pkg/lockservice/service.go:676 +0x2a5
2024-05-27T03:47:09.0283512Z github.com/matrixorigin/matrixone/pkg/lockservice.(*service).Lock(_, {_, _}, _, {_, _, _}, {0xc000014f5c, 0x4, 0x4}, ...)
2024-05-27T03:47:09.0283849Z /home/runner/work/matrixone/matrixone/pkg/lockservice/service.go:149 +0x3e4
2024-05-27T03:47:09.0284496Z github.com/matrixorigin/matrixone/pkg/lockservice.TestLockSuccWithKeepBindTimeout.func1(0xc0002a8be0?, {0xc0000ac090, 0x1, 0xc0002a8be8?})
2024-05-27T03:47:09.0284862Z /home/runner/work/matrixone/matrixone/pkg/lockservice/service_test.go:1364 +0x29e
2024-05-27T03:47:09.0285512Z github.com/matrixorigin/matrixone/pkg/lockservice.runLockServiceTestsWithLevel.func1.1({0x1224230?, 0xc0002a8be0}, {0xc00004e5b0, 0x1, 0x0?})
2024-05-27T03:47:09.0285878Z /home/runner/work/matrixone/matrixone/pkg/lockservice/service_test.go:3442 +0x1d6
2024-05-27T03:47:09.0286548Z github.com/matrixorigin/matrixone/pkg/lockservice.RunLockServicesForTest(0xff?, {0xc00026de80, 0x1, 0x1?}, 0x3b9aca00?, 0xc0001ffc88, 0x0, {0x0, 0x0, 0x0})
2024-05-27T03:47:09.0286896Z /home/runner/work/matrixone/matrixone/pkg/lockservice/test_helper.go:82 +0xc4e
2024-05-27T03:47:09.0287293Z github.com/matrixorigin/matrixone/pkg/lockservice.runLockServiceTestsWithLevel.func1()
2024-05-27T03:47:09.0287663Z /home/runner/work/matrixone/matrixone/pkg/lockservice/service_test.go:3433 +0x126
2024-05-27T03:47:09.0287987Z github.com/matrixorigin/matrixone/pkg/common/reuse.RunReuseTests(0xc000276da8)
2024-05-27T03:47:09.0288332Z /home/runner/work/matrixone/matrixone/pkg/common/reuse/checker.go:145 +0x5a
2024-05-27T03:47:09.0289190Z github.com/matrixorigin/matrixone/pkg/lockservice.runLockServiceTestsWithLevel({0x121c1c0?, 0xc0065d01a0}, 0xff, {0xc000276e80, 0x1, 0x1}, 0x3b9aca00, 0xc000276e70, 0x0, {0x0, ...})
2024-05-27T03:47:09.0289570Z /home/runner/work/matrixone/matrixone/pkg/lockservice/service_test.go:3432 +0x285
2024-05-27T03:47:09.0290023Z github.com/matrixorigin/matrixone/pkg/lockservice.TestLockSuccWithKeepBindTimeout(0xc0065d01a0)
2024-05-27T03:47:09.0290395Z /home/runner/work/matrixone/matrixone/pkg/lockservice/service_test.go:1346 +0xc5
2024-05-27T03:47:09.0290540Z testing.tRunner(0xc0065d01a0, 0x10a8850)
2024-05-27T03:47:09.0290833Z /opt/hostedtoolcache/go/1.21.5/x64/src/testing/testing.go:1595 +0x262
2024-05-27T03:47:09.0290987Z created by testing.(*T).Run in goroutine 1
2024-05-27T03:47:09.0291276Z /opt/hostedtoolcache/go/1.21.5/x64/src/testing/testing.go:1648 +0x846
2024-05-27T03:47:09.0291281Z
2024-05-27T03:47:09.0291430Z goroutine 49996 [select, 26 minutes]:
2024-05-27T03:47:09.0291888Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0002afb60})
2024-05-27T03:47:09.0292203Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0292616Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0292970Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0293469Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0293825Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0293831Z
2024-05-27T03:47:09.0293963Z goroutine 49988 [select, 26 minutes]:
2024-05-27T03:47:09.0294412Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0002af830})
2024-05-27T03:47:09.0294724Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0295132Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0295475Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0296087Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0296497Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0296502Z
2024-05-27T03:47:09.0296639Z goroutine 49976 [select, 26 minutes]:
2024-05-27T03:47:09.0297108Z github.com/matrixorigin/matrixone/pkg/lockservice.(*detector).doCheck(0xc004df2480, {0x1222f98, 0xc0003c47b0})
2024-05-27T03:47:09.0297450Z /home/runner/work/matrixone/matrixone/pkg/lockservice/deadlock.go:123 +0x29d
2024-05-27T03:47:09.0297848Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0298192Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0298689Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0299037Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0299042Z
2024-05-27T03:47:09.0299178Z goroutine 49983 [select, 26 minutes]:
2024-05-27T03:47:09.0299618Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8e4b0, {0x1222f98, 0xc0003c5ef0})
2024-05-27T03:47:09.0299937Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0300329Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0300679Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0301172Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0301589Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0301594Z
2024-05-27T03:47:09.0301731Z goroutine 50017 [select, 26 minutes]:
2024-05-27T03:47:09.0302172Z github.com/matrixorigin/matrixone/pkg/lockservice.(*server).handle(0xc003b8f2c0, {0x1222f98, 0xc0005b5b30})
2024-05-27T03:47:09.0302488Z /home/runner/work/matrixone/matrixone/pkg/lockservice/rpc.go:375 +0x1e7
2024-05-27T03:47:09.0302878Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0303224Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0303713Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0304062Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0304067Z
2024-05-27T03:47:09.0304195Z goroutine 50007 [select, 26 minutes]:
2024-05-27T03:47:09.0304752Z github.com/matrixorigin/matrixone/pkg/lockservice.(*service).handleFetchWhoWaitingMe(0xc000152780, {0x1222f98, 0xc00010fce0})
2024-05-27T03:47:09.0305118Z /home/runner/work/matrixone/matrixone/pkg/lockservice/service_remote.go:546 +0x13a
2024-05-27T03:47:09.0305627Z github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask.func1()
2024-05-27T03:47:09.0305974Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:277 +0xe8
2024-05-27T03:47:09.0306484Z created by github.com/matrixorigin/matrixone/pkg/common/stopper.(*Stopper).doRunCancelableTask in goroutine 49972
2024-05-27T03:47:09.0306836Z /home/runner/work/matrixone/matrixone/pkg/common/stopper/stopper.go:272 +0x167
2024-05-27T03:47:09.0307098Z FAIL github.com/matrixorigin/matrixone/pkg/lockservice 1800.097s
from matrixone.
Related Issues (20)
- [Bug]: rolling-update CN behind proxy cause prepared stmt lost HOT 1
- [Bug]: force flush failed. HOT 3
- [Bug]: snapshot bvt may panic HOT 3
- [Bug]: restore account HOT 3
- [Bug]: restore account if exists fk table ref other databasee will report "can not drop database. It has been referenced by foreign keys" HOT 1
- [Tech Request]: rename restrict operator to filter
- [Tech Request]: optmize duplicate check memory usage for sql like insert into t1 selct from t2
- [Bug]: stock_level txn timeout in 60s in tpcc 10-10 longrunning during statbility test on distributed HOT 2
- [Bug]: [tke regression] tpcc 100-1000 test oom. HOT 6
- [Bug]: CI/multi cn e2e run bvt `load data LOCAL infile` got FileNotFoundException HOT 4
- [Bug]: UT TestKill got error `panic BUG: StartStatement called twice`
- [Tech Request]: support read from cache without copy HOT 1
- [Tech Request]: add txn id in log
- [Bug]: restore fk table report can not find table by id in ci HOT 1
- [Bug]: Optimizer doesn't support ProjectList in SEMI Join
- [Bug]: restore single db multi table with fk report 'no such table'. HOT 1
- [Bug]: panic during merge when object is empty
- [Bug]: `context deadline exceeded` error occurs HOT 2
- [Bug]: New account used about 16GB storage HOT 2
- [Bug]: restore reported `w-w conflict` during race-enabled bvt test HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from matrixone.