First, thank you for the great project. I found it easy to use so far and it solved an

if you still use the the whitelist in the linked ticket you should check the se

Thank you for the hint <a class="user-mention notranslate" data-hovercard-type="user"

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

I am a little further with debugging. This error also happens

(<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/u

Finalizing this post. The issue had nothing to do with SSH <a class="user-mention notr

Randomly failing syncoid pulls with `CRITICAL ERROR ... failed: 256 at /usr/sbin/syncoid line 889.`,about jimsalterjrs/sanoid

Comments (8)

phreaker0 commented on June 1, 2024 1

if you still use the the whitelist script in the linked ticket you should check the server logs for "non-whitelisted command issued" log lines.

from sanoid.

phreaker0 commented on June 1, 2024

256 shifted by 8 bit is 1 which for ssh means "generic error". There is also no zfs error output so this seems to be an ssh issues and not a syncoid one. Probably the same type of error you had here: #879

from sanoid.

Sieboldianus commented on June 1, 2024

Thank you for the hint @phreaker0 - this helps to narrow down the source of the issue. It is a basic SSH connection, nothing limited or restricted as in the referenced issue, so I am still a bit clueless..

from sanoid.

Sieboldianus commented on June 1, 2024

@phreaker0 Thank you. I don't use the whitelist script. I am trying to first get Syncoid to work properly without it. At the moment, I do not know what is causing these "generic ssh errors". It is strange that the error only appears every second pull. Perhaps something like this - I will debug this further and report.

SSH Server logs appear fine:

2024-01-31T06:06:45.992059+01:00 main sshd[3120541]: Accepted publickey for zfs-backup from ....10.7 port 40266 ssh2: ED25519 SHA256:3Vn...
2024-01-31T06:06:46.000812+01:00 main sshd[3120541]: pam_unix(sshd:session): session opened for user zfs-backup(uid=996) by (uid=0)
2024-01-31T06:06:46.006517+01:00 main systemd-logind[2799]: New session 250962 of user zfs-backup.
2024-01-31T06:06:46.049796+01:00 main (systemd): pam_unix(systemd-user:session): session opened for user zfs-backup(uid=996) by (uid=0)
2024-01-31T06:06:46.304763+01:00 main sshd[3120541]: pam_env(sshd:session): deprecated reading of user environment enabled
2024-01-31T06:06:46.996109+01:00 main sshd[3120567]: Received disconnect from ....10.7 port 40266:11: disconnected by user
2024-01-31T06:06:46.996735+01:00 main sshd[3120567]: Disconnected from user zfs-backup ....10.7 port 40266
2024-01-31T06:06:46.996965+01:00 main sshd[3120541]: pam_unix(sshd:session): session closed for user zfs-backup

from sanoid.

Sieboldianus commented on June 1, 2024

I am a little further with debugging.

This error also happens when regularly calling syncoid, not just with an additional script
I have enabled LogLevel DEBUG3 and passed --sshconfig=~/.ssh/config, which confirms the SSH error
See ssh error below

Click to see

DEBUG: SSHCMD: ssh -F ~/.ssh/config    -i /home/recvuser/.ssh/id_ed25519
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/recvuser/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/recvuser/.ssh/known_hosts2'
debug3: ssh_connect_direct: entering
debug1: Connecting to 192.168.100.100 [192.168.100.100] port 22.
debug3: set_sock_tos: set socket 3 IP_TOS 0x10
debug1: Connection established.
debug1: identity file /home/recvuser/.ssh/id_ed25519 type 3
debug1: identity file /home/recvuser/.ssh/id_ed25519-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u2
debug1: Remote protocol version 2.0, remote software version OpenSSH_9.2p1 Debian-2+deb12u2
debug1: compat_banner: match: OpenSSH_9.2p1 Debian-2+deb12u2 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to 192.168.100.100:22 as 'zfs-backup'
debug3: record_hostkey: found key type ED25519 in file /home/recvuser/.ssh/known_hosts:1
debug3: record_hostkey: found key type RSA in file /home/recvuser/.ssh/known_hosts:2
debug3: record_hostkey: found key type ECDSA in file /home/recvuser/.ssh/known_hosts:3
debug3: load_hostkeys_file: loaded 3 keys from 192.168.100.100
debug1: load_hostkeys: fopen /home/recvuser/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug3: order_hostkeyalgs: have matching best-preference key type [email protected], using HostkeyAlgorithms verbatim
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: [email protected],curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c,[email protected]
debug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],rsa-sha2-512,rsa-sha2-256
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,[email protected],zlib
debug2: compression stoc: none,[email protected],zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: [email protected],curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,[email protected]
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,[email protected]
debug2: compression stoc: none,[email protected]
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug3: kex_choose_conf: will use strict KEX ordering
debug1: kex: algorithm: [email protected]
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:ROWPirqvnDTMlPkMNnkyydn0mjftM3Q/Jd67ohiXeos
debug3: record_hostkey: found key type ED25519 in file /home/recvuser/.ssh/known_hosts:1
debug3: record_hostkey: found key type RSA in file /home/recvuser/.ssh/known_hosts:2
debug3: record_hostkey: found key type ECDSA in file /home/recvuser/.ssh/known_hosts:3
debug3: load_hostkeys_file: loaded 3 keys from 192.168.100.100
debug1: load_hostkeys: fopen /home/recvuser/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host '192.168.100.100' is known and matches the ED25519 host key.
debug1: Found key in /home/recvuser/.ssh/known_hosts:1
debug3: send packet: type 21
debug1: ssh_packet_send2_wrapped: resetting send seqnr 3
debug2: ssh_set_newkeys: mode 1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: ssh_packet_read_poll2: resetting read seqnr 3
debug1: SSH2_MSG_NEWKEYS received
debug2: ssh_set_newkeys: mode 0
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /home/recvuser/.ssh/id_ed25519 ED25519 SHA256:3Vndp99lQO1ERwL3T+6H2oHj+....../......AnnQ explicit
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],ssh-dss,ssh-rsa,rsa-sha2-256,rsa-sha2-512>
debug1: kex_input_ext_info: [email protected]=<0>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/recvuser/.ssh/id_ed25519 ED25519 SHA256:3Vndp99lQO1ERwL3T+6H2oHj+....../...... explicit
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: /home/recvuser/.ssh/id_ed25519 ED25519 SHA256:3Vndp99lQO1ERwL3T+6H2oHj+....../...... explicit
debug3: sign_and_send_pubkey: using [email protected] with ED25519 SHA256:3Vndp99lQO1ERwL3T+......+....../......
debug3: sign_and_send_pubkey: signing using ssh-ed25519 SHA256:3Vndp99lQO1ERwL3T+6H2oHj+....../......
debug3: send packet: type 50
debug3: receive packet: type 52
Authenticated to 192.168.100.100 ([192.168.100.100]:22) using "publickey".
debug1: setting up multiplex master socket
debug3: muxserver_listen: temporary control path /tmp/syncoid-proxmox-1706847290-3540.QM3NPZQALAKgbGwk
debug2: fd 4 setting O_NONBLOCK
debug3: fd 4 is O_NONBLOCK
debug3: fd 4 is O_NONBLOCK
debug1: channel 0: new mux listener [/tmp/syncoid-proxmox-1706847290-3540] (inactive timeout: 0)
debug3: muxserver_listen: mux listener channel 0 fd 4
debug2: fd 3 setting TCP_NODELAY
debug3: set_sock_tos: set socket 3 IP_TOS 0x08
debug1: control_persist_detach: backgrounding master process
debug2: control_persist_detach: background process is 35831
debug2: fd 4 setting O_NONBLOCK
debug2: mux_client_hello_exchange: master version 4
debug3: mux_client_forwards: request forwardings: 0 local, 0 remote
debug3: mux_client_request_session: entering
debug3: mux_client_request_alive: entering
debug3: mux_client_request_alive: done pid = 35833
debug3: mux_client_request_session: session request sent
debug1: mux_client_request_session: master session id: 2
debug3: mux_client_read_packet: read header failed: Broken pipe
debug2: Received exit status from master 0
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/recvuser/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/recvuser/.ssh/known_hosts2'
debug2: fd 3 setting O_NONBLOCK
debug2: mux_client_hello_exchange: master version 4
debug3: mux_client_forwards: request forwardings: 0 local, 0 remote
debug3: mux_client_request_session: entering
debug3: mux_client_request_alive: entering
debug3: mux_client_request_alive: done pid = 35833
debug3: mux_client_request_session: session request sent
debug1: mux_client_request_session: master session id: 2
debug3: mux_client_read_packet: read header failed: Broken pipe
debug2: Received exit status from master 0
DEBUG: checking availability of lzop on source...
DEBUG: checking availability of lzop on target...
DEBUG: checking availability of lzop on local machine...
DEBUG: checking availability of mbuffer on source...
DEBUG: checking availability of mbuffer on target...
DEBUG: checking availability of pv on local machine...
DEBUG: checking availability of zfs resume feature on source...
DEBUG: checking availability of zfs resume feature on target...
DEBUG: syncing source rpool/ROOT/pve-1 to target tank/proxmox/rpool/ROOT/pve-1.
DEBUG: getting current value of syncoid:sync on rpool/ROOT/pve-1...
ssh -F ~/.ssh/config    -i /home/recvuser/.ssh/id_ed25519 -S /tmp/syncoid-proxmox-1706847290-3540 proxmox  zfs get -H syncoid:sync ''"'"'rpool/ROOT/pve-1'"'"''
DEBUG: checking to see if tank/proxmox/rpool/ROOT/pve-1 on  is already in zfs receive using  ps -Ao args= ...
DEBUG: checking to see if target filesystem exists using "  zfs get -H name 'tank/proxmox/rpool/ROOT/pve-1' 2>&1 |"...
DEBUG: getting current value of receive_resume_token on tank/proxmox/rpool/ROOT/pve-1...
  zfs get -H receive_resume_token 'tank/proxmox/rpool/ROOT/pve-1'
DEBUG: no receive token found
DEBUG: getting list of snapshots on rpool/ROOT/pve-1 using ssh -F ~/.ssh/config    -i /home/recvuser/.ssh/id_ed25519 -S /tmp/syncoid-proxmox-1706847290-3540 proxmox  zfs get -Hpd 1 -t snapshot guid,creation ''"'"'rpool/ROOT/pve-1'"'"'' |...
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/recvuser/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/recvuser/.ssh/known_hosts2'
debug2: fd 3 setting O_NONBLOCK
debug2: mux_client_hello_exchange: master version 4
debug3: mux_client_forwards: request forwardings: 0 local, 0 remote
debug3: mux_client_request_session: entering
debug3: mux_client_request_alive: entering
debug3: mux_client_request_alive: done pid = 35833
debug3: mux_client_request_session: session request sent
debug1: mux_client_request_session: master session id: 2
debug3: mux_client_read_packet: read header failed: Broken pipe
debug2: Received exit status from master 0
DEBUG: getting list of snapshots on tank/proxmox/rpool/ROOT/pve-1 using   zfs get -Hpd 1 -t snapshot guid,creation 'tank/proxmox/rpool/ROOT/pve-1' |...
NEWEST SNAPSHOT: autosnap_2024-02-02_00:00:19_daily
DEBUG: getting current value of -p used on tank/proxmox/rpool/ROOT/pve-1...
  zfs get -H -p used 'tank/proxmox/rpool/ROOT/pve-1'
DEBUG: checking to see if tank/proxmox/rpool/ROOT/pve-1 on  is already in zfs receive using  ps -Ao args= ...
DEBUG: getting estimated transfer size from source -S /tmp/syncoid-proxmox-1706847290-3540 proxmox using "ssh -F ~/.ssh/config    -i /home/recvuser/.ssh/id_ed25519 -S /tmp/syncoid-proxmox-1706847290-3540 proxmox  zfs send -R -w  -nvP -I ''"'"'rpool/ROOT/pve-1@autosnap_2024-01-31_00:00:19_daily'"'"'' ''"'"'rpool/ROOT/pve-1@autosnap_2024-02-02_00:00:19_daily'"'"'' 2>&1 |"...
DEBUG: sendsize =
Sending incremental rpool/ROOT/pve-1@autosnap_2024-01-31_00:00:19_daily ... autosnap_2024-02-02_00:00:19_daily (~ UNKNOWN):
DEBUG: ssh -F ~/.ssh/config    -i /home/recvuser/.ssh/id_ed25519 -S /tmp/syncoid-proxmox-1706847290-3540 proxmox ' zfs send -R -w  -I '"'"'rpool/ROOT/pve-1'"'"'@'"'"'autosnap_2024-01-31_00:00:19_daily'"'"' '"'"'rpool/ROOT/pve-1'"'"'@'"'"'autosnap_2024-02-02_00:00:19_daily'"'"' | lzop  | mbuffer  -q -s 128k -m 16M' | mbuffer  -q -s 128k -m 16M | lzop -dfc | pv -p -t -e -r -b -s 0 |  zfs receive  -s -F 'tank/proxmox/rpool/ROOT/pve-1' 2>&1
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/recvuser/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/recvuser/.ssh/known_hosts2'
debug2: fd 3 setting O_NONBLOCK
debug2: mux_client_hello_exchange: master version 4
debug3: mux_client_forwards: request forwardings: 0 local, 0 remote
debug3: mux_client_request_session: entering
debug3: mux_client_request_alive: entering
debug3: mux_client_request_alive: done pid = 35833
debug3: mux_client_request_session: session request sent
debug1: mux_client_request_session: master session id: 2
debug3: mux_client_read_packet: read header failed: Broken pipe                                                                                                                      ]
debug2: Received exit status from master 0
 289MiB 0:00:04 [69.8MiB/s] [              <=>                                                                                                                                       ]
CRITICAL ERROR: ssh -F ~/.ssh/config    -i /home/recvuser/.ssh/id_ed25519 -S /tmp/syncoid-proxmox-1706847290-3540 proxmox ' zfs send -R -w  -I '"'"'rpool/ROOT/pve-1'"'"'@'"'"'autosnap_2024-01-31_00:00:19_daily'"'"' '"'"'rpool/ROOT/pve-1'"'"'@'"'"'autosnap_2024-02-02_00:00:19_daily'"'"' | lzop  | mbuffer  -q -s 128k -m 16M' | mbuffer  -q -s 128k -m 16M | lzop -dfc | pv -p -t -e -r -b -s 0 |  zfs receive  -s -F 'tank/proxmox/rpool/ROOT/pve-1' 2>&1 failed: 256 at /usr/sbin/syncoid line 889.

The things seem to start going wrong at:

debug3: mux_client_read_packet: read header failed: Broken pipe

The ssh connection seems to be killed in the middle of the transfer:

debug3: mux_client_request_alive: done pid = 160718
debug3: mux_client_request_session: session request sent
debug1: mux_client_request_session: master session id: 2
debug3: mux_client_read_packet: read header failed: Broken pipe                <=>                                                                                                   ]
debug2: Received exit status from master 0
1.72GiB 0:00:18 [97.8MiB/s] [                                                     <=>                                                                                                ]
CRITICAL ERROR: ssh -F ~/.ssh ...

I have added ControlPersist 30000 to ~/.ssh/config, but no difference.

from sanoid.

Sieboldianus commented on June 1, 2024

(@phreaker0 ) I think I solved it. First run without CRITICAL ERROR!

[Edit]: False Alarm. Still not working.. * head bang * . All first syncoid runs for each dataset, with or without running syncoid in a script/loop or not, break.

What I did is adding ControlMaster auto and ControlPersist yes to /home/recvuser/.ssh/config
and then using --sshconfig=/home/recvuser/.ssh/config as syncoid argument.

This is my .ssh/config now:

Host proxmox
    Hostname 192.168.100.100
    User zfs-backup
    LogLevel DEBUG3
    ControlPersist yes
    ControlMaster auto

and this my syncoid command:

syncoid --no-sync-snap --no-privilege-elevation \
    --sshkey /home/recvuser/.ssh/id_ed25519 \
    --sendoptions=Rw   --debug \
    --sshconfig=/home/recvuser/.ssh/config \
    proxmox:tank_hdd/data tank/hdd/data

There were very few reports online about this error. I found some info here, here, here and here.

But it seems "ControlMaster" is not honored, this get somehow overwritten by Syncoid SSH call, which is also requesting Master mode.

from sanoid.

Sieboldianus commented on June 1, 2024

Next step: I do think I solved it now.

First, I went through the docs again and saw that for rootless,
I was missing rollback,destroy zfs permissions for my recvuser (on the receiving end). I thought that since I am using --no-sync-snap, these were not needed. After adding those, I had a single run without error.

Further, I noticed that something is wrong with my systemd-logind service:

sudo systemctl status systemd-logind.service
> × systemd-logind.service - User Login Management
>      Loaded: loaded (/lib/systemd/system/systemd-logind.service; static)
>      Active: failed (Result: exit-code) since Sat 2024-02-03 05:29:20 CET; 15s ago
>        Docs: man:sd-login(3)
>              man:systemd-logind.service(8)
>              man:logind.conf(5)
>              man:org.freedesktop.login1(5)
>     Process: 1048027 ExecStart=/lib/systemd/systemd-logind (code=exited, status=1/FAILURE)
>    Main PID: 1048027 (code=exited, status=1/FAILURE)
>         CPU: 31ms
> 
> Feb 03 05:29:20 bserver systemd[1]: systemd-logind.service: Scheduled restart job, restart counter is at 5.
> Feb 03 05:29:20 bserver systemd[1]: Stopped systemd-logind.service - User Login Management.
> Feb 03 05:29:20 bserver systemd[1]: systemd-logind.service: Start request repeated too quickly.
> Feb 03 05:29:20 bserver systemd[1]: systemd-logind.service: Failed with result 'exit-code'.
> Feb 03 05:29:20 bserver systemd[1]: Failed to start systemd-logind.service - User Login Management.

Debugging this was difficult. E.g. see:

journalctl _PID=1048027  
> Feb 03 05:29:20 bserver systemd-logind[1048027]: Failed to connect to system bus: No such file or directory
> Feb 03 05:29:20 bserver systemd-logind[1048027]: Failed to fully start up daemon: No such file or directory

I eventually found this and then noticed that I pulled the rootpool-dataset without unsetting its mountpoint, so it was overwriting my backup-server's mountpoint during boot. Doh!

After fixing it:

zfs set mountpoint=none tank/proxmox/rpool/ROOT/pve-1

I haven't seen any syncoid errors so far. Keeping fingers crossed.

Lastly, I also noticed in the syncoid logs, that it seems to have problems getting the size of the snapshot:

DEBUG: getting estimated transfer size from source -S /tmp/syncoid-proxmox-1706933175-6238 proxmox using "ssh -F /home/recvuser/.ssh/config    -i /home/recvuser/.ssh/id_ed25519 -S /tmp/syncoid-proxmox-1706933175-6238 proxmox  zfs send -R -w  -nvP -I ''"'"'tank_hdd/data@autosnap_2024-02-02_18:00:18_hourly'"'"'' ''"'"'tank_hdd/data@autosnap_2024-02-03_04:00:18_hourly'"'"'' 2>&1 |"...
DEBUG: sendsize =
Sending incremental tank_hdd/data@autosnap_2024-02-02_18:00:18_hourly ... autosnap_2024-02-03_04:00:18_hourly (~ UNKNOWN):

Note the UNKNOWN and sendsize =. I have no clue why this is happening or if it has consequences.

from sanoid.

Sieboldianus commented on June 1, 2024

Finalizing this post. The issue had nothing to do with SSH @phreaker0

I was missing ZFS permissions for rollback and destroy on the receiving side.

I reproduced this. The error would not appear with these allow rules:

sudo zfs allow -u recvuser compression,mountpoint,create,mount,receive,rollback,destroy tank/hdd/data
sudo zfs allow -u recvuser compression,mountpoint,create,mount,receive,rollback,destroy tank/hdd/test
sudo zfs allow -u recvuser compression,mountpoint,create,mount,receive,rollback,destroy tank/hdd/lxc/lbsndb_docker

but it does appear after I set this:

sudo zfs unallow -u recvuser rollback,destroy tank/hdd/data
sudo zfs unallow -u recvuser rollback,destroy tank/hdd/test
sudo zfs unallow -u recvuser rollback,destroy tank/hdd/lxc/lbsndb_docker

I don't know why this produces an ssh error, but apparently it does.

I also don't know why my recvuser rollback and destroy, since I am using --no-sync-snap. Anyway, reason found.

Thanks for all the work with syncoid!

from sanoid.

Randomly failing syncoid pulls with `CRITICAL ERROR ... failed: 256 at /usr/sbin/syncoid line 889.` about sanoid HOT 8 CLOSED

Comments (8)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent