framps / raspibackup Goto Github PK

mkfs can check for bad blocks during formatting. This will make sure the SD card used for restore has no bad blocks.

From the log:

20171026-162244: DBG << calcSumSizeFromSFDISK 64013265920
20171026-162244: DBG -- sourceSDSize: 64013265920 - targetSDSize: 64012419072
20171026-162244: DBG -- sourceBlockSize: 124932096 - adjusted targetBlockSize: 124930442
20171026-162244: MSG !!! RBK0004W: Zweite Partition wird von 59.57 GiB auf 59.57 GiB angepasst.
20171026-162244: DBG >> resizeRootFS
20171026-162244: DBG -- RESTORE_DEVICE: /dev/loop0
20171026-162244: DBG -- ROOT_PARTITION: /dev/loop0p2
20171026-162244: DBG -- partitionLayout of /dev/loop0
20171026-162244: DBG -- Disk /dev/loop0: 59,6 GiB, 64012419072 bytes, 125024256 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x19aa3ab1

Device Boot Start End Sectors Size Id Type
/dev/loop0p1 8192 93813 85622 41,8M c W95 FAT32 (LBA)
/dev/loop0p2 94208 125026303 124932096 59,6G 83 Linux
20171026-162245: DBG -- PartitionStart:
20171026-162245: MSG ??? RBK0111E: Fehler beim Erstellen der Partitionen.RC
.
20171026-162245: DBG >> exitError 112
20171026-162245: DBG << exitError 112
20171026-162245: DBG >> cleanup
20171026-162245: DBG >> cleanupRestore
20171026-162245: DBG -- Got trap EXIT
20171026-162245: DBG -- rc: 112
20171026-162245: DBG >> isMounted /tmp/raspiBackup
20171026-162245: DBG -- /dev/root
20171026-162245: DBG << isMounted 1
20171026-162245: DBG -- Deleting dir /tmp/raspiBackup
20171026-162245: MSG ??? RBK0077E: Restore wurde fehlerhaft mit RC 112 beendet. Siehe vorhergehende Fehlermeldungen.
20171026-162245: DBG << cleanupRestore - exit with 112
20171026-162245: DBG >> cleanupTempFiles
20171026-162245: DBG -- Removing mailfile /root/script/raspiBackup.maillog
20171026-162245: DBG << cleanupTempFiles
20171026-162245: DBG -- Terminate now with rc 112

I restored the 'tgz' format file to my new micro sdcard, when finished . put the new micro sdcard into raspberry pi3 . After it power on , the screen show : "failed to start dhcpcd on all interfaces". raspberry can't connect internet , and then I execute command "sudo systemctl disable dhcpcd.service" ,reboot . the raspberry can't connect internet too.
I think if there are some stuff caused inconsistent backup. Everything is ok before backup.
If there should be do any initialization after restore to a new sdcard? It looks like some configurations of network card had changed after backup. such as eth0 ,eth1.

There exist operating systems like armbian or ubuntu which don't have two partitions (/boot and /) and just one with /boot and /.

Right now raspiBackup requires two partitions with /boot and / for both backup modes..

Right now new options have to be added manually in the config files when a new version with new options is published. Would be great to have the new options being added during version update.

This directory is useless when upgrading raspiBackup and shouldn't be created.

Hi i get the error Message ??? RBK0096E: Language C not supported.
How could i fix this ?
Many thanks.

--- RBK0085I: Backup of type tar started. Please be patient.

??? RBK0021E: Backupprogram for type tar failed with RC 2.
--- RBK0007I: Starting services: 'service nfs-common start %1%1 service cron start'.
--- RBK0026I: Saving logfile in /root/raspberrypi-raspiBackup.log.
--- RBK0043I: Removing incomplete backup in /backup/raspberrypi/raspberrypi-tar-backup-20180318-195037. This will take some time. Please be patient.
--- RBK0105I: Deleting new backup directory /backup/raspberrypi/raspberrypi-tar-backup-20180318-195037.
??? RBK0005E: Backup failed. Check previous error messages for details.

See original request in German

Hi,

I'm trying to use raspiBackup but I get this error:

lpuerto@opl-cubox:~$ sudo raspiBackup.sh -a : -o : -m detailed
--- RBK0009I: opl-cubox: raspiBackup.sh V0.6.3.2a (1d77d56) started at Tue Apr 24 13:39:54 EEST 2018.
--- RBK0128I: Using logfile /backup/opl-cubox/opl-cubox-rsync-backup-20180424-133954/opl-cubox-backup.log.
--- RBK0116I: Using config file /usr/local/etc/raspiBackup.conf.
??? RBK0142E: Unable to detect boot device.
--- RBK0026I: Saving logfile in /home/lpuerto/opl-cubox-raspiBackup.log.
--- RBK0105I: Deleting new backup directory /backup/opl-cubox/opl-cubox-rsync-backup-20180424-133954.
??? RBK0005E: Backup failed. Check previous error messages for details.

Instead of using a USB as backup device I'm using a folder in an already mounted hard drive, so:

# path to store the backupfile
DEFAULT_BACKUPPATH="/srv/dev-disk-by-label-OpL1TB/Backups"

Do you know what could be the problem here?

I'm not using it on raspbian, but in armbian.

Hi @framps

I have used your code to backup and restore a raspbian created with NOOBS. It was a dd backup and it has been restore with sudo ./raspiBackup.sh -Z -d DEVICE BACKUP

It looks it's working but the permissions of the filesystem are wrong (root is not the owner anymore):

And I get errors like sudo: /usr/bin/sudo must be owned by uid 0 and have the setuid bit set.

Any idea how can I fix it?

Thank you!

A user using ubuntu reported the useless folders as /proc, /dev, /tmp etc are saved for rsync backup type even they are excluded in the rsync call.

Please check if perl installed. If perl is not installed, then you get empty lines with no information. There is no error message. You only see blank lines. I think of testing for perl in the begining of the script like the check for BASH. Tested on VoidLinux for Raspberry Pi.

sudo mount -o rw /dev/sda1 /backup
raspiBackup.sh

error message:

raspiBackup.sh V0.6.3.2 (81d74e1) started at Sun 18 Mar 19:41:59 HKT 2018.
mkdir: cannot create directory ‘/backup/raspberrypi/raspberrypi-tar-backup-20180318-194159’: Operation not permitted
??? RBK0172E: Unable to create directory /backup/raspberrypi/raspberrypi-tar-backup-20180318-194159.

Thanks for your great tool!

I have a problem with my installation.

I get this error message and do not know how to fix this.

… 20180102-113322: DBG -- mountpoint /boot: 179:1 mountpoint /: 179:2 20180102-113322: DBG -- part: /dev/mmcblk0p1 20180102-113322: DBG -- BOOT_DEVICE: mmcblk0 20180102-113322: DBG -- BOOT_DEVICENAME: /dev/mmcblk0 20180102-113322: DBG >> getPartitionPrefix: mmcblk0 20180102-113322: DBG << getPartitionPrefix: mmcblk0p 20180102-113322: DBG -- BOOT_PARTITION_PREFIX: mmcblk0p 20180102-113322: DBG << inspect4Backup 20180102-113322: DBG >> commonChecks 20180102-113322: DBG << commonChecks 20180102-113322: DBG >> getFsType: /media/nas-backup 20180102-113322: DBG << getFsType: 20180102-113322: DBG -- Found partitions on /dev/mmcblk0: 2 20180102-113322: DBG >> isPathMounted: /media/nas-backup 20180102-113322: DBG -- Path: /media/nas-backup 20180102-113322: DBG << isPathMounted: 0 20180102-113322: DBG >> supportsHardlinks: /media/nas-backup 20180102-113322: DBG -- Links: 2 20180102-113322: DBG << supportsHardlinks: 0 20180102-113322: DBG >> supportsSymlinks: /media/nas-backup 20180102-113322: DBG -- Links: 1 20180102-113322: DBG << supportsSymlinks: 20180102-113322: DBG -- rsync version: 3.1.2 20180102-113322: MSG ??? RBK0019E: Option -a und -o nicht angegeben. 20180102-113322: DBG >> exitError 107 20180102-113322: DBG << exitError 107 20180102-113322: DBG >> cleanup 20180102-113322: DBG >> cleanupBackup 20180102-113322: DBG -- Got trap EXIT 20180102-113322: DBG -- rc: 107 …

I hope you could help me.

Best,
Jan

raspiBackup has a lot of options but there is no way to retrieve the current version. -v is already used to enable verbose mode.
The option parser should be enhanced to accept option --version to return the current version of raspiBackup.

Hi,

I am running OSMC on a Rasp Pi 3. I just typed : curl -s -L -O https://www.linux-tips-and-tricks.de/raspiBackupInstall.sh && sudo bash raspiBackupInstall.sh
I received the fellowing error: Installation: line 248: !msgVar: unbound variable
It is a clean install, I have just install transmission client and amba via the app store, mysql via apt-get and syncthing.

Did I miss something or is it a bug?

Check if all used commands are available (e.g. parted, fdisk, fsck.vfat, ...)

Backup type rsync requires the backup space to be mounted via rsync. raspiBackup doesn't refuse to use rsync with a samba mounted backup space and creates headache for raspiBackup users because the backup seems to hang (looks like the hang is some rsync issue)

Send also an eMail if there is a new version available

in essence

1. add a timestamp to each log line generated
2. remove timestamping information inside the log itself

format of timestamp, I suggest a simple, easy to parse if needed
date +'%m-%d-%Y %T'

example instead of this
--- RBK0009I: jeedom-cugy: raspiBackup.sh V0.6.3.1 (590ae9c) started at Fri Mar 9 05:00:01 CET 2018.
--- RBK0128I: Using logfile /mnt/usblogs/jeedom-cugy/jeedom-cugy-rsync-backup-20180309-050001/jeedom-cugy-backup.log.
--- RBK0116I: Using config file /usr/local/etc/raspiBackup.conf.
--- RBK0031I: Checking whether new version is available.
--- RBK0159I: Checking for beta version.

generate this
03-09-2018 08:13:21 - RBK0009I: jeedom-cugy: raspiBackup.sh V0.6.3.1 (590ae9c) started
03-09-2018 08:13:21 - RBK0128I: Using logfile /mnt/usblogs/jeedom-cugy/jeedom-cugy-rsync-backup-20180309-050001/jeedom-cugy-backup.log.
03-09-2018 08:13:22 - RBK0116I: Using config file /usr/local/etc/raspiBackup.conf.
03-09-2018 08:13:22 - RBK0031I: Checking whether new version is available.
03-09-2018 08:13:22 - RBK0159I: Checking for beta version.

Starting with Jessie a bsd tar is shipped with raspbian which supports acls and xattrs. There is no support by raspiBackup.

Example commandline:
sudo /usr/local/bin/raspiBackup.sh -F -o "service avahi-daemon stop" -a ":" -k 3 -t ddz -L current -l debug -m detailed -v /backup
leads to the error
??? RBK0090E: Option -o requires a parameter.

This is caused by the dash (-) in the avahi-daemon string.
The same command with service cron stop works as expected.
The issue also regards to the -a parameter.

This is version
Version: 0.6.3.2 CommitSHA: 81d74e1 CommitDate: 2018-03-11 CommitTime: 08:35:59

lg, Christian

Numbers are empty. Root cause: Format for free command changed.

Right now options can be turned on or toggled via command line options. There is no way to turn an option off via command if it's turned on in a config file.

Before using the script on my precious pi projects I wanted to try it on a dummy VM first. It seems that it requires a /dev/sd to be present. Is the script really only applicable on pi?

dd includes the risk of bad filesystem and corrupt image contents.
tar would avoid this.

tar is a pretty old tool. Investigate whether fsarchiver is a better tool alternative than tar.

I wanted to test a simple restore (.img file to device) but the script seems to be hanging when the user should confirm the restore:

--- RBK0009I: beemon: raspiBackup.sh V0.6.3.1 (590ae9c) started at Thu 14 Dec 12:48:07 CET 2017.
--- RBK0128I: Using logfile /raspiBackup.log.
--- RBK0116I: Using config file /usr/local/etc/raspiBackup.conf.
--- RBK0138I: Using bootbackup /backups/dd-backup-20171213-190402.img.
--- RBK0068I: Using bootpartition backup files starting with backup from directory /backups/dd-backup-20171213-190402.img.
!!! RBK0065W: Device /dev/sda will be repartitioned and all data will be lost.

I have to ctrl+c the script. I also noticed that the bash does not show any inputs anymore after stopping the script. Only reset revolves this issue. Seems to me that there could be some issues with reading from tty. Maybe I will find some time later to go deeper and find the root cause.

Or is there a known workaround?

Right now the sender eMailAddress is hard coded. Make it configurable.

Hello framp!

For simplification of a restore, I'd like to request that every backup type should copy the /etc/fstab file as an extra file into the backup directory as plain text file.

I've noticed that especially "Windows" guys (that also involves me) have troubles to recover the mounts to re-access the backup volume in case of a restore (or at least it is time consuming to figure out the settings).
With a plain copy of the fstab it is a copy/paste job to re-enable the backup mount to further process the restore operation.

Regards, Christian

Given actual version 0.6.3 the config file will be renamed into raspiBackup_0.6.3.conf even the old raspiBackup version is 0.5.8.

rsync doesn't make sense if hardlinks and symlinks are not possible. It's ossible to run raspiBackup even symlink error message is displayed (See #18)

raspiBackupRestore2Image works on Jessie but fails on Wheezy

Hi,

currently I won't use it, but was impressed about what you're working on, and also about language Go.

So this is just a recommendation to make port configurable, not fixed to 8080, as many tools use this port.

https://github.com/framps/raspiBackup/blob/master/RESTAPI/raspiBackupRESTListener.go#L167

cu, Christian

rsync: symlink "/media/nas-backup/raspberrypi/raspberrypi-rsync-backup-20171024-135259/usr/bin/mailx" -> "/etc/alternatives/mailx" failed: Operation not supported (95)

Environment: rsync backup on a samba mounted partition

on a beaglebone black you can install your linux on "one" partiton. Specialy to the on-board EMMC storage. You get the error ??? RBK0142E: Unable to detect boot device. The /boot is on the / file system and there is no seperate mountpoint for /boot. This beaglebone is running Parabola Linux.
onefilesystem.txt

When I start a backup via cron or manually, it runns smoothly for quite some time.

--- RBK0009I: orangepipc: raspiBackup.sh V0.6.1.3b um Sun Feb 5 20:48:25 CET 2017 gestartet
--- RBK0128I: Logdatei ist /mnt/BackupNas/orangepipc/orangepipc-tgz-backup-20170205-204823/orangepipc-backup.log
--- RBK0116I: Konfigurationsdatei /usr/local/etc/raspiBackup.conf wird benutzt
--- RBK0008I: Services werden gestoppt: '/etc/init.d/samba stop'
--- RBK0081I: Backup vom Typ tgz wird in /mnt/BackupNas/orangepipc/orangepipc-tgz-backup-20170205-204823 erstellt
--- RBK0085I: Backuperstellung vom Typ tgz läuft. Bitte etwas Geduld
--- RBK0036I: Partitionslayout wird gesichert
--- RBK0044I: Backup der Bootpartition wird in /mnt/BackupNas/orangepipc/orangepipc-tgz-backup-20170205-204823/orangepipc-backup.img erstellt
--- RBK0044I: Backup des Partitionlayouts wird in /mnt/BackupNas/orangepipc/orangepipc-tgz-backup-20170205-204823/orangepipc-backup.sfdisk erstellt
--- RBK0046I: Backup des Masterbootrecords wird in /mnt/BackupNas/orangepipc/orangepipc-tgz-backup-20170205-204823/orangepipc-backup.mbr erstellt

After a while, a new backup is started every minute.
This is an extract from ps -ef:

root 16412 1 0 22:02 ? 00:00:00 /bin/bash /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas
root 16528 16412 0 22:02 ? 00:00:00 /bin/bash /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas
root 16529 16412 0 22:02 ? 00:00:00 /bin/bash /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas
root 16530 16528 0 22:02 ? 00:00:00 tee -a /home//orangepipc-raspiBackup.log
root 16532 16529 0 22:02 ? 00:00:00 tee -a /home//orangepipc-raspiBackup.log
root 16752 16412 1 22:02 ? 00:00:04 dd if=/dev/mmcblk0p1 of=/mnt/BackupNas/orangepipc/orangepipc-tgz-backup-20170205-220202/orangepipc-backup.img bs=1M
root 16815 1 0 22:03 ? 00:00:00 /bin/bash /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas
root 16915 16815 0 22:03 ? 00:00:00 /bin/bash /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas
root 16916 16815 0 22:03 ? 00:00:00 /bin/bash /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas
root 16917 16915 0 22:03 ? 00:00:00 tee -a /home//orangepipc-raspiBackup.log
root 16918 16916 0 22:03 ? 00:00:00 tee -a /home//orangepipc-raspiBackup.log
root 17128 16815 0 22:03 ? 00:00:03 dd if=/dev/mmcblk0p1 of=/mnt/BackupNas/orangepipc/orangepipc-tgz-backup-20170205-220302/orangepipc-backup.img bs=1M
root 17207 1 0 22:04 ? 00:00:00 /bin/bash /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas
root 17323 17207 0 22:04 ? 00:00:00 /bin/bash /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas
root 17324 17207 0 22:04 ? 00:00:00 /bin/bash /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas
root 17325 17323 0 22:04 ? 00:00:00 tee -a /home//orangepipc-raspiBackup.log
root 17327 17324 0 22:04 ? 00:00:00 tee -a /home//orangepipc-raspiBackup.log
root 17547 17207 0 22:04 ? 00:00:02 dd if=/dev/mmcblk0p1 of=/mnt/BackupNas/orangepipc/orangepipc-tgz-backup-20170205-220401/orangepipc-backup.img bs=1M
root 17622 1 0 22:05 ? 00:00:00 /bin/bash /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas
root 17713 17622 0 22:05 ? 00:00:00 /bin/bash /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas
root 17714 17713 0 22:05 ? 00:00:00 tee -a /home//orangepipc-raspiBackup.log
root 17715 17622 0 22:05 ? 00:00:00 /bin/bash /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas
root 17717 17715 0 22:05 ? 00:00:00 tee -a /home//orangepipc-raspiBackup.log
root 17937 17622 0 22:05 ? 00:00:02 dd if=/dev/mmcblk0p1 of=/mnt/BackupNas/orangepipc/orangepipc-tgz-backup-20170205-220501/orangepipc-backup.img bs=1M
root 18026 1 0 22:06 ? 00:00:00 /bin/bash /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas
root 18168 18026 0 22:06 ? 00:00:00 /bin/bash /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas
root 18169 18026 0 22:06 ? 00:00:00 /bin/bash /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas

This is the command executed: /usr/local/bin/raspiBackup.sh -t tar -e [email protected] -k 30 -z -n /mnt/BackupNas &> /dev/null
I am running an OrangePi PC with Armbian.

If a user can restore a backup into an image file it's possible to use pishrink to create a minimized image of tar/rasync backup.

Create a tool which creates an image file, restores a tar or rsync backup and executes pishrink in order to minimize the restored image file.

This requires #12 to be implemented

As of now raspiBackup requires a SD card on /dev/mmcblk0. raspi3 allows to run without SD card. Users with raspi3 cannot use raspiBackup.

Root cause is that Jessie uses partuuid instead of partition names now. Therefore a dumb restore of the backup doesn't work any more and some postprocessing of /boot/cmdline.txt and /etc/fstab is required when the backup was restored.

Hello Framp

Is it possible to integrate an program like pv to see the Status from the raspibackup when your are starting the program from the commandline ?

for ex. : dd if=/dev/sdX | pv | dd of=/pfad/zur/datei

btw. i tried to create an message on your Website but i cannot see any Picture for captcha verification ..

regards John

Example: -L /home/pi/logs/raspiBackup.log

Extended attributes cannot be saved via NFS. NFS doesn't support this. Actually the extended attributes are for 99% of the raspiBackup users not needed and they have to manually modify the option

DEFAULT_RSYNC_BACKUP_OPTIONS="-aHAXx"
into
DEFAULT_RSYNC_BACKUP_OPTIONS="-aHAx"