Today I saw What is a database backup (back to the basics), which I probably should have read more closely than I did.
I wanted a backup of my Thunderbird data on my Debian desktop, and I created it like this:
jj5@tact:~$ tar -c -f jj5-.thunderbird.tgz --use-compress-program='pigz -p 7 --best' .thunderbird
So I was running a backup with rsync and I saw this:
lib/mysql/ibdata1 437.40M 33% 4.53MB/s 0:03:10 inflate returned -3 (0 bytes) rsync error: error in rsync protocol data stream (code 12) at token.c(557) [receiver=3.1.2] rsync: connection unexpectedly closed (155602 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(235) [generator=3.1.2]
The issue seems to be that if you’re using rsync compression and the remote file gets changed while the rsync copy is in progress then shit gets corrupted. My solution was to handle error level ’12’ and retry without compression. If the file changes while the rsync is in progress the file will be corrupt, so you shouldn’t rely on the integrity of such files.