20220414-speak_of_the_dedup.md (2431B)
1 We all have our own robust and reliable backup solutions so there's no point in me telling you how important backups are, or how _more_ important a working restore is. 2 3 The /home partition on my main Linux system is a 3 disk RAID5 array giving me 3TB of storage with some resilience. I use [dedup](https://git.z3bra.org/dedup/file/README.html) for backing up important files on a daily basis, combined with [rclone](https://rclone.org/) to my cloud storage giving me an offsite backup. Every so often I will do a full backup to an external 5TB encrypted drive, which lives in my bug out bag. 4 5 To use `dedup` you have to first initialise a repository 6 ``` 7 dup-init <repo> 8 ``` 9 10 Encryption is always a good idea so instead you can generate a secret key, then use the key to initialise a repo 11 ``` 12 dup-keygen <keyfile> 13 dub-init -k <keyfile> -E XChaCha20-Poly1305 <repo> 14 ``` 15 16 Make sure you keep that key safe and secure, it is required for restoring. 17 18 It is best to use `tar` for backing up directories as `dedup` only handles single files. To backup a directory into the new repo incant 19 ``` 20 tar -cf - /path/to/dir | dup-pack -k <keyfile> -r <repo> <snapshot_name> 21 ``` 22 23 The `snapshot_name` can be anything you want, I like to use the date. 24 25 Restoring a backup is straightforward, using `tar` again to extract the archive 26 ``` 27 dup-unpack -k <keyfile> -r <repo> <snapshot_name> | tar -xf - 28 ``` 29 30 My daily cronjob runs a script to backup a list of important files and directories then sync the backups with my cloud storage using `rclone`. 31 32 Once `rclone` is authenticated with your cloud storage the command to copy is literally that 33 ``` 34 rclone copy /local/path/to/backups <storage_name>:/remote/path 35 ``` 36 37 My script uses [ntfy](https://ntfy.sh/) to alert me if a backup fails (see my [previous post on ntfy](20220104-notify,_only_smaller.html)). 38 39 My full backup is not automated as it requires unlocking my external drive, so I don't do this as often as I would like. I usually pick a weekend every couple of months when I will get the drive out and back up all my devices. 40 41 I try to run a practice restore every few weeks to ensure everything is still in good form and the process becomes habit. Panicking over lost data is never fun, so at least if I can perform a restore calmly I won't be completely losing my mind. 42 43 This system has been working well for me for a long time. I try not to overcomplicate things, K.I.S.S. as they say.