pyratelog

commit 330081bc8dc4fec6c60cf7b68174066124d58f63
parent d8e70621e7021b1991a54a443fb3914b1be34ed2
Author: pyratebeard <root@pyratebeard.net>
Date:   Thu, 14 Apr 2022 21:06:53 +0100

speak_of_the_dedup

Diffstat:
A
entry/speak_of_the_dedup.md
 | 
43
+++++++++++++++++++++++++++++++++++++++++++

1 file changed, 43 insertions(+), 0 deletions(-)
diff --git a/entry/speak_of_the_dedup.md b/entry/speak_of_the_dedup.md
@@ -0,0 +1,43 @@
+We all have our own robust and reliable backup solutions so there's no point in me telling you how important backups are, or how _more_ important a working restore is.
+
+The /home partition on my main Linux system is a 3 disk RAID5 array giving me 3TB of storage with some resilience.  I use [dedup](https://git.z3bra.org/dedup/file/README.html) for backing up important files on a daily basis, combined with [rclone](https://rclone.org/) to my cloud storage giving me an offsite backup.  Every so often I will do a full backup to an external 5TB encrypted drive, which lives in my bug out bag.
+
+To use `dedup` you have to first initialise a repository
+```
+dup-init <repo>
+```
+
+Encryption is always a good idea so instead you can generate a secret key, then use the key to initialise a repo
+```
+dup-keygen <keyfile>
+dub-init -k <keyfile> -E XChaCha20-Poly1305 <repo>
+```
+
+Make sure you keep that key safe and secure, it is required for restoring.
+
+It is best to use `tar` for backing up directories as `dedup` only handles single files.  To backup a directory into the new repo incant
+```
+tar -cf - /path/to/dir | dup-pack -k <keyfile> -r <repo> <snapshot_name>
+```
+
+The `snapshot_name` can be anything you want, I like to use the date.
+
+Restoring a backup is straightforward, using `tar` again to extract the archive
+```
+dup-unpack -k <keyfile> -r <repo> <snapshot_name> | tar -xf -
+```
+
+My daily cronjob runs a script to backup a list of important files and directories then sync the backups with my cloud storage using `rclone`.
+
+Once `rclone` is authenticated with your cloud storage the command to copy is literally that
+```
+rclone copy /local/path/to/backups <storage_name>:/remote/path
+```
+
+My script uses [ntfy](https://ntfy.sh/) to alert me if a backup fails (see my [previous post on ntfy](20220104-notify,_only_smaller.html)).
+
+My full backup is not automated as it requires unlocking my external drive, so I don't do this as often as I would like.  I usually pick a weekend every couple of months when I will get the drive out and back up all my devices.
+
+I try to run a practice restore every few weeks to ensure everything is still in good form and the process becomes habit.  Panicking over lost data is never fun, so at least if I can perform a restore calmly I won't be completely losing my mind.
+
+This system has been working well for me for a long time.  I try not to overcomplicate things, K.I.S.S. as they say.