Timely backups are important. So much so that backing up software is a common topic of discussion, even here on the Fedora Magazine. This article demonstrates how to automate backups with restic using only systemd unit files.
For an introduction to restic, be sure to check out our article Use restic on Fedora for encrypted backups. Then read on for more details.
Two systemd services are required to run in order to automate taking snapshots and keeping data pruned. The first service runs the backup command needs to be run on a regular frequency. The second service takes care of data pruning.
If you’re not familiar with systemd at all, there’s never been a better time to learn. Check out the series on systemd here at the Magazine, starting with this primer on unit files:
If you haven’t installed restic already, note it’s in the official Fedora repositories. To install use this command with sudo:
$ sudo dnf install restic
Backup
First, create the ~/.config/systemd/user/restic-backup.service file. Copy and paste the text below into the file for best results.
[Unit]
Description=Restic backup service
[Service]
Type=oneshot
ExecStart=restic backup --verbose --one-file-system --tag systemd.timer $BACKUP_EXCLUDES $BACKUP_PATHS
ExecStartPost=restic forget --verbose --tag systemd.timer --group-by "paths,tags" --keep-daily $RETENTION_DAYS --keep-weekly $RETENTION_WEEKS --keep-monthly $RETENTION_MONTHS --keep-yearly $RETENTION_YEARS
EnvironmentFile=%h/.config/restic-backup.conf
This service references an environment file in order to load secrets (such as RESTIC_PASSWORD). Create the ~/.config/restic-backup.conf file. Copy and paste the content below for best results. This example uses BackBlaze B2 buckets. Adjust the ID, key, repository, and password values accordingly.
BACKUP_PATHS="/home/rupert"
BACKUP_EXCLUDES="--exclude-file /home/rupert/.restic_excludes --exclude-if-present .exclude_from_backup"
RETENTION_DAYS=7
RETENTION_WEEKS=4
RETENTION_MONTHS=6
RETENTION_YEARS=3
B2_ACCOUNT_ID=XXXXXXXXXXXXXXXXXXXXXXXXX
B2_ACCOUNT_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
RESTIC_REPOSITORY=b2:XXXXXXXXXXXXXXXXXX:/
RESTIC_PASSWORD=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Now that the service is installed, reload systemd: systemctl –user daemon-reload. Try running the service manually to create a backup: systemctl –user start restic-backup.
Because the service is a oneshot, it will run once and exit. After verifying that the service runs and creates snapshots as desired, set up a timer to run this service regularly. For example, to run the restic-backup.service daily, create ~/.config/systemd/user/restic-backup.timer as follows. Again, copy and paste this text:
[Unit]
Description=Backup with restic daily
[Timer]
OnCalendar=daily
Persistent=true
[Install]
WantedBy=timers.target
Enable it by running this command:
$ systemctl --user enable --now restic-backup.timer
Prune
While the main service runs the forget command to only keep snapshots within the keep policy, the data is not actually removed from the restic repository. The prune command inspects the repository and current snapshots, and deletes any data not associated with a snapshot. Because prune can be a time-consuming process, it is not necessary to run every time a backup is run. This is the perfect scenario for a second service and timer. First, create the file ~/.config/systemd/user/restic-prune.service by copying and pasting this text:
[Unit]
Description=Restic backup service (data pruning)
[Service]
Type=oneshot
ExecStart=restic prune
EnvironmentFile=%h/.config/restic-backup.conf
Similarly to the main restic-backup.service, restic-prune is a oneshot service and can be run manually. Once the service has been set up, create and enable a corresponding timer at ~/.config/systemd/user/restic-prune.timer:
[Unit]
Description=Prune data from the restic repository monthly
[Timer]
OnCalendar=monthly
Persistent=true
[Install]
WantedBy=timers.target
That’s it! Restic will now run daily and prune data monthly.
Photo by Samuel Zeller on Unsplash.
Rajat Kumar Singh
when we run this command systemctl –user daemon-reload got this error
“Failed to connect to bus: Operation not permitted”
Stephen
You would likely need to use sudo to do that.
Stephen
Sorry, my bad. You may need two dashes ahead of user so
Link Dupont
Yes, this appears to be a formatting error with the HTML. WordPress renders it as two dashes, but when viewing the post itself, the two dashes appear to be squashed together into an emdash. Since all the unit files are installed as the running user, you must interact with the user systemd session.
Rajat Kumar Singh
tried the same systemctl –user daemon-reload with root access.
[Rajat@www ~]$ su
Password:
[root@www Rajat]# sudo systemctl –user daemon-reload
Failed to connect to bus: No such file or directory
or
[Rajat@www ~]$ sudo systemctl –user daemon-reload
[sudo] password for Rajat:
Failed to connect to bus: No such file or directory
Link Dupont
You do not need to run this as root. The service files should be installed into your user systemd directory: ~/.config/systemd/user. So you should interact with systemd as your normal user (via the –user flag).
[link@localhost ~]$
Rajat Kumar Singh
Thank you it works , somehow for learning what would be the changes in restic-backup.conf file for local repository .
we want to take backups from local machine and store in the same machine in another directory
BACKUP_PATHS=”/home/rupert”
BACKUP_EXCLUDES=”–exclude-file /home/rupert/.restic_excludes –exclude-if-present .exclude_from_backup”
RETENTION_DAYS=7
RETENTION_WEEKS=4
RETENTION_MONTHS=6
RETENTION_YEARS=3
B2_ACCOUNT_ID=XXXXXXXXXXXXXXXXXXXXXXXXX
B2_ACCOUNT_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
RESTIC_REPOSITORY=b2:XXXXXXXXXXXXXXXXXX:/
RESTIC_PASSWORD=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Sebastiaan Franken
The single dash should be two, so
systemctl (dash dash)user daemon-reload
Otherwise use sudo
duba
as root?
Zerocool
Install package for dbus user session
ert
duplicity are better
and normal rsync for other server
Daniel
Better how? Both duplicity and Restic supports backup encryption and backing up to multiple cloud providers as well as local and networked backup destinations.
Restic lets you deduplicate data from multiple systems in a single backup repository. It can give you really great savings on how much storage is required to backup a single and especially multiple systems. The program has excellent snapshot management and you can mount any snapshot to explore previous versions of your files.
Lance Haverkamp
Old School (Duplicity)
In the old school model, data is written sequentially to the storage medium. Once a section of data is recorded, new data is written starting where that section of data ends. It’s not possible to go back and change the data that’s already been written.
New School (Restic)
With the arrival of non-sequential storage medium, such as disk drives, and new ideas such as deduplication, comes the new school approach, which is used by Restic. Data can be written and changed anywhere on the storage medium. This efficiency comes largely through the use of deduplication. Deduplication is a process that eliminates redundant copies of data and reduces storage overhead. Data deduplication techniques ensure that only one unique instance of data is retained on storage media, greatly increasing storage efficiency and flexibility.
Sounce: https://www.backblaze.com/blog/backing-linux-backblaze-b2-duplicity-restic/
Steve
Starting to get feature parity with stuff I wrote 5+ years ago:
https://www.crc.id.au/backups-and-filesystem-deduplication/
John Mesgrenado
Duplicity is also good but restic is much lighter, you should give it a try
Hans
For remote servers rdiff-backup also works nicely.
Daniel
You do not want to run “ExecStartPost=restic forget” after each backup run as suggested here. It can be very time consuming for large backups. Replace the second service file with “restic forget –prune” instead to run both of the time consuming maintenance operations at the same time.
You also don’t want to run “ExecStartPre=restic unlock” before every backup. You could interrupt another operation like the pruning task and potentially causing it to never finish or outright break. The backup repository is usually locked for a reason. You might want to run unlock when you boot your system to clean out any orphaned locks, however.
This all being said: Restic is a really excellent piece of software and this article doesn’t do it justice. You just have to set aside some time to learn how to use it properly.
Link Dupont
Thank you for the suggestions; I will make those adjustments to the sample unit files here in the article. I am still learning restic myself, but I find it a very useful backup tool.
Link Dupont
I have found that restic forget does not take a long time. It inspects the repository and just drops whatever snapshots no longer meet the keep policies. What part of running forget is time consuming?
Sten
I like this, but I don’t like having “restic unlock” in “ExecStartPre” – wouldn’t that risk removing a lock for a backup that was already running?
Similarly, could systemd prevent running a prune while a backup is running? If you ran into a situation where prune happened during a backup, I expect it would fail, then you won’t prune again until the next month.
Brian Exelbierd
Very timely and useful article for me – Thank you!
How long have you been running restic? Have you found any love/hate with it?
Manos Th.
Where is the restore process or what happens if i want only one file from a backup!!!
All of this are in the backup process
Igor Bajo
My questions:
The B2_ACCOUNT_ID and a B2_ACCOUNT_KEY need to be created?
The RESTIC_REPOSITORY could be a second harddrive on the same computer, or a USB harddrive?
I need to set RESTIC_PASSWORD where?
Thanks
Link Dupont
These are environment variables that restic references to determine how and where to back up. I include them in the service via the EnvironmentFile= key, but you need to decide what backup destination works best for your set up. See the docs for details on what environment variables make sense for your use case.
langdon
is there anyway to instruct the systemd timer to only run if the system isn’t busy?
Brian Exelbierd
I’d like to see how to do this too, however, for restic in my use case it may not be needed. Inspired by this article I have been doing some manual testing. After a day of use, the update backup of most of my home directory runs in less than a minute.
Brian Exelbierd
@langdon doing a little reading I think “IOSchedulingClass=idle” in systemd might do the trick. See /usr/lib/systemd/user/systemd-tmpfiles-clean.service — I haven’t had time to experiment yet.
Arshad Mahmood
It is really worthful and all-needed solution for Back up. But it is silent about what to do when this Pruned Backup needs to be restored. Please write a few words about Restoring it.
Thanks
Link Dupont
If you’re unfamiliar with restic, make sure you understand the basics of taking backups and restoring them. Take a look at our article introducing restic to get started.
Fredrik Rambris
I have this setup now. Thanks for the article. I have made the following tweaks:
restic-backup.service runs daily. Does only backup (no forget or prune). I changed the type to simple and have Restart=on-failure and RestartSec=30. This is because I run this job on multiple hosts to the same shared repository. The other hosts connect to central repository over sftp.
restic-backup.timer added RandomizedDelaySec=300 so that two restic instances does not start att the exact same time on different hosts. I was paranoid about the locks.
restic-prune.service runs monthly with forget –prune, grouping by host,tag,path. Runs only on the host with the repository.
rclone is used to sync the local repository to backblaze. Restic can interact directly with the repo at backblaze too, if needed. I prefer to have quick access to backups but also stored offsite in case of disaster.
I have everything deployed with ansible. I did not use the copr repository but rather just put the binary in ansible. Different distros had different versions. The binary is quite portable.
Brian Exelbierd
I like your setup ideas and would like to implement a variation of them for me. Do you have your ansible playbook publicly viewable?
Fredrik Rambris
https://mega.nz/#!1AdkmKBT!1QdGk95XLiQnV9x6tPtRJO382kwS2E9F2slxmtcwZYQ
Joffa Mac
I tried to find this without having to ask here but was unsuccessful – my apologies for asking something that maybe isn’t related.
I have followed the steps above and have it working, however I have 70GB of photos that with my ridiculously slow internet will take a week to backup. Without changing my system sleep/suspend settings, I use the Gnome extension ‘Caffeine’ to do so, prevent my computer suspending etc.
I wanted a better way so after some previous reading about systemd-inhibit, thought I could use it but I can’t find anything on how I make it work with another systemctl function. i.e. I can’t run systemd-inhibit systemctl –user start restic-backup.service (well at least for me nothing happened).
So I ask here for any tips on keeping my computer automatically running whilst these backup functions work, then allowing suspend etc. to kick in as usual.
Thanks in advance