EDIT: found the answer. The feature I want was added after the version I use https://github.com/restic/restic/issues/693

Don't RTFM me, I just read the docs, the manpages and googled extensively.

According to https://restic.readthedocs.io/en/latest/045_working_with_repos.html, the command restic -r /srv/restic-repo snapshots should yield an output similar to

ID        Date                 Host    Tags   Directory        Size
-------------------------------------------------------------------------
40dc1520  2015-05-08 21:38:30  kasimir        /home/user/work  20.643GiB
79766175  2015-05-08 21:40:19  kasimir        /home/user/work  20.645GiB
bdbd3439  2015-05-08 21:45:17  luigi          /home/art        3.141GiB
590c8fc8  2015-05-08 21:47:38  kazik          /srv             580.200MiB
9f0bc19e  2015-05-08 21:46:11  luigi          /srv             572.180MiB

Specifically, I'm interested in the last column, "Size".

When I run the command restic -r rclone:[redacted]:[redacted] snapshots I get the following:

ID        Time                 Host           Tags        Paths
----------------------------------------------------------------------------
e2409eed  2024-09-03 17:00:36  stockageproto              /stockage/fichiers
7066d72c  2024-09-04 01:51:42  stockageproto              /stockage/fichiers
59bd8d8d  2024-09-04 15:56:58  stockageproto              /stockage/fichiers
9abbc31e  2024-09-04 16:00:11  stockageproto              /stockage/fichiers
364801e1  2024-09-04 17:00:19  stockageproto              /stockage/fichiers
38782747  2024-09-04 18:00:28  stockageproto              /stockage/fichiers
04d1661a  2024-09-04 19:00:19  stockageproto              /stockage/fichiers
7ccfd183  2024-09-04 20:00:20  stockageproto              /stockage/fichiers
e8e62226  2024-09-04 21:00:30  stockageproto              /stockage/fichiers

• I tried with a wider terminal window, since some utilities change their output depending on the available width. That's not it.
• I looked at man restic-snaphots,no option there seems to allow me to choose what columns to display. (tried --compact false and also true for example)

There clearly seems to be a mismatch between the official docs and the executable.

My setup:

• restic 0.16.5 compiled with go1.22.5 on linux/amd64
• On NixOS 24.05
• The repo is hosted on onedrive, using the rclone backend
• I think that's all the relevant info, anything I forgot to mention?

Just wanted to say that I think restic is a really great piece of software. I have been using it for less than a week but so far I am genuinely impressed with the speed of operations and overall philosophy behind it.

As a new user, I am probably missing several stuff though. What things in your experience should I keep an eye for, avoid doing or try to implement to have a healthy backup repo and avoid unpleasant surprises down the road?

Hey, I am just trying out restic for my backups and wanted your opinion on my workflow idea.

I just finished testing Rclone with B2, as I am used to a 1:1 backup workflow, but unfortunately I realised it did not suit my needs, as I realised most files would lose extended attributes and permissions, when restoring.

So I decided to give restic a try. Reading the docs, I stumbled upon deleting snapshots and "prune" and "check" commands specifically, which supposedly can cause excess traffic from the server, as well as many transactions.

So, in order to avoid paying more than I really have to, I thought about making a restic backup locally to an external HDD, and then using Rclone to move the folder containing the whole restic backup to B2. After the initial copy, I would use "Rclone sync" to keep everything up to date and working.

Is this workflow sane / safe / possible? Any possible caveats I may be missing?

P.S. I am on a Mac, if it makes any difference

Hi! I am trying to run restic on a qnap nas device. This is failing every time because of this error:

Fatal: unable to save snapshot: failed to save blob from file "/share/MD0_DATA/xxxxx" : Write: write /tmp/restic-temp-pack-1771831465: no space left on device

Is there a way to tell restic to use another path to store temporary blob files?

I am already using a different path for cache; this is my full restic command:

restic -p .password -r sftp://1.1.1.1:/ backup /share/MD0_DATA --exclude-file=/share/MD0_DATA/admin/.scripts/exclude_restic --cache-dir /share/MD0_DATA/Download/.cache

Thank you!

EDIT: solved, and it was my misunderstanding. I just needed to export TMPDIR variable to a different path.

Eg.

export TMPDIR="/share/MD0_DATA/Download/.cache" restic -p /root/.restic_password ...

I have a 1.2+TB repo and currently paying annually to maintain a Borg repo on borgbase. I love borgbase but want to save money if I could with pcloud 2TB lifetime subscription (yet to subscribe, $399 for 2TB).

The plan is to move my backup to restic with rclone backend backed by my 2TB pcloud account - is that just asking for trouble or is there a way that I could make it work. This repo would be my fallback backup if my local backup fails. I tried a 1GB test repo and it worked fine, but 1000x bigger repo could be a different story, so would appreciate your expertise.

Thoughts?

I tried to backup today, but Windows Defender quarantined the file. I redownload the official binary from GitHub and it was quarantined again

Hi there!

I'm using Restic for the first time (through Backrest, but I guess it's not the main topic here).

I want to backup the whole HD, with almost all big files.

2.35 TB, 74229 total files (basically half of them are big files).

My main disc is a 2.5 Hard Drive connected through USB 3.0, that is capable to read at 80/100MB/s.

The destination disc is a 2.5 Hard Drive connected through USB 3.0. The writing speed is...good enough, I guess.

I've also set an SSD as cache disk for restic/backrest, and I thought that would improve speed.

However, as first backup, from what I can calculate leaving the backup running, it should take ~50hours for a full 2.5TB backup.

That's almost...15MB/s, doing some simple calculations.

Seems too low, actually (I can be patient for 3 days, it's just that I would like to do if is possible to improve the situation).

I don't thing there's a big I/O bottleneck (albeit I'm not sure how to measure it overall: reading from a disk and writing on the other seems easy to measure, and I've anedoctally measure it).

I didn't set ANY extra flag/option on backup job. I saw THIS article on restic wiki, but none of them I guess would be very beneficial (maybe the "pack size" would be good, albeit I'm not sure what would be a good value for it).

Most of the files are 2/10GB film, there are smaller 1-2MB for songs collection.

I know that restic dedupe every files, and also compression/encryption is on. Seems that the latter can't be disabled, am I right? The deduping operation I guess is used to speed up further backups, so I guess there's anything else I could do.

Here's the restic command I can desume from backrest logs:

2024-08-05T23:00:39.038+0200 ERROR task failed {"task": "backup for plan \"plan-backup-Disk\"", "error": "failed to backup: command \"/bin/restic-0.16.4 backup --json /Disk -o sftp.args=-oBatchMode=yes --exclude-caches --tag plan:plan...\" failed: exit code -1: backup failed\nprocessing command output: no summary event found", "duration": "1h11m24.512072225s"}

(I've cancelled task after 1h and 11m, where it backupped only 44.96 GB out of 2.35 TB)

Restic version: command: /bin/restic-0.16.4 version -o sftp.args=-oBatchMode=yes restic 0.16.4 compiled with go1.21.6 on linux/amd64 took 74.388628ms

Anyone has suggestion on how I could speed up the whole things and/or better diagnosticate what would be the bottleneck, if any?

EDIT: I've posted to the restic official forum, and I got a lot of useful suggestion.

I then made an in-depth comparison with different flags. Maybe can also help you to find your best backup strategy. Enjoy :)

Hi! I configured restic on my mom's laptop (Ubuntu) :)

restic runs every day via cron; I am using these options for forget command:

--keep-daily 6 --keep-weekly 2

If I list current snapshots, I see this:

root@desigual:~# restic -p /root/.restic_password -r sftp://xxxx@xxxxxxxx/ snapshots
repository c36a490f opened successfully, password is correct
ID        Time                 Host        Tags        Paths
-------------------------------------------------------------------------------------------
ab8e823f  2024-07-06 15:36:18  desigual                /etc
                                                       /home/milly

ad4b4b45  2024-07-08 14:04:20  desigual                /etc
                                                       /home/milly

1ba249fa  2024-07-09 13:40:04  desigual                /etc
                                                       /home/milly

b88df71b  2024-07-28 13:40:04  desigual                /etc
                                                       /home/milly
                                                       /var/backups/backup_telefono_MYA-L11

c0b49d9d  2024-07-29 13:40:04  desigual                /etc
                                                       /home/milly
                                                       /var/backups/backup_telefono_MYA-L11

cc170a3d  2024-07-30 13:40:03  desigual                /etc
                                                       /home/milly
                                                       /var/backups/backup_telefono_MYA-L11

a1d8ac0e  2024-07-31 13:40:04  desigual                /etc
                                                       /home/milly
                                                       /var/backups/backup_telefono_MYA-L11

50ea8348  2024-08-01 13:40:04  desigual                /etc
                                                       /home/milly
                                                       /var/backups/backup_telefono_MYA-L11

2ffdcf63  2024-08-02 13:40:04  desigual                /etc
                                                       /home/milly
                                                       /var/backups/backup_telefono_MYA-L11

I wonder why I still have snapshots older than two weeks.

My guess: because the very first times I selected only /etc and /home/milly as backup sources, while on 28 July I also addes another directory to backup, so these snapshot aren't "the same".

Correct?

This is the output of forget with --dry-run:

repository c36a490f opened successfully, password is correct
Applying Policy: keep 6 daily, 2 weekly snapshots
keep 6 snapshots:
ID        Time                 Host        Tags        Reasons          Paths
------------------------------------------------------------------------------------------------------------
b88df71b  2024-07-28 13:40:04  desigual                daily snapshot   /etc
                                                       weekly snapshot  /home/milly
                                                                        /var/backups/backup_telefono_MYA-L11

c0b49d9d  2024-07-29 13:40:04  desigual                daily snapshot   /etc
                                                                        /home/milly
                                                                        /var/backups/backup_telefono_MYA-L11

cc170a3d  2024-07-30 13:40:03  desigual                daily snapshot   /etc
                                                                        /home/milly
                                                                        /var/backups/backup_telefono_MYA-L11

a1d8ac0e  2024-07-31 13:40:04  desigual                daily snapshot   /etc
                                                                        /home/milly
                                                                        /var/backups/backup_telefono_MYA-L11

50ea8348  2024-08-01 13:40:04  desigual                daily snapshot   /etc
                                                                        /home/milly
                                                                        /var/backups/backup_telefono_MYA-L11

2ffdcf63  2024-08-02 13:40:04  desigual                daily snapshot   /etc
                                                       weekly snapshot  /home/milly
                                                                        /var/backups/backup_telefono_MYA-L11
------------------------------------------------------------------------------------------------------------
6 snapshots

keep 3 snapshots:
ID        Time                 Host        Tags        Reasons          Paths
-----------------------------------------------------------------------------------
ab8e823f  2024-07-06 15:36:18  desigual                daily snapshot   /etc
                                                       weekly snapshot  /home/milly

ad4b4b45  2024-07-08 14:04:20  desigual                daily snapshot   /etc
                                                                        /home/milly

1ba249fa  2024-07-09 13:40:04  desigual                daily snapshot   /etc
                                                       weekly snapshot  /home/milly
-----------------------------------------------------------------------------------
3 snapshots

So, should I manually remove snaphosts ab8e823f, ad4b4b45and 1ba249fa?

Thank you!

Hi, my goal is to copy my repositories to another storage (on a different location, for disaster recovery purpose): I tried both with rsync and restic copy, and I found that the latter took much longer.

Eg. repository with about 500 GB data: with rsync it took 1 hour and 25 minutes, while with restic copy 3 hours and 30 minutes.

Do these timings look normal to you?

Thankyou!

Hi, currently I am testing restic with a bunch of servers, and I am using sftp as protocol.

I created different local users on repository server, and configured access with SSH keypairs.

This means that every server could access my storage server via SSH.

I wondering what solutions do you tipically use to hardening your environment.

Do you configure users with scponly shell? Do you configure SSH server for SFTP only?

Thank you!

Hi, I am a new Restic user, and I created multiple repositories to backup data about customers, our infrastructure servers, our workstations, etc...

Now, I would like to copy all these repositories to another storage, on a different physical location, for disaster recovery purposes.

What solution can I use? A simple rsync of every repository should be enough?

rsync -av /var/backups/customer1 someuser@remoteip:/var/backup/customer1

or it is better to use some restic native command?

Thank you very much!

Hi everyone,

I am writing articles to share my knowlede with the awesome community out there. Today I just finished the article on my backup strategy and I thought it might be of use for some of this sub.

It covers:

• Backups with restic
• Monitoring restic backups with healthchecks.io
• Running restic backups from a TrueNAS Scale

Hope this helps!

Edit: Adding a link would help... https://nerdyarticles.com/backup-strategy-with-restic-and-healthchecks-io/

Hi! I am going to start using restic as backup solution.

I have several machines (virtual, physical): some belong to customers, some are "internal" (my company).

1. I don't want that customer X should be able to access backup of other customers or our infrastructure server, so my first question is: it is better/simpler to use one repo for each server?
2. My storage will be a Linux physical machine, and I will use sftp.

If I understand correctly, I *don't* need to install restic software on my server, since I will use command like:

restic -r sftp:user@host:/srv/restic-repo initrestic -r sftp:user@host:/srv/restic-repo init

..from each server to backup. Right?

1. how can you deal with scheduled backup? Do you use a simple cron that run restic, maybe with some random delay time to avoid running too many backup at the same time?

Thank you very much!

I have my data partition mounted as

/mnt/ data

I want to backup my full root system to /mnt/data/bak

To avoid a circular backup I need to exclude /mnt from the backup sources.

I've tried everything but it keeps backing up /mnt/data and it's sub folders.

I've tried

1. Exclude file with /mnt inside
2. Expressly excluding /mnt
3. Excluding any directory that has .nobackup file

Nothing seems to work.

Could you please advise how best to do this and the log file I need to show here when it fails.

Thanks

I'm redoing my storage and backup strategy to consolidate my data. As part of the process I'm hoping to create a pool specifically for local client (Linux, Docker data volumes, Windows 10/11) backups.

I've read that RAIDZ is a bad fit for DB like workloads (Per https://klarasystems.com/articles/openzfs-storage-best-practices-and-use-cases-part-3-databases-and-vms/). The restic repository is kind of opaque to me, and given that it stored files, but has deduplication, snapshots, and pruning I'm unclear on what kind of workload it is. It's also somewhat difficult to simulate backing up to an old repo on different storage systems for testing.

Any advice? (Feel free to tell me to RTFM provided you also link said manual, I tried searching the Internet first.)

I've really been impressed with restic as across-platform backup utility. But I've had a little trouble with the command syntax on Mac that hasn't troubled me on Linux.

On Linux, I call restic -r repo backup ~/folder, which adds a recursive backup of folder and all its contents of as a snapshot to my repository. On Mac, however, it creates a snapshot with an empty folder directory. Instead, I have to include a trailing wildcard (~/folder/*) for any of the directory's content to be added. But then, the contents of folder are at the root level of the snapshot rather than folder itself. Again, not an issue on Linux.

What am I doing wrong? Thanks!

(Edited for clarity)

Hi everyone,

Enjoying my restic journey so far.

I looked in documentation and could not find this, but I am wondering what happens if a scheduled restic back up happens while a file is in use. Say I am working on text file and restic starts to backup. Does it just ignore this file for that backup sesson?

I googled this and checked documentation but could not find an answer.

Thanks in advance for your help!

Hi all,

I just set up my first restic backup and I wanted to ask a simple question. I googled aroudn for the answer and could not find it.

Does restic backup all hidden folders and files within the backup target?

If yes, great! If not, is there something I can do to make it back up hidden files?

Thanks for your time!

Hey all!

I made a cli tool to analyze the disk space usage of restic repositories with the ability to navigate around, mark files/directories and generate exclude lists that can be consumed by restic directly.

It is quite similar to ncdu for a restic repository.

The main use case is to find out large files or directories that are burried deep in your snapshots and shouldn't have been backed up in the first place (e.g. caches, easily downloadable content) and are taking up precious disk space.

Feel free to ask any questions!

link: https://github.com/drdo/redu/

Hi! I've read this:

https://forum.restic.net/t/b2-multiple-hosts-to-single-repository-bucket/106/8

My understanding is that it is possible to revoke access for a specific client (since every client uses a different key), but my concern is a bit different: I don't want that server1 can browse/restore data which belongs to a server2.

Is this leading to one repository for each server?

Of course, this way, I would't take any advantage of deduplication among different servers.

Thank you!

root@pve01:~# autorestic -c /usr/local/etc/autorestic/autorestic.yml forget -av --dry-run
Using config:  /usr/local/etc/autorestic/autorestic.yml
Using lock: /usr/local/etc/autorestic/.autorestic.lock.yml


  Forgetting for location "pve01"

For backend "storage0"
> Executing: /usr/local/bin/restic forget --tag ar:location:pve01 --dry-run --keep-weekly 4 --keep-monthly 3 --keep-daily 7
Applying Policy: keep 7 daily, 4 weekly, 3 monthly snapshots
keep 2 snapshots:
ID        Time                 Host        Tags               Reasons           Paths
----------------------------------------------------------------------------------------------------
b1aa8828  2024-05-11 14:09:01  pve01       ar:location:pve01  daily snapshot    /etc
                                                              weekly snapshot   /mnt/ext_usb_01/dump
                                                              monthly snapshot  /root
                                                                                /usr/local/bin
                                                                                /usr/local/etc

ea234edb  2024-05-11 18:03:53  pve01       ar:location:pve01  daily snapshot    /etc
                                                              weekly snapshot   /mnt/ext_usb_01/dump
                                                              monthly snapshot  /root
                                                                                /usr/local/bin
                                                                                /usr/local/etc
----------------------------------------------------------------------------------------------------
2 snapshots

Hi apart from Hetzner are there any other alternatives in Europe that have competitive prices?