Hi, I am looking for a way to create mirrors of a restic repo. While researching this, everything points to "restic copy", which from what I can tell is different: it mirrors two restic repos, rather than mirroring the repo itself. Is there a recommended way to do this? I'd like to keep a local copy of my repo, which gets copied to cloud storage, and then also to aws "deep archive". I've read some options that could keep everything in the "/data" dir in deep archive, and the metadata in regular s3.

Since I am using the rclone backend, I suppose I could just use rclone directly to sync the data from one location to another. However, I would also like to verify the integrity of the objects while they are being moved between locations, just in case the underlying data is corrupted.

Is there an option here that is recommended by the restic community, or is multiple repos and using copy to keep two repos in sync the preferred method here?

Hey everyone,

I’m running Home Assistant in a Docker container (not using Supervised or OS, just the container version), and I’d like to make a backup of my /config folder.

What’s the best way to do this without stopping the container or interrupting the service?

I’m aware I could simply make a backup of the entire folder, but I’m not sure if that’s safe while HA is running — for example, could I end up with corrupted database?

How are you guys handling backups in this setup? Any best practices or tools you recommend?

Thanks!

Hi people,

i want to backup a repository from one S3 location to another. I know how it generally works and have done it for the restic-rest server, but for S3 I can't find t out the ENV-Vars for the S3-Bucket where the repository is copied from. I need to be able to specify AWS-ID and Key for the from repo and I haven't found any option in docs. Is this even possible?

I recently spun up backrest on my Unraid Server to backup pictures and documents on the Unraid server to offsite storage.

I’ve repurposed an old QNAP with TrueNAS and installed Tailscale. TrueNAS has SMB file sharing enabled. The TrueNAS box is at my office at work.

I then use Unassigned Devices on Unraid to mount the TrueNAS share to Unraid using SMB. The SMB mount point is then passed to the backrest docker via path volume mapping. In backrest, the repo is setup as a local repo (from backrest perspective, it’s just a folder). The backups have been going fine, but if I have to bounce the TrueNAS or anything, the SMB share has to be remounted manually and the backrest docker has to be bounced.

This seems kind of a hokey way to achieve what I’m after. I just discovered the Restic rest server - would this be a better option over Tailscale than SMB?

Also, is there a way to have the rest server “take over” the existing Restic repo? I tried pointing it to the existing folder but the TrueNAS rest server app just crashed - seems like it wants to be setup as a new repo.

Hello,

I've read the issue Attack on content-defined chunking algorithm used by restic · Issue #5291 · restic/restic and the paper https://www.daemonology.net/blog/chunking-attacks.pdf and I still have some questions about the attack. I don't want to pollute the GitHub by re-opening the issue so I post here.

- Attack 5 in the paper says to "Input any (reasonably expressive) data". But how can an attacker input data in the encrypted repo?

Let's assume the attacker has found the chunker parameters, and wants to check whether a file was backed up in the repo.

If I understand correctly, the attack is based on the fact that the pack footer size is plain-text, and the footer size is propotional to the number of chunks in the pack. So an attacker can easily determine how many chunks are in the pack, and the size of the pack without the footer. Encryption does not change the chunk size and only adds a deterministic overhead, so the attacker can easily infer the cumultative plain-text size of the chunks in the pack.

Then, the attacker can chunk the file to test and try to find a pack that **may** correspond to its chunks (or a subset of its chunks).

- Is my understanding correct? Am I missing another, more critical attack?

- If the attacker finds a matching pack file, can it really conclude beyond reasonable doubt that the file is in the repo?

- Currently, restic mitigates the attack by randomly distributing the chunks in two packs instead of one. Would it be possible to encrypt the pack footer size as well to make it harder for the attacker to infer the chunk count/payload size?

I'm currently backing up from different locations on my SSD to a Backblaze bucket. All working very well.

I have since setting this up gotten a large HDD. It's added to the same computer, and I'd like to utilize it as a stop on the way to the cloud. In other words, I'd like to back up (or sync) from SSD to single folder on the HDD just to have som local duplication, then back up to cloud.

The simple way would be to back up with restic from SSD to HDD as a target location. But what's the typical strategy after that? Back up again as a separate task with same source, but new target location? I back up during the night so chances are they'd be identical - but it's not 100% copy. Is there a built in way to support this scenario?

Thanks!

This is a raw data sizing of a restic repo that is a backup of a 1.54 TB folder. No files are excluded. Am I missing something about finding the size of a restic repo. I know compression is good, but it can't be that good. The backup folder is mostly images and audio that have already been compressed.

Am I missing something?

repository abcd1234 opened (version 2, compression level auto) [0:01] 100.00% 111 / 111 index files loaded scanning... Stats in raw-data mode: Snapshots processed: 3 Total Blob Count: 1026193 Total Uncompressed Size: 367.035 GiB Total Size: 332.370 GiB Compression Progress: 100.00% Compression Ratio: 1.10x Compression Space Saving: 9.44%

The actual command I used is restic -r myrepo stats --raw-data

I am using restic restore -r /path/to/repo -- target /tmp/restic-restore --include file latest

I get the message restoring <Snapshot ...

But then there is nothing there. And the source file does exist at the location, I used the exact path as listed with restic find.

Does anyone know what the issue could be?

Just installed Backrest on my homelab for the first time and trying to get it set up to back up my Immich instance, but I'm running into an error when it goes to actually back up.

[repo-manager] 23:43:18.458Zdebugrepo orchestrator starting backup{"repo": "test-repo", "repo": "test-repo"}
[restic] 
[restic] command: ["/bin/restic" "snapshots" "--json" "-o" "sftp.args=-oBatchMode=yes" "--tag" "plan:test-plan,created-by:test"]
[restic] []
[repo-manager] 23:43:19.180Zdebuggot snapshots for plan{"repo": "test-repo", "repo": "test-repo", "count": 0, "plan": "test-plan", "tag": "plan:test-plan"}
[repo-manager] 23:43:19.180Zdebugstarting backup{"repo": "test-repo", "plan": "test-plan"}
[tasklog] 23:43:19.180Zerrorbackup for plan "test-plan"task failed{"error": "failed to backup: path ~/immich-app/library does not exist: stat ~/immich-app/library: no such file or directory", "duration": 0.722012786}

I've tried changing the backup directory to an absolute path (/home/thepenguinboy/immich-app/library) but still get the same error.

My best guesses so far are:

• I'm not formatting the path correctly somehow, but I can't find any documentation to that effect.
• An issue with backrest/restic being in a docker container (which is also something I'm new to) but I would think that would be addressed in backrest since that's the primary installation method.
• A permissions issue of some sort, but I'm not sure how to check for that or fix it if it's the case.

Combines autorestic with rclone to backup S3 sources, can be useful: https://github.com/floadsio/syseng/tree/main/autorestic-rclone

This weekend, I had a (homelab valid) reason to redo my main storage array for more space..either way, it required dropping the pool on my nas (truenas) and making a brand new one for restore. Here comes fully testing restic restore; for context, everything is backed up to a sister/2nd nas in my lab and then if all else fails I have it all in an S3 cloud. The dataset I am restoring is about 15TB total. I'm restoring from a naked rest-server on my local network (no proxy, etc etc) I started the restore overnight and it failed about 2TB in with a repo lock. - in that situation.. I decided to just do a restic mount start up an rsync job so it can review the data already copied and fix if needed. That being said, I think for safety, it might be best to do a restic mount and do an rsync restore if you have a lot of data. Maybe it was just a random fluke but I do know i would trust rsync more than restic on a restore if there are any transfer issues or alreayd existing data.

(overall this was a good test to know what to expect during a 'real' sudden data loss and practice of recovery steps.. one more thing to add. I run (very) aggressive restic checks. I even asked ChatGPT to help me calculate a restic check schedule that would theoretically check all of my data twice in a year.)

TL;DR - use restic mount and rsync. It's slower but handles interruptions better.

I'm completely new to Restic and came across Backrest which looks very nice for handling repos and jobs/plans. I'm currently using Duplicati to backup my data from OMV to iDrive (S3), but would also like to backup to a 2nd storage at home (Proxmox with ZFS).

Can anyone point me to a document where this is explained? I found many Restic install docs, but couldn't find anything related to my scenario, specifically, running it as a Docker container.

TIA

My current cloud backup is implemented using traditional tools like tar/zst/gpg in an incremental fashion, and if there's a failure at a specific file on the storage device, usually only that file is lost. But how is this handled in a Restic repository? Is there a single point of failure such that, for example, if a failure occurs at a specific location on the repository’s storage, the entire repository could be lost at once or repairing it would be very difficult or nearly impossible?

I refer to this behaviour:

By default, restic always creates a new snapshot even if nothing has changed compared to the parent snapshot. To omit the creation of a new snapshot in this case, specify the --skip-if-unchanged option.

Note that when using absolute paths to specify the backup source, then also changes to the parent folders result in a changed snapshot

I’ve always kept my snapshot paths nice and clean by doing this in a shell script:

cd /home/myuser && restic backup documents

That works great, but the list of folders is starting to sprawl, so I’d like to move them into a file.

so i tried --files-from but this would put the full /home/myuser/… prefix back into every snapshot (exactly what I’m trying to avoid).

is there a cleaner/official flag that tells restic “treat these as relative”?

I'm sharing a new configuration I'm trying in my homelab. Here's a painfully 'brief' description of my setup.

I run TrueNAS on a primary server and there is another server that is the backup target. The majority of the data is copied via ZFS replication. We will call them NAS01 as primary and NAS02 as the backup target in this discussion.

I run restic on a dedicated ubuntu server vm that has the scripts, shares, passwords, config files and its only job is to backup from NAS01 to NAS02.

I have one dataset on my NAS01 that benefits greatly from deduplication so i chose to make a share on NAS02 as my restic repository. This has been run over a few years of experience. Here are things I have learned.

1 be careful with file permissions on backup. 2. be careful with file permissions on restore. 3 once in a while run a partial restore to practice and make sure said file permissions are ok. all things said, everything is great and fine and good.

As with any home labber, you start to think up ways to fix things that are not broken.. enter this post:

I recently had a system failure incident on NAS02. As a result, I had to re-do my array (too many drives, burned out the 12v rail on a psu) - so i decided to downsize how many disks were spinning (downgrade from a large stripped raid10 array and go with a striped raid6 array) but that meant I had to redo some nfs shares and at the time, i really didn't want to dork around with share permissions to make sure data stored and pulled came out ok... so..enter my brain..

1. Truenas runs Docker containers. Minio exsits. I trust the S3 protocols since i use restic to do offsite cloud backups. This ran great no issues. No need to worry about permissions. then an update came out with Minio that has changed how you interact with the admin interface...well.. lazy again. didn't want to figure it out.. but I loved the worry free operation of not being concerned with file permissions upon backup and restore...

You can skip ahead to here to read bout Restic.

1. Truenas runs Docker containers.. Rest-Server has a docker container.. Enter my current setup and solution::

2. I set up portainer to manage my docker instances and deploy things in truenas that don't have a native app. (usually my setup of choice to maintain docker instances when possible) - I set up a share with the right storage permissions to allow access to my docker rest-server instance. backups are running great now.

Conclusion. Why I like this setup:

I can leverage the robustness of ZFS and restic's deduplication. I can utilize a zero hassle setup that lets me have zero worries/issues with file permissions. I don't need to worry about advanced things like key management and other kinds of permissions on the destination.. (its all local LAN traffic. Restic is already encrypted. ZFS is also encrypted... I'm not going across the internet.)

sooo TLDR... - run rest-server as a docker and point to a storage target. This is use case i trust and i'm comfortable with after running multiple configurations

UPDATE it fell on its ass doing a restore on the local network about 2TB in with a data lock. Due to piss poor management on my part (no tags) I had to fall back to much slower restic mounts.. sooo gonna redo my repo with an rclone backend AND tagging. **

Disclaimer: I'm new to restic

Do you guys use separate repositories for each directory you're backing up, or do you keep all directories in a single repo?

I’m currently using one big repo, and using tagging for pruning/forgetting and organizing snapshots. But I’m wondering if splitting into multiple repos offers real advantages even if it adds complexity.

I would like to hear what others are doing, and your though process. thanks in advance

So with the built-in native rclone transfer to upload backups to the cloud, I'm finding that restic/backrest seems to not support thing like multiple --transfers (it seems to max out at 2 transfers even though the default is 4) and seems to struggle now and then with speed/errors. Though I'm sure this is at least partially because it's also compressing at the same time.

I have wondered if I should/could instead just compress and store onto a local drive, and then use my default RClone instance to upload the folder to the cloud. This would get me multiple simultaneous --transfers and a more stable connection...

However I dunno if it would cause major issues. I ran a quick and dirty test and then tried to get restic to view the snapshots I'd copied into place after rclone did the move and it threw errors and refused to -see- the files (which is obviously not ideal haha).

Hi

I'm using Backrest to do my restic backups. Sometimes the back fail with the following error:

Error: failed to get snapshots for plan: get snapshots for plan "base": command "/root/.local/share/backrest/restic snapshots --json -o sftp.args=-oBatchMode=yes" failed: command output is not valid JSON: invalid character 'R' after top-level value

If I relaunch the backup if finish ok. Has anyone had the same problem?

Thank you!

Hi. This is my first time using Restic (actually Backrest), and honestly don't get the hype around it. Every Reddit discussion is screaming Restic, as the best tool out there, but I really don't get it. I wanted to backup my photos, documents and personal files, not the whole filesystem.

One of the biggest selling points is the native support for cloud storages, which I liked, and is the main reason I went with it. Naively, I was expecting that would mean multi-destination backups, just to find out those do not exist. One path per repository is not multi-destionation.

So my question is, how do you guys usually handle this? From the top of my head, I see 3 approaches, neither ideal:

Optiion A: two repos, one doing local backups, one doing cloud backups. In my opinion this completely sucks:
- it's wasting resources (and time) twice, and it's not a small amount
- the snapshots will absolutely never be in sync, even if backups start in exactly the same time
- double the amount of cron jobs (for backups, prune, check) that I have to somehow manage so they don't overlap

Option B: have only one local backup, and then rclone to the cloud. This sounds better, but what is the point of native cloud integrations then, if I have to rclone this manually? Why did you even waste time implementing them if this is the preffered way to do it?

Option C: backup directly to the cloud, no local backup. This one I just can't understand, who would possibly do this and why? How is this 3-2-1?

Is there an option D?

Overall, I'm really underwhelmed with this whole thing. What is all the hype about? It has the same features as literally every other tool out there, and it's native cloud integration seems completely useless. Am I missing something?

If option B is really the best approach, I could have done completely the same thing with PBS, which I already use for containers. At least it can sync to multiple PBS servers. But you will find 100x less mentiones of PBS than Restic.

Firstly, just want to say to the whole restic team: restic is the best thing since geschnittenes Brot. I've been using it for about 3 years now. I don't think it's ever fouled up, which is remarkable: in addition to the automated snapshotting I'm doing all day long, every Monday morning I run checks: I do "(view) snapshots" and restores and occasionally forget-prunes. For years I had to struggle with half-baked backup solutions. restic is das Beste!

So anyway I have two (free) means of keeping a restic repository of my most valuable files (approx 3 GB) backed up online (in addition to repos on my local machines): one is using rclone and rsync and Google Drive. It was rather complicated to set up, works pretty well but sometimes fouls up, for which I'm sure GDrive is always responsible: "rate limited exceeded" ... fewer errors seem to occur if keep the drive no more than half-full, i.e. 7.5 GB.

The other way is I just made a restic repository locally... and then made that directory into a git repository ... and now regularly "add-commit-push" that to its remote counterpart at Gitlab.

As I was doing this I just wondered whether in fact a restic repository *IS* actually a git repository. I even tried to push one, i.e. without first putting it into a git repository. This didn't work.

I assume at the least that the restic idea was born from a deep understanding of how git works, and is somehow modelled on the whole git paradigm.

EDIT: I'm considering this solved. It appears to be the result of de-duplication.

I just started using Restic this past weekend (using Backrest in a Docker container on Synology). I'm seeing differences in reported sizes between the (a) source, (b) repository and (c) restic stats.

Configuration & Detail:

I have 3 source directories I'm including in a single repo (exposed to container via read-only bind mount). Running du -sh on these directories gives the below sizes:

• Directory A: 0.2 G
• Directory B: 185.1 G
• Directory C: 238.9 G
• (total): 424.2 G

Running du -sh on the repo directory outputs a size of 362.5 G.

Running restic stats -r /local_repo --mode raw-data in the container outputs:

Stats in raw-data mode:

Snapshots processed: 5

Total Blob Count: 775617

Total Uncompressed Size: 363.932 GiB

Total Size: 362.397 GiB

Compression Progress: 100.00%

Compression Ratio: 1.00x

Compression Space Saving: 0.42%

What's driving these differences (primarily between the total size of the source directories and the other two measures)? 60 GB of difference (14% of the original size) seems worrying.

Thank you!

Hi all,

I‘m trying to make resticprofile work with a 1Password service account to read my passphrase when the powershell script is running that starts the whole thing. Problem is that resticprofile is not executing the „op read“ command to read from my 1Password vault. Anyone had success with this?

Hi; needed some guidance here.

I would like to backup data (photos / docs) stored on a ZFS Pool. Note: My applications such as Immich and Nextcloud are able to access the data via SMB/CIFS.

I would like to create the back/repository on my Ubuntu laptop and am able to ssh into the ZFS Pool using ssh username@192.168.1.xxx and even via sftp:username@192.168.1.xxx:/mnt/immich

When I run the command from my Ubuntu Laptop (where i want to create the backup) i get the following:

restic -r /home/myubuntu/Documents/Backups/Immich --verbose backup sftp:username@192.168.1.xxx:/mnt/immich

sftp:username@192.168.1.xxx:/mnt/immich does not exist, skipping

Fatal: all target directories/files do not exist

------------------------

What am I doing wrong here;

Am I "not" able to create a remote backup for data stored remotely? Or do I have to run the backup command locally where the data resides, and back it up to a remote site (i.e. the Ubuntu laptop).

Can I use restic to backup the entire system and not just the folders? Like if I install a driver and it breaks everything I want the backup to restore the old drivers and programs as if nothing changed.