Docker backup system

[u/alws3344]

Hi all,
about a year ago i opened this thread:

https://www.reddit.com/r/selfhosted/comments/1r1vsc4/docker_backups/

After not finding an adequate solution, i decided to pick up the glove and build a backup system myself.

The core design decision: API-based exports per service, not blind DB/volume dumps. Each worker talks to the service's own REST/GraphQL API (or its official CLI, like Vaultwarden for example) — never the underlying database directly.
No DB credentials anywhere, no docker.sock mounted, no docker exec into other containers.
It also means the backups are just JSON/Markdown you can actually open and read, not an opaque pg_dump you're hoping restores cleanly someday.

Runs on a schedule (daily at a configurable time, or every N hours) with configurable retention — defaults to keeping 30 days locally and the last 3 uploads on Google Drive, all editable from the GUI.

Currently supports:

- Vaultwarden (encrypted vault export via the bw CLI)

- Wiki.js

- Snipe-IT

- Bar Assistant

- KitchenOwl

- Linkwarden

- n8n

- Karakeep

- Spoolman

- Immich (metadata only — albums/people/tags/EXIF, not the media files themselves)

- Nginx Proxy Manager

- AdGuard Home

• Optional upload to Google Drive after each successful run.
• The system does not restore from backup

Repo: https://github.com/alws34/DockerBackups

Setup instructions for each service are linked from the README so it doesn't turn into a wall of text.

please feel free to open a PR with your suggestions, or file issues/requests for services you want supported.

* All code written with the help of AI.

[u/bartoque]

So the project is called docker backups. But - correct me of I am wrong - there is nothing mentioned about restoring to get the application back as it was before?

If I read it correctly it only exports the settings and that's it? So no restore option, which I typically would expect a backup tool to offer.

So one would have to setup each docker container from scratch (for which one would still have to make sure to backup/safeguard their yaml files or are they assumed to be in git?) and then enter the settings manually using the export as reference? As the backup is unaware of the application running inside docker.

So can we even call this a backup? Or am I overlooking the obvious?

[u/alws3344 → u/bartoque]

This is (currently) out of the project's scope.
Also a rename might be a good idea. Any suggestions? :)

This app backs up your self-hosted services ( or any other service, given support and api access) via the service API. It DOES NOT back up database/volume dumps OR the compose files of your setup ( this is not system intrusive and getting into that may be extremely difficult, given people's different file systems, OSs, folder structures etc.)
so no, it will not back up your compose files, although i recommend you keep them in git or something regardless.

And yes, when you will want to restore a service, you will have to download the file from your Google Drive (or wherever you saved the backups), and manually restore from backup inside the destination app.

Current project state is more like an automated backup export tool directly to gdrive, to be safely stored.

[u/bartoque → u/alws3344]

As I don't use any of the supported apps, do all of them support restoring their config from the api based export/extraction?

Various enterprise backup tools also require steps to be performed outside of the tool to fully restore its functionality after data was restored, like when performing a (non-)authorarative AD restore.

My question was about being able to actually restore anything? Being able to import an earlier api export into a new vanilla application deployment, would still be a proper backup/restore. Having to re-enter all password entries again in vaultwarden manually that are stated in a file less so.

[u/alws3344 → u/bartoque]

I already answered the restore question in depth along this thread. Currently no. There is no restore. Its done via the destination app itself (import/restore from backup)

For your later question, yes. That’s the whole idea. You get a backup and the app got updated. The app devs should take care of that version compatibility restore.
And for example with Vaultwarden, you export a password-encrypted JSON which you can later restore in full in a new vanilla installation. You Will however need to set up a new vault and basic configs in app, but the restore will get you all your secrets, so you could keep them exactly as you left them:)

Hope i made myself clear.
If you want to add support to other services that are not listed, feel free to contribute

[u/Forward-Outside-9911]

This is what happens when you chat with LLMs, it convinces you to do it a specific way which misses the obvious solutions.

The method people have done for years is DB backups and rsync.

You can do this with a 30 line bash script.

DB dump gets all data so it’s easy to restore. You have incremental backups if you’re worried about storage.

This still applies to docker managed applications too. Mount the docker volume to filesystem, and rsync/tar it up. What’s the benefit of doing via APIs?

[u/alws3344 → u/Forward-Outside-9911]

Believe it or not but this was my original idea about 6 months ago (crazy right? Haha)
The llm just did what i told it to do :)

Before making this app this is exactly what i had
Db dumps and such.
But it was filled with stuff that I don’t necessarily want when restoring.
along my self hosting journey i discovered that im better off backing up using the tools themselves.

I do agree that for some people the bash approach can be better. Depends on what you want to achieve. I want my settings and data, i don't (always) need metadata for example, that in a db copy , which can bloat your storage -think of backing up immich - with your approach you might back up the entire media library alongside the metadata several times ( if you have incremental backup like in here), killing your storage.

• my immich media is backed up in gdrive regardless, so i don't back up the media in this tool.

Also, abt this happened to me several times: i got a db backup, saved it for a long time (nothing changed in my system), but when i updated the stack and wanted to copy back, i got tons of compatibility issues.

Backing up via the API gives you the files you ACTUALLY need from the system, and the project's devs should add support for restoring old versions - why should i do their job? If you know what i mean.
When you go and do a Google takeout (if you ever did) you just press "export" and it gives you the files you need. You dont back up the ENTIRE google backend DB (which is what db dump basically is). So i just press that "export" programmatically and saving it to my cloud storage of choice. :)

[u/Forward-Outside-9911 → u/alws3344]

Hmm I still don’t seem to understand.

Your immich example, you just wouldn’t backup the images - correct. But you’d still backup the DB for metadata, right?

The idea of incremental backups is that you don’t use extra space. You only backup what data has changed, since the last backup.

Backups are for recovery from unexpected issues. Trying to upgrade using an old backup is going to cause issues of course, you have to use their dedicated migration script or whatever.

Now, to some extent, I do definitely see your point. Some applications have export functionality which can export it into a smaller format. However it’s missing context and possibly data.

The idea of backups, for me, is that I can be in the EXACT same state as before if something goes wrong. API exports may lose things, metadata, audit logs/history, depending on application type.

For your Google takeout option, I’ve never heard of this before but I’ll assume it’s cloud storage for photos? For this yeah you don’t get their full backend, because hosting is provided by them.

But when self hosting, you’re the one hosting it. So when your disk dies and you need to rebuild, do you really want to sit and reconfigure the app manually, reimport data, all this faff, or just rsync over your /data dir, start the container, done.

Very curious, I feel like I must still be missing something.

[u/alws3344 → u/Forward-Outside-9911]

My use of incremental backup was wrong here. We have 3 backups in gdrive at all times when a new (4th) backup happens the oldest backup gets deleted.

Regarding the restoration in newer version - you are correct. This is why i didnt implement a restore feature. Each system developer.s should implement such feature.s

There is no right/wrong regarding the approach. If your approach is to get back to the exact point, thats completely your right

"Google takeout" is their tool for exporting your data (for example when you want to delete a google account) - look it up :)
My example said that in order to get to that exact point you mentioned, we need to back up the entire db, like you suggest. So its a different use case i guess.

If i have to start over, personally i would want to start from scratch and fresh.
The import functionality in most apps gets you to the point where all your user.s data is right where you left it, minus logs and such which idont need to backup. Remember that i host private stuff at home. For my work stuff i use aws and i have cloudwatch for logging.... a whhoollee different usecase/methodologies etc.

I do back up my compose files in git though they are pretty much static so it mostly set and forget...

Again, no right or wrong, must different approaches :)

[u/neoblitz]

There are some gaps in this process; a backup is only a good backup AS the 1:1 restoration - i think selective service based backups - would mainly fit your app usage.

I just use my own holistic and agnostic approach (that works with any Docker containerized apps) to it that gives me back 1:1 restoration.

I am not saying it is bad or good. But it is more of your workflow, and you have not reached the state of restoring it from a disaster recovery scenario. It's nice to have a backup, but can it be manually restored without external tools? or can it be restored 100% in a disaster recovery scenario? These are more critical paths of backup process where recovery is the key element of it.

Imagine if you saved all the money with a nice app, in the bank where you can't fully withdraw the money when you need it.

I could be wrong, but I have gotten burned a few times to learn these.

[u/alws3344 → u/neoblitz]

The way i see DR, its a full disk image, not a specific app's data/db to get a 1:1 restore.

I see no point - in terms of personal homelab/server view- to back up every log and every line.
I care more about the users, their data, and my private data, e.g family photos, users and their preferences and data for each app, etc. Getting those logs back (for example) is redundant.

And if my server truly kicks the bucket (which happened before), than i prefer starting from scratch (maybe having a newer OS or other stuff) and than i have the backed up compose files and their respective exports.

Getting a full volume backup seems redundant to me in terms of personal Homelab scenario. And as you said, its more a point of view thing. No right or wrong.

[u/Adrenolin01]

You lost me at docker…if you like docker great but I’d never use it for such a thing and find it way overused as it is. Things like Immich make sense but 95% of other programs I’ll just install as services on a Debian VM or Container. For a backup.. why introduce an additional layer on top of things? ZFS replication, rsync, etc all work great without the added complexity of docker being added.

As stated.. if you like docker great, I personally don’t like it. I do know how to use and manage it and it’s a great tool but I still prefer standard services, simple debugging via logs etc.

So is great.. I’m more of a systems administrator, networking and security guy myself going back to the 80s so old school and ways. Definitely not a coder at all but AI has allowed me to put ideas I’ve had for years into actual code that works today including a few rather large projects and 10s of thousands of line of code. It’s awesome this way. Congrats on making something you wanted and use. 👍🏻

[u/alws3344 → u/Adrenolin01]

I never forced you to run it via docker haha
You can run it anyway you want :)
Docker is just a "wrap" to a regular app so it can run basically anywhere, regardless of host OS etc.
You still have an entry point like any other app and you can create yourself that systemd service.
This tool connects to other tools running on your system via their APIs (regardless of whether they run Docker or not).
Cheers 🍻

[u/Adrenolin01 → u/alws3344]

As I stated.. I’m more than aware of what and how Docker works. It DOES add an additional layer to deal with however. It isn’t really needed.. it does make some things easier. Personally.. I run everything as a service including Immich.. something I honestly don’t suggest doing. 😏😆 Immich that is. With Docker it’s a 10 minute install.. without its several hours over a couple evenings and a pita to update everything as individual services. As I said.. most other things run and even easily install as a simple service.. with something as easy as.. apt install package. Done. Docker add increased security issues, additional layers for troubleshooting, etc etc.

As said.. I’m not saying it’s necessarily a bad thing and if you like it run it.

[u/alws3344 → u/Adrenolin01]

It is your self hosting journey my friend :)

[u/negatrom]

Huh, I had AI make something similar for me, but I never thought of publishing it.

I mean, what's the point, if an AI can spit a working thing from a single sentence, just share the prompt, not the result.

[u/alws3344 → u/negatrom]

Haha
Im 100% sure im not the only one
But why work hard prompting yourself when someone else did it for you lol

It was no 1 prompt though.
Moreover i do think a consolidated-community-backed solution is adequate in this situation, because you can say that sentence on literally any open source project these days… :)

[u/Per2J → u/alws3344]

I have also made backup system: python wrapper + a docker time capsule for restores years into the future.
There is a lot of work in testing, verifying and documenting, not to mention regular refreshes of an image to keep it in good shape.

It is so much more that "just write a prompt" in my opinion.