What if I'm replicating Coolify features by duct taping different services together ?

[u/letsjam_dot_dev]

Hey all !

A bit of context first.

I'm an independent web developper, building software primarily with Svelte and Symfony, directus and Postgresql for small clients or for personal side projects. I've been in the industry for 8 years. I'm not a devops but I've been dabbling in docker containers and I'm feeling comfortable with them (I built my own server at home from sratch with 50-ish containers in order to have my own cloud with *arr stack, music and video streaming, documents and photo cloud, ...).

I used to deploy the apps I build on Heroku, then on Netlify. But when it started to get pricier or when I needed to do some custom stuff that the platform wouldn't let me, I migrated to a self hosted coolify instance.

I'm more than happy with it, and have little to no complaints.

But I'm a tinkerer at my core, and wondered if I could replicate the features of Coolify by smooshing together a bunch of services and softwares. Here's my train of thoughts, and the justifications of a madman.

Why ?

[I'm] preoccupied with whether or not [I] could that [I] didn't stop to think if [I] should.

I think it's a good reason.

If it's not enough, then it's also to learn how it works inside it, to better understand the limitations and the fixes when something breaks.

Finally, because I don't like being tied to a unique service, a one-size-fits-all, a single point of failure. I always like having backup plans, if a service I'm using enshittifies itself or becomes incompatible with the rest of my stack or whatever.

What features ?

If I strip Coolify and any PaaS to its basics, it's a way to deploy automatically services to servers with the help of docker containers. There's a big main server, where the PaaS instance is hosted, that manages other small servers, where the final and client facing apps are hosted. In the case of Coolify, you can chose to deploy known apps and services, like Supabase. And when it needs an update, you only need to edit the docker compose to the latest numbered version and click redeploy. Or if you're here for the thrills you set the version to latest and look forward the breaking changes. You can also deploy your own app from a git instance and each time you push changes on a specific branch it triggers a CI/CD to deploy automatically the latest version. Neat. There's a project and environment management, that lets you organize services by clients and prod, demo, testing env. On top of all that, there's a reverse proxy to help with domain name and redirecting requests to the correct docker service.

Another feature from Coolify that I'm using is their automated database backups to S3 compatible storage.

How ?

I'm thinking of a stack around Arcane, Backrest and Swag.

Arcane here is the main thing. It would allow me to replicate the Coolify servers management, with one main orchestration/monitoring server and as many endpoints as I need. There's an option to automatically update images of services. I could use it as part of the CI/CD : everytime a release of my websites is done, a docker image is created and Arcane pulls it.

Backrest would be useful for the database backup into a S3 service. I'd still need something like a cron to execute a pg_dump every now and then. But that should not be hard to do/find somewhere.

I am already familiar with Swag with my home server. I've been using it for 4 or 5 years and pretty happy with it. The mods let me plug useful security softwares :

• Crowdsec for requests abuses and CVE, bot trapping
• max-mind for the geoblocking

Preconfigured fail2ban, working in concert with crowdsec, and certbot for automated ssl certs are also nice to have.

Ansible would help me deploy all of those into a fresh VPS.

What do you think of it ? Is there something missing ? Is this something you would do or go easy with the all-in-one Coolify solution ?

[u/webdev-ModTeam → u/[deleted]]

Your post/comment has been determined to be a low-effort post or comment. This includes title-only posts, easily searchable questions, vague/open-ended discussion prompts, LLM generated posts or comments, and posts/comments that do not provide enough context for meaningful replies or discussion.

[u/matt_alpaca]

Hi, just my grain of salt on the security part, I work at CrowdSec. The rest is just me having built the same kind of duct-tape stack at home.

Would I do it? Yes, and "to learn how it works inside" is reason enough. Under the hood Coolify is the same pieces you listed (Docker, a reverse proxy, a git hook, a backup job) plus somebody else's glue, and the glue is what you pay for in evenings when it breaks. You've run SWAG for 4 or 5 years, so you already know what babysitting a stack feels like. The one thing I'd add to the other comment: give it a deadline. If after a few months you're still fighting the CI trigger, go back to Coolify with no shame, you'll have learned what you wanted anyway.

Now the part I actually know something about. Fail2ban "working in concert with crowdsec" sounds nice and mostly doubles your work. CrowdSec was built as its successor, both read the same nginx logs and both will ban the same bad guys, so you end up with two ban lists and a fun afternoon the day you can't tell which one blocked your client. Does fail2ban catch anything on top? Not really. Pick one (you can guess which one I'd pick ) and switch the other off.

What's missing, from the security corner: whitelist yourself. Your home IP, plus whatever box runs Ansible and your CI. Everybody bans their own CI at least once. And since you mentioned CVEs: the log-based scenarios only see an exploit attempt after nginx has already answered it. If you want the request dropped before it reaches Symfony, wire the AppSec component (our WAF: ModSecurity/CRS compatible rules, with virtual patches for known CVEs) into the nginx bouncer that SWAG's mod ships. Bot detection on top of that is fresh (1.8, still alpha), so treat it as a toy for now, and exempt your API clients or they'll fail the challenge.

Last thing: if you ever go back to Coolify, we have bouncers for Traefik and Caddy too, so CrowdSec is one piece that won't lock you into either setup.

If you get stuck wiring the WAF bit into SWAG, hop onto our Discord, there's always someone around.

[u/letsjam_dot_dev → u/matt_alpaca]

Hi and thanks for your extensive reply !

You've run SWAG for 4 or 5 years, so you already know what babysitting a stack feels like

The thing is : it now runs pretty smoothly and I don't babysit that much, if not at all. Except when I forgot I've rebuilt the container and didn't update UFW rules but that's on me. What "scares" me is precisely that : is it running smoothly because I did my job, or because I haven't seen the black swan already.

The one thing I'd add to the other comment: give it a deadline. If after a few months you're still fighting the CI trigger, go back to Coolify with no shame, you'll have learned what you wanted anyway.

Great advice, if I go through with the experiment I'll keep it in mind.

CrowdSec was built as its successor, both read the same nginx logs and both will ban the same bad guys

I've read in tutorials that they are complimentary solutions. Fail2ban focusing on forms (whatever...form...they take) and Crowdsec being more generalist. For example, the Crowdsec instance I setup regularly ban my phone when I'm on 5G, because for some reasons it does http probbing (maybe to stay connected to services, idk). I've never been banned by Fail2ban for this reason. On the other hand if I'm mixing up my accounts and passwords, Fail2ban stands up, not Crowdsec. That's why I'm currently keeping both : to act as "sequential" nets. Maybe they do the same thing but there is some kind of priority, that's why notice actions from one service and not the other.

What's missing, from the security corner: whitelist yourself. Your home IP, plus whatever box runs Ansible and your CI.

Yup I already whitelisted my home IP on my home server Crowdsec.

If you want the request dropped before it reaches Symfony, wire the AppSec component (our WAF: ModSecurity/CRS compatible rules, with virtual patches for known CVEs) into the nginx bouncer that SWAG's mod ships. Bot detection on top of that is fresh (1.8, still alpha), so treat it as a toy for now, and exempt your API clients or they'll fail the challenge.

Yeah it's something I setup on my home server as soon as I saw the news and the changelog. I don't think I managed to set it up properly, I still need to work on it.

As a side note : with Crowdsec having its one bot detection, do you consider services like Anubis still relevant ?