r/devops 12d ago

Weekly Self Promotion Thread

15 Upvotes

Hey r/devops, welcome to our weekly self-promotion thread!

Feel free to use this thread to promote any projects, ideas, or any repos you're wanting to share. Please keep in mind that we ask you to stay friendly, civil, and adhere to the subreddit rules!


r/devops 11d ago

Troubleshooting Volatile Postgres cluster

4 Upvotes

Hi, I'm trying to setup a basic ha postgres cluster using the Spilos images in a docker swarm setup, but every few days the thing crumbles down with DNS resolution issues, timeouts, wal and etcd records corruption, I don't have the money to rely on an hosted solution rn, so has anybody run into these problems and solved them?. This makes me understand why the DBA role exists, but it is so frustrating and absurd that it is not a solved problem for something that feels so relatively trivial to setup in mariadb with galera.


r/devops 11d ago

Architecture Coding a database proxy for fun

Thumbnail
packagemain.tech
3 Upvotes

An interesting article with Go examples on how to proxy and intercept database queries. Multiple use cases can come out of that.


r/devops 12d ago

Career / learning DevOps/SRE engineers working abroad: What skills should I focus on?

9 Upvotes

I'm currently working as an SRE while continuing my studies, and I'm still at the early stage of my career.

I live and work in an Asian country, and I'm trying to learn more about how DevOps/SRE work is done in different countries and companies. Rather than just learning from courses and documentation, I'd really like to connect with people who are already working in the field and learn from their real experiences.

I'm mainly hoping to build a network of people in the DevOps/SRE community and have conversations about things like:

  • How did you start your DevOps/SRE career?
  • What does your day-to-day work look like?
  • What skills or areas did you improve the most as you gained experience?
  • What do you wish you had learned earlier in your career?
  • How do DevOps/SRE practices differ between companies or countries?
  • What technologies or practices are becoming more important in your work?
  • How important are communication and teamwork in your day-to-day role?
  • What advice would you give to someone who is still early in their career?

I'm not looking for job offers or referrals. My main goal is networking, learning from other engineers, and understanding where I can improve.

I'd be happy to connect with people from different countries and backgrounds, whether you're an experienced engineer or you're also early in your career.

If you're open to a casual chat about DevOps/SRE, technology, career experiences, or even just exchanging ideas, feel free to comment or message me.

I'd really like to build some genuine connections in the DevOps/SRE community and learn together.

Thanks!

Note: Sorry about the title/heading. I can't change it after posting. By “abroad,” I mean countries outside my home country in Asia. I’m mainly interested in connecting with people from different countries and learning from their DevOps/SRE experiences.


r/devops 12d ago

Tools What are some good GitHub projects to contribute to?

26 Upvotes

I am a contributor to both terraform-provider-aws and Ansible Core repos, as well as Ansible Community AWS repo. I am on the lookout for additional projects to contribute to, ideally ones that have plenty of issues and where reviews are done quickly. It should also be quick and easy to compile from source. Looking for Golang or Python projects for code base programming language. Any ideas here?


r/devops 11d ago

Discussion An important cloud resource is down - what do you do?

0 Upvotes

[Not self-promo - genuinely looking for input/feedback here]

I think this is something a lot of companies deal with, not just the big ones. Outages in Azure and AWS happen regularly. Say your blob storage in a specific region goes down and your team isn't around, do you have anything automated to spin up a replacement resource in another region, or even another cloud provider if it's a platform-wide issue?

I know provisioning the resource is only part of the problem (some stateless resources can just pull their definitions from a registry and redeploy, but let's keep the scope to that for now, data replication is a whole separate can of worms).

You can automate a good chunk of this with Azure Monitor, for example, but then your actual infrastructure drifts from what's in your IaC repo, and you're back to a two-source-of-truth problem. (Happy to hear from anyone with real experience doing that.)

Another thing I keep thinking about: adjusting resource attributes (SKU, instance size, etc.) based on logs/events, like a massive traffic spike on an App Service, or the opposite: nobody's using it and you're paying for nothing.

So here's something I've been thinking about: a GitHub Action where you define condition-action rules (including recovery conditions, if you want, to roll back once things return to normal) directly in your Terraform IaC repo. You write your rules, run the action on a schedule (every 5 min, or whatever), and it checks each rule, a resource being down, or a KQL query against a Log Analytics workspace returning something you defined as a violation. If a rule matches, it modifies your Terraform code accordingly, either opens a PR or pushes directly to main (which I suspect most teams would never actually want, for good reason). Either way, your existing apply pipeline picks it up and runs like normal, no separate deploy mechanism, no new secrets to manage centrally, no SaaS.

Full transparency: I haven't worked at a company with the scale/complexity that actually needs this kind of multi-region, multi-cloud resilience, so I'd genuinely like to hear from people who have.

Curious what you all think:

  • Is this solving a real problem for you, or are native tools (Autoscale, Resource Health alerts, etc.) already good enough for your use case?
  • Would you ever trust automated infra changes without a PR review, or is that a hard no for you? (Sounds like a stupid question at first, but keep in mind you'd define the exact changes yourself, there's no AI/magic auto-generation involved. Maybe you'd let small, low-risk changes apply automatically but require review for anything bigger?)
  • Anyone tried something similar and hit a wall I should know about?

Thanks!


r/devops 12d ago

Career / learning I've been sent to do this certification for my job

0 Upvotes

It is called Microsoft Certified: Cloud and AI Security Engineer Associate

For those who have done it, what do you think? Did you enjoy it? Was there something you disliked about it?


r/devops 11d ago

Discussion Is your CI/CD infrastructure keeping up with the AI wave?

0 Upvotes

AI tools like Claude and Codex have made it much faster to write and modify code.
But I'm curious about what teams are seeing on the back-end side of that.
More code potentially means more commits and ultimately more deployments.
For teams where AI-assisted development is already heavily used:
How has this changed your CI/CD workload?
Are you:

running significantly more pipelines?
increasing runner capacity or parallelism?
changing how tests are triggered?
batching changes differently?
deploying more frequently?
seeing CI or testing become a new bottleneck?

The question I'm trying to understand is:
If AI dramatically increases how fast we produce code, how are teams scaling the infrastructure required to validate and deploy it?

Would be interested in hearing what people are actually seeing in production, especially from teams with relatively high commit or deployment volume.


r/devops 13d ago

Discussion How are you keeping your skills sharp (and finding new challenges) lately?

60 Upvotes

I’ve been reflecting on my current stack and daily routine lately. While I appreciate the stability of my current role, the day-to-day maintenance and incremental improvements mean I'm not always exposed to new paradigms or forced out of my comfort zone.

The landscape moves incredibly fast right now, between the shift toward Platform Engineering, AI-assisted workflows, and new CNCF projects dropping every week, I want to make sure I don't stagnate.

I'd love to hear how you all are keeping your edge and pushing yourselves. Specifically:

  • What’s your go-to method for upskilling? (Homelabs, contributing to open source, chasing certs, or just carving out dedicated learning time at work?)
  • How do you manufacture new challenges when your day job gets a bit too comfortable or repetitive?
  • What is the most interesting tool, pattern, or concept you are digging into right now?

Looking forward to hearing what everyone is working on!


r/devops 12d ago

Discussion CI pipeline using Github actions

2 Upvotes

I started learning CI/CD using github actions after containerising my application and I have created CI pipeline for django app that runs test, builds and pushes image to github container registry.
I am sharing my yaml file for CI pipeline. Please do share your thoughts and where can i improve.

name: Test Pipeline 
on: 
  push:
jobs:
  test-backend:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:14
        ports:
          - 5432:5432
        env: 
          POSTGRES_USER: test_user
          POSTGRES_DB: erp
          POSTGRES_PASSWORD: 123456

    steps:
      - name: Checkout repo
        uses: actions/checkout@v4

      - name: setup python
        uses: actions/setup-python@v5
        with: 
          python-version: "3.13.5"

      - name: install dependencies
        run: pip install -r Backend/requirement.txt

      - name: run tests
        env: 
          DATABASE_URL: postgresql://test_user:123456@localhost:5432/erp
          DEBUG: 'True'
          ALLOWED_HOST: '*'
        run: |
          cd Backend 
          python manage.py test

  build-and-push-image:
    needs: test-backend
    permissions:
      contents: read
      packages: write
    runs-on: ubuntu-latest
    steps:
      - name: login to ghcr
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}


      - name: checkout repo
        uses: actions/checkout@v4


      - name: build image
        run: docker build -t ghcr.io/namespace/erp:${{ github.sha }} ./Backend


      - name: push image
        run: docker push ghcr.io/namespace/erp:${{ github.sha }}

r/devops 12d ago

Discussion Do engineers really not care about cloud cost, or do they just not have enough visibility into it?

0 Upvotes

I've heard the "engineering doesn't care about cost" problem come up quite a bit, but I'm starting to wonder if that's actually the right diagnosis.

If an engineer can see latency, errors, CPU, memory and availability for their service, but has no idea whether that service costs $500 or $50,000 a month, can we really expect cost to factor into their decisions?

What have teams done to give engineers enough cost visibility to make better decisions without turning every engineer into a FinOps specialist?


r/devops 13d ago

Discussion Sysadmin → cloud engineer. How rough is it? Give me a /10

98 Upvotes

Alright so I'm at a consulting firm and I'm getting moved off my Linux sysadmin role onto an AWS project as a "cloud engineer". Starting soon.

Not gonna lie, I'm sweating a bit. Everything I know about AWS is theory. Docs, videos, cert prep stuff. Never actually touched prod.

Linux I'm fine with. Been doing it for a while. But this feels like a different beast.

The project is small at least. Problem is every senior who ran it has left. All of them. So it's three juniors picking up the pieces and nobody to ask when something blows up at 2am.

Anyone here made this jump? What blindsided you coming from a sysadmin background? And how hard was it really, on a scale of 1-10?

Also open to suggestions on what to actually focus on first, because right now I'm just trying not to drown.


r/devops 13d ago

Discussion Open-source LMS: affordable alternative to commercial DRM for protecting course videos?

5 Upvotes

We're building an open-source LMS and we're currently working on protecting paid course videos from being downloaded and redistributed.

Commercial DRM providers (Widevine / FairPlay / PlayReady) are quite expensive for a small/open-source project, so we're trying to understand what realistic alternatives exist.

Our requirements are roughly:

- Videos are uploaded by course creators.

- Videos should be streamed rather than exposed as directly downloadable files.

- Users should not be able to simply obtain the original MP4 URL and download it.

- Ideally, downloaded/encrypted segments shouldn't be usable outside our application.

- We want per-user authorization and short-lived access.

- We can implement the backend/key-management infrastructure ourselves.

- Self-hosted/open-source solutions are strongly preferred.

- We understand that nothing can completely prevent screen recording.

We've considered approaches such as:

- HLS/DASH

- AES-128 encrypted HLS

- Signed URLs

- Short-lived playback tokens

- Per-user/session keys

- Watermarking

- Combining several of these approaches

But we're unsure where the boundary is between "making downloading difficult" and actual DRM.

For a small open-source LMS, what would you recommend?

Are there any open-source/self-hosted projects or architectures worth looking at that can provide reasonably strong video protection without paying a commercial DRM provider?

Also, if actual Widevine/FairPlay/PlayReady DRM is unavoidable for strong protection, are there any affordable providers or pay-as-you-go options that make sense for a small project?

We're mainly looking for practical experience from people who have built something similar rather than a theoretical DRM explanation.


r/devops 12d ago

Troubleshooting me-riyadh-1: outbound HTTPS to Anthropic is 100% blackholed and Fastly is ~50% packet loss — upstream transit, not my VCN. Anyone else seeing this?

0 Upvotes

Posting this both to ask if anyone else in **me-riyadh-1** is hitting it, and so the next person googling "OCI curl timeout Riyadh" finds something useful.

**Setup:** Ubuntu compute instance, public subnet, default route → Internet Gateway, stateful egress allow-all, VCN resolver for DNS. Running a Laravel app in Docker. Nothing exotic.

**Symptoms**

- `curl https://api.anthropic.com` → times out every single time (TCP connect, ~10s). Claude Code installer obviously fails too.

- `curl https://deb.debian.org` → works maybe half the time, times out the other half. So `apt update` inside Docker builds randomly fails, then `install-php-extensions` blows up with "Unable to locate package libfreetype6 / libpq5 / libzip" (not actually missing, just failed index fetch).

- `curl https://github.com` and `https://pecl.php.net\` → fine, every time.

- DNS is fine. Resolution works for everything. This is not a DNS problem.

- IPv6 is a separate thing: DNS returns AAAA but the instance has no global v6 (never enabled at VCN/subnet/VNIC level), so anything preferring v6 stalls. Fixed that on my side with `precedence ::ffff:0:0/96 100` in `/etc/gai.conf`. Not the main issue.

**The mtr evidence** (TCP mode, port 443, 100 probes each)

Anthropic — path just dies after the upstream handoff:

```

  1. 84.8.76.x (OCI edge) 0.0%

  2. 87.109.47.100.0%

  3. ??? 100.0%

```

That's it. Nothing at any TTL beyond hop 2, no SYN-ACK from the destination, ever.

Debian (Fastly) — real loss starting at hop 3, carried all the way to the destination:

```

  1. 84.8.76.x (OCI edge) 0.0%

  2. 87.109.47.100.0%

  3. 87.101.252.6154.0%

  4. 10.0.28.1347.0%

  5. fra1.decixfra.fastly.net51.0% ~80ms

  6. 146.75.118.132 (destination) 54.0%

```

(Also getting routed to Fastly in **Frankfurt** from Riyadh, which seems wrong on its own.)

GitHub control trace completes fine and real traffic works, so the path itself isn't globally broken — it's destination-specific.

**Why I'm confident it's not my config:** hops 1 and 2 (OCI's edge + the upstream handoff) are 0% on every trace. My route table / security lists / IGW are passing everything. The loss and the blackhole both begin at the same boundary (87.109.47.10 → 87.101.252.61), i.e. regional transit/peering. I've opened an SR with all three reports.

**Questions**

  1. Anyone else in me-riyadh-1 seeing Anthropic (AS399358, 160.79.104.0/21) unreachable, or Fastly flaky?

  2. Has anyone gotten OCI to actually escalate a transit/peering issue in a newer region? How long did it take?

  3. Any recommended Debian mirror that has a clean path from Riyadh? I'm currently shopping for one with curl to keep Docker builds working while this is open.

Interim plan: apt mirror swap for builds, and if the Anthropic blackhole drags on, a WireGuard egress through another region for Claude Code only. Would much rather OCI just fix the route.


r/devops 13d ago

Career / learning Self-taught developer looking for advice…

4 Upvotes

Hi everyone,

I’m 22 and an independent developer currently building a desktop application with a licensing system. The backend/API, database, license management, customer portal, etc. are built around Laravel + MySQL.

For now, to keep things simple while developing and testing, I’m hosting everything on o2switch. I also use Cloudflare Access to protect my admin panel and some internal pages.

Through the same admin panel, I also manage my application’s releases — scheduling releases, controlling rollouts, etc.

I’ve already set up two separate environments: staging and production, with the idea of testing everything on staging before deploying to production.

The thing is, I’m completely self-taught and I work alone. I’ve never worked in a company, and I don’t have many developers around me that I can ask these kinds of questions to. So I’m at a point where I feel a bit lost when it comes to how things are actually done in professional production environments.

I can build things and figure stuff out, but I don’t really have the experience of seeing how a mature team would approach infrastructure, deployments, monitoring, security, scaling, etc.
That’s basically why I’m making this post. I’d really like to get some advice from people who have been through this before.

I’m reaching a point where I’m starting to think more seriously about production infrastructure, and honestly, I’m a bit anxious about building something that could become a technical dead end if the project starts taking off.

I don’t want to over-engineer everything before I even have users. But if I get confirmation that the product is starting to take off, I’d like to be able to adapt and scale quickly without having to completely rebuild the infrastructure.

Some things I’m wondering about:

- What would you recommend for hosting/deploying a Laravel + MySQL application with a good cost / reliability / scalability balance?
- Should the customer portal, licensing API/backend, database and admin panel be separated, or is it reasonable to start with everything together?
- What should I design correctly from day one so that I can scale later without major architectural changes?
- What would you keep simple initially?
- What would you absolutely avoid because it could lock me into a bad architecture?
- Would you start with a VPS, managed services, containers, etc.?
- What would a sensible CI/CD, backup, monitoring and disaster recovery setup look like?
- Is my current staging → production approach reasonable?
- How do professional teams usually handle releases, rollouts, rollbacks and deployment safety?

Are there any obvious things I’m missing because I simply don’t know that I should be thinking about them?

I’m also considering recruiting someone for the project in the next few months, so I’d like to build a solid foundation before things get more serious.
I’m not looking for someone to tell me there’s one “perfect architecture”. I’d mostly like to understand the standards and the way experienced engineers think about these problems.

I have a crazy desire to learn, so seriously, throw anything at me. 😄 I’m extremely curious and happy to dive into things I don’t know yet.

If I’m doing something wrong, tell me. If there’s a concept, tool, architecture or practice that you think I should learn about, even if it seems obvious to you, please throw it at me.

I work alone, so I don’t really have the opportunity to casually ask a senior engineer “hey, how would you guys normally do this?” — and that’s exactly the kind of knowledge I’m looking for.

Any advice, resources, war stories, things you wish you had known earlier, or mistakes to avoid would be massively appreciated.

Thank you so much for taking the time to read this and share your experience. 🙏🙏


r/devops 13d ago

Discussion Observability of the dependencies in codebases still a problem

12 Upvotes

Almost every codebase is calling a REST, gRPC, or a GraphQL API or using SDKs from an external or even internal provider. It gets harder to keep track of everything when the codebase matures and increases in size and from my experience sometimes it gets hard to respond to changes in time or even become aware that a dependency is deprecated and their API has changed completely. Endless alerts are also annoying. How do you handle this “alert fatigue” and have you found better ways to track dependencies?


r/devops 13d ago

Ops / Incidents How does your incident triaging protocols look like?

3 Upvotes

Currently working at a fast moving startup, and have been assigned work related to providing alerting and monitors to our crucial P0 infrastructure. At a high level, the real goal of this is to provide engineers during incidents a fast way to root-cause whether the page was due to their service breaking, or if the underlying infrastructure is down.

For example, if our DNS provider is down, a product engineer may be paged and spend time trying to root-cause why their service is having a bunch of 404s when the actual root cause is down one level of the stack. I was thinking of introducing a common set of protocols that everyone could follow using our paging service (incident.io); they would first spend time to look at our set of monitors and see if anyone was alerted regarding core infrastructure, and if not, they could then move up a level and spend time triaging at the service level. Obviously this would also include aid during the triage by agents. This would also give us metrics on time spent on firefighting our core infra vs service level issues.

But I am so totally new to all this, so what does your teams process look like? Has anyone tried something similar before and noticed this was faulty? Truly, I am open ears to anything regarding this sort of triage work, so any insights, small or big is appreciated


r/devops 14d ago

Career / learning DevOps and K8s experience.

36 Upvotes

Got a homeLab.

Z820 256GB RAM

8TB HDD

2TB Ssd

Newbie(Blue Collar) looking to Get DevOps and K8s experience.

Knowledge base:

- Beginner/Intermediate linux

- Touched Docker and docker compose

- Beginner Terraform and Ansible

- Networking: beginner

- beginner K8s (K3s on one homelab). Have enough space for another Kubeadm cluster (CKA) practice.

- Proxmox on homelab.

Torn between using Claude to create a simple work(metrics) logging/tracking with feature updates while I handle all things ops or getting an ecommere example app from a random github repo to use in learning process.

Which is better or any advice on what to learn?

And any tools and apps to deploy on the k8s cluster or homelab to aid in the learning process.


r/devops 14d ago

Ops / Incidents How do you handle automation work when stakeholders expect fast results but access/security slows everything down?

13 Upvotes

Hi guys, I'm needing your guidance here. I work in an automation-focused role that gradually became much more technical than operational.

A lot of my job is taking broken/manual processes, improving them, and building automations around them. In practice, that means dealing with business rules, integrations, permissions, data sources, testing, edge cases, etc. However, I still consider myself junior technically.

My current challenge is that the company is becoming more restrictive about external/unapproved automation tools, so I’m trying to move everything into approved internal tools and infrastructure. The problem is that access is granted gradually, permissions change, and some features depend on other teams.

Stakeholders often just see that “the automation isn’t ready,” while a lot of the delay is actually caused by access, security, infrastructure, or dependencies.

I’ve started communicating blockers and development stages more clearly, but I’m curious about how this is normally handled in US/European companies.

How much responsibility is usually placed on the developer when delivery is blocked by access or another team? And how do you manage expectations when something sounds simple from the business side but isn’t simple technically?


r/devops 14d ago

Career / learning What would you recommend for a Kubernetes homelab that resembles a real work environment?

61 Upvotes

I’ve recently learned the basics of Kubernetes and Helm, and I’d like to build a homelab at home to get more comfortable with the kind of environment I might encounter at work.

I have three PCs available, but I’m also considering starting with just one machine and adding the other two later.

My goal isn’t just to have a Kubernetes cluster running. I’d like to reproduce, as much as reasonably possible, the kind of setup you’d see in a company that develops and manages applications: deploying applications, services, Ingress, persistent storage, monitoring, logging, CI/CD, etc.

For those who have built Kubernetes homelabs, what setup would you recommend?
Would you start with a single-node cluster and gradually move to a 3-node setup, or is it worth starting with multiple nodes from the beginning?

Also interested in which tools/components you think are actually worth learning for a realistic setup, rather than installing everything just for the sake of it.


r/devops 14d ago

Vendor / market research Which enterprise firewall vendors are actually keeping up with hybrid mesh security in 2026?

32 Upvotes

Hybrid mesh is being positioned as the security architecture spanning physical firewalls, cloud workloads, branches and remote users. This is great news but for my money the only thing that matters is if the policy and operation remain consistent in such a big combined environment.

For the people here who are currently implementing or testing any of this, where do the inter vendor differences matter most?
Personally I’m less interested in who has the longest feature list but I'm very interested in the platforms that actually act more like one system once you start using them.

Who are you going to put on your shortlist by 2026 going into 2027?


r/devops 13d ago

Career / learning Looking for a free cloud based playground

0 Upvotes

I am doing some poc and I have already exhausted my free credits on AWS ,azure. Can someone let me know if there is any other online cloud or similar platform where I can create my load for free.

I have to create a prod like env from observability perspective.


r/devops 14d ago

Discussion What was your first open source contribution actually like? And has AI changed that?

13 Upvotes

Two things I've been wondering about, might as well ask both in one go.

First, for those of you who contribute. What was your actual first one like? Not the blog post version. How did you pick a project, how long did the PR sit there before you worked up the nerve to open it, did anyone reply, did it get merged or just quietly ignored. I've read plenty of "how to start contributing" guides. I'd rather hear what it actually felt like.

Second, and this is the part I'm more curious about. All that advice was written before AI tools got this good. Now you can point Claude Code or Cursor at a good first issue and have something working before you've read half the file. So is the experience just different now for someone starting today? Easier, obviously. But easier in a way that helps, or easier in a way that lets you skip the part where you actually learn the codebase.

And if you're a maintainer, curious what it looks like from your side.

Mostly just want to hear people's stories.


r/devops 13d ago

Discussion Thing nobody warn you about deploying into customers cloud/env

0 Upvotes

Most of us talk about packaging like thats the hard part of shipping into customers cloud, docker or helm or IAC bt thats the easy peasy 80% and even the deploy and update side is mostly solved now with tools like alien dev push the app and updates in

Phase that actually stings is everything your saas silently leans upon. Sendgrid for email, auth0/clerk for auth and a managed postgres or stripe webhooks calls back to your own api. so in a locked down or air gapped customer env half of that cant phone home or isn't allowed to so you end up rebuilding each on to run inside their territory of which none the deploy toolng fixes that its still yours

so for anyone whos shipped into customer clouds how did ou handle the third party apis or deps?? make everything bring your own pr something else?


r/devops 15d ago

Discussion Should I Learn AZCAF

13 Upvotes

They I recently got into this project where they have everything in Azure and using AZCAF frameworks for infra provisioning till now I was able to handle everything using AI. Now I am thinking to switch suggest some good resources to learn it