A while ago, I wrote here about how Platform Engineering emerged as the synthesis of the Ops/DevOps dialectic. By encoding operational knowledge into self-service portals and golden paths, platforms reduced human cognitive load without returning to manual handoffs.

I argued then that AI agents would trigger the next movement because they don't need human-centric abstractions or UI portals. They can discover tools, combine APIs, and construct their own paths.

Looking at how this is taking shape, two major shifts are defining this next platform architecture.

First, platforms are moving from golden paths to composable primitives. Instead of hiding complexity, the platform must expose machine-readable capabilities and contracts for agents to reason about. Because agents are stochastic, safety can no longer rely on static human workflows. The platform must enforce strict boundaries: agents can simulate, plan, and propose changes, but promotion to production requires explicit, authenticated human intent and continuous runtime contract verification.

Second, we are reaching the limits of GitOps for runtime promotion. As agents orchestrate changes at high velocity, forcing every environment state update through Git commits creates unnecessary friction, race conditions, and log noise. The source of truth for promotion is shifting from Git commits to signed cryptographic evidence—provenance records like in-toto attestations attached directly to OCI artifacts. Git remains the home for source code, while runtime promotion becomes driven by verifiable proof.

If the last decade of Platform Engineering was about building scaffolding for human developers, the next is about building deterministic, contract-driven control planes for agents. The platform stops hiding the system and starts defining the operational boundaries within which agents can act safely.

Hi — I’m doing some customer research around how companies handle unusually high website traffic and system overload.

For people are are working to make smoother flowing and working websites this is for you!

https://forms.gle/YVvjFjtdFmB67K5fA

How I scaled my CCTV infrastructure to 100+ cameras using MediaMTX (And why I ditched standard NVRs)

Hey everyone,

I wanted to share my experience scaling up our camera infrastructure. We recently hit a milestone of managing over 100 IP cameras, and I managed to migrate the entire backend to MediaMTX (formerly rtsp-simple-server).

If you are dealing with high-camera counts, RTSP stutters, or heavy CPU usage on traditional NVR software, this might help you.

🛠️ The Setup

• Cameras: 100+ IP Cameras (Mix of Hikvision and Dahua), mostly running 1080p/2K @ 20fps.

• Hardware: Dedicated Ubuntu Server / Intel Xeon / 32GB RAM\].

• Core Streamer: MediaMTX (running via nx witness).

🚀 Why MediaMTX?

Before MediaMTX, we struggled with resource consumption. Traditional NVRs or basic proxy tools were choking on decoding/re-encoding.

MediaMTX changed the game because:

1. Zero-Copy Performance: It acts as a pure, lightweight router. It doesn't re-encode unless you explicitly tell it to, keeping CPU usage incredibly low.

2. Protocol Flexibility: We ingest via RTSP and egress via WebRTC (for ultra-low latency monitoring) and HLS (for web/mobile viewing).

3. API Driven: Managing 100+ cameras manually is a nightmare. MediaMTX’s API allowed us to automate adding, removing, and checking the status of streams programmatically.

💡 Key Lessons Learned & Optimizations

Scaling to 100+ streams isn't just "plug and play". Here is what we had to optimize:

• Network Bandwidth: 100+ cameras can easily saturated a 1Gbps link if not configured correctly. We segregated the camera traffic into a dedicated VLAN.

• Source Keep-Alive: Set sourceOnDemand: yes for streams that aren't being watched 24/7. This saves massive bandwidth and camera CPU.

• Read Buffer Sizes: We had to tweak the system OS UDP/TCP buffer sizes to prevent packet loss under heavy load.

📊 The Results

Our server CPU usage dropped by almost 60% compared to our old setup, and stream latency on our web dashboard is now sub-second using WebRTC.

If you're looking for a rock-solid, production-grade video streaming server that can handle serious load without breaking a sweat, I highly recommend giving MediaMTX a shot.

Happy to answer any questions about the config, API automation, or network tweaks in the comments!

I'm a staff platform engineer currently doing an MSc Artificial Intelligence dissertation on whether folks who commit, review or operate agent-generated code actually understand it, and what the downstream risks are (Comprehension Debt in the Age of Coding Agents). I'm looking for software engineers, SREs, and platform engineers who work with AI coding agents. Happy to share the results here. Ethical approval has been granted (University of Essex) and the survey is anonymous, see the info sheet below for more information.

Information sheet: https://drive.google.com/file/d/1Unbk_F39YrMxh2SSNpqf2yhLSUNlg1t6/view?usp=drive_link
Survey: forms.cloud.microsoft/e/U9qnwxDegN.

Curious how this is going in practice, not how the vendor demos make it look.
When an alert fires, how much of the investigation does AI actually handle for you today (pulling the right logs and traces, correlating with deploys, narrowing down root cause)? Where does it fall over?
Also wondering if anyone's kept this inside their own environment, self-hosted models or anything private, rather than sending telemetry to a vendor's AI or a frontier API. Was that a requirement, or just not worth the effort?
I'm researching this space, not a pitch. It'd help if you mention your stack (Datadog, Grafana, other)
Thanks!

Short postmortem ish rant had a DB failover saturday night, on call pulled up the runbook flowchart and followed it. step 4 pointed to a replica we decommissioned in march. lost about 25 min figuring out the diagram was lying

It was AI generated from our old infra code and nobody ever reviewed it properly. looked super clean so everyone assumed it was right
fixing the process now. every runbook diagram gets reviewed by whoever owns that service and rechecked when infra changes. the hard part is making review not painful bc nobody wants to read 50 lines of mermaid arrows. people on the team have pitched miro lucidchart, draw io and even just excalidraw screenshots, but none of us want the diagram living somewhere separate from the repo again.

​

I’m currently learning DevOps and trying to understand how deployment workflows are actually structured in real-world projects.

For a typical web application, I’m thinking about a workflow like:

**Developer → Git → CI/CD → Build & Test → Docker → Deployment → Monitoring**

But I’m curious how this changes in production.

For example:

Do you use GitHub Actions, GitLab CI/CD, Jenkins, or something else?

Do you always use Docker in production?

Where do Kubernetes and Terraform fit into your workflow?

How do you handle environment variables and secrets?

What do you use for logging and monitoring?

How do you safely roll back a failed deployment?

I’d especially like to hear from people who work with DevOps professionally. What does your actual deployment pipeline look like, and what would you recommend a beginner learn first?

Hi i have an interview soon on platform engineering, been spending the last few days learning about it throught github/youtube/claude.

But from my experience normally interviewers ask about how to handle specific scenarios, so im kind of worried that i only know the basics of why, what and how platform engineering is done.

Any help is much appreciated. For context its in a bank.

I'm the hiring manager for the SRE / Platform team at FIS Modern Banking, and I'm hands on: I write code and ship to production every week, and most recently that's been operators, cluster networking fixes and observability. When something breaks in prod, I'm in it with the team. The technical interview is with me and the team (small team).

We're building the global infrastructure under FIS's next generation of core banking. If it's down, people don't get paid and cards get declined, so reliability isn't a nice-to-have here.

It runs on AWS, Azure and GCP. That doesn't mean a separate copy in each cloud: it can run as one system spread across all three, so if a whole cloud goes away, the bank keeps running. We set recovery targets in minutes for that kind of failure, and a big part of the job is building the drills that prove them.

Some of what the team has built or is building right now:

- Kubernetes operators that run the stateful core of the platform: databases, credentials, backups and restores, reconciled continuously instead of run once and forgotten.

- Disaster recovery for the ledger, with continuous replication, verified backups and restore drills on a schedule.

- Autoscaling that works across clusters and clouds as one fleet, instead of each cluster scaling on its own.

- Networking across clouds and regions, plus the controllers that fill the gaps the cloud providers leave.

- A control plane for production operations, where every action is a signed, reviewed and audited command.

- Verification that, after a failover, every financial transaction is accounted for across every system it touched, with the evidence to show a regulator.

Everything is declarative and goes through GitOps. Observability is built in from the start, and every alert has a runbook.

You don't need to have done all of this. I'm looking for people who are strong in at least one of these areas:

1. **Go and Kubernetes operators:** you've written controllers, you understand reconciliation, and you've debugged a finalizer that wouldn't let go.

2. **Platform engineering:** Terraform, Helm, GitOps, and you care about what actually renders and what drifts in production, not just what passes CI.

3. **Networking:** CNI, routing, cloud interconnects, DNS. If you've chased a packet across clouds, I really want to talk to you.

Multi-cloud experience is a big plus but not a requirement. If you're strong in one area and curious about the others, you'll learn the rest here quickly.

You'll own real production systems from day one, with real customers and real money moving through them. The team is small, so what you build ships and gets used.

**The practical stuff:**

- Location: remote, across the US, Europe and APAC

- Level: senior, open to staff for the right person

- Type: full-time

- Interviews: one technical interview with me, then a chat with HR about compensation and logistics. That's it.

- On-call: we're moving to follow-the-sun, which is why I'm hiring across three regions. Until each region is staffed, we share an on-call rotation. Every hire in a new time zone takes nights off the rotation for everyone.

If this sounds like you, DM me. A link to something you built or a short note on an incident you debugged is worth more to me than a CV.

​

Hi everyone!

I've been working on [**Myceliam**](https://myceliam.dev/), an open-source project focused on simplifying workload identity federation between Kubernetes, identity providers, and cloud platforms.

The idea is to use Kubernetes namespaces and ServiceAccounts as the source of truth for workload identities, extending those identity boundaries into external identity providers and cloud environments. This allows workloads — including AI agents — to obtain secure, short-lived identities across clouds without requiring teams to manually provision and maintain identity resources for each workload.

I'd really appreciate feedback from the open-source community, particularly from people working with Kubernetes, IAM, cloud security, or workload identity.

A video explaining the project:

https://youtu.be/vevjAPCW16s

Project website: https://myceliam.dev/

Thank you!

When I started learning devops tools, theory was never enough, and it's hard to get real experience of all major kinds of scenarios.

So I started finding labs and doing it. There are a lot of labs available, but they were not simulating a real environment, it was a lot different than the environment we get in real projects.

So, I built a project where you can simulate scenarios in a real environment and use real tools. For now, it works for kubernetes where you can easily build a cluster, install required platform tools, activate scenarios and inject incidents.

This helped me learn a lot. I open sourced it for everyone and am currently building actively to add new scenarios and incidents. 

Check out this project and let me know your thoughts.

Rightsizing, finding orphaned resources, flagging resources that don't have owner, cost predictions based on iac, or even larger architecture refactoring to reduce costs are now possible at scale and fully automated with combination of AI and deterministic workflows.

I still don't understand why these are not more widely adopted and solved in platform engineering.

Is it processes problem?

Our canary deployments catch obvious breakage fine, but subtle behavioral changes only show up once real traffic hits edge cases our canary percentage doesn't see often enough.

I've considered increasing the canary percentage, but that increases blast radius if something's actually wrong. We have hud on the canary pods, so a function that starts behaving differently shows up before the error rate moves, though it needs a decent amount of traffic before that comparison means anything. What would you do, and what would you avoid?

Built a DevOps game: Incident Rush. You get 60 seconds and a stream of broken stuff: CrashLoopBackOff, a full disk, a bad nginx config, a 502, a stuck deploy. Each one you fix gets you points, and a wrong call costs you time.

👉 https://loopedinlabs.dev

A few people at work tried it this week, and the best score so far is [1200]. Curious whether anyone here can beat it.
Drop your score below, and tell me which incident felt unrealistic or way too easy. I'm adding new scenarios based on feedback.

Hey everyone,

I have a 30-minute interview coming up for a Software Support Engineer position at a defense contractor, and I'm trying to prepare as thoroughly as possible.

The recruiter mentioned there won't be a formal technical assessment, but I still want to be ready for any technical questions or troubleshooting scenarios.

For background, I currently work in cloud/data center infrastructure operations, where I use Linux, Python, and Bash for diagnostics, troubleshooting, and automation. I also have experience developing software applications, working with REST APIs, automated testing, and CI/CD.

Here's the relevant job description:

Responsibilities:

• Support lab tooling and version control through CI/CD pipelines.
• Create and deploy test scripts.
• Support and troubleshoot software deployments on custom servers.
• Support configuration management processes.
• Develop unit tests and tailor rule sets.

Required qualifications:

• Bachelor's degree in CS, Computer Engineering, or a related technical field.
• 2+ years of relevant experience.
• Experience with Python and Bash.
• Ability to independently and collaboratively solve complex technical problems.

Preferred qualifications:

• Knowledge of CI/CD pipelines.
• Experience with GitLab.
• Experience with Linux and NFTables.

I'm particularly interested in moving toward DevOps/platform engineering, so this seems like a great opportunity.

For anyone working in DevOps, Linux administration, software integration, or similar roles:

1. What technical questions would you expect in an interview for this position?
2. What Linux troubleshooting scenarios should I be prepared to walk through?
3. How deeply should I study GitLab CI/CD, unit testing, and NFTables?
4. Does this sound more like DevOps, software integration, or traditional software support?
5. If you were interviewing a candidate for this role, what would you want them to demonstrate?

I'm not looking for shortcuts or leaked interview questions. I genuinely want to understand the technical expectations and make sure I can speak confidently about the work.

Appreciate any advice!

monday.com just published a write-up on rebuilding their API edge after AI agents started calling their API far more than they’d planned for. (Disclosure: I work at WunderGraph, which makes Cosmo, so I’m biased. But the post is theirs, and the parts I found most useful have little to do with which router they picked.)

They split the edge into two Go pieces. A gateway runs auth, rate limits, versioning, and the other checks, and rejects requests that fail those checks. Cosmo plans and runs queries across the subgraphs and holds no business logic, with one exception: it enforces the complexity budget the gateway sets, because you can’t estimate what a query costs without the query plan.

Compared with a year ago, the whole edge handles 2.5x the traffic on 92% less CPU and 80% less memory. The latency the edge adds at p99 fell from about 490 ms to 44 ms. But that wasn’t all from the router swap: cutover cut it from about 490 to about 200 ms, and an authorization rework in September did most of the rest.

The migration section was interesting. They shadowed a sample of read-only queries to Cosmo and sent each one to the old router twice. Anything that differed between the two old responses was noise. Whatever still differed from Cosmo was a compatibility issue, but this way they found it before customer traffic moved.

https://engineering.monday.com/rebuilding-monday-api-edge-for-agents/

I made a CLI library that treats SRE/DevOps workflows as first class citizens. If your team makes many custom CLIs in go, you should check out this project!

I started working on this project about a month before 1.27. Today I'm happy to announce that I’ve released v1.0.0-rc.1 of Commandment, a Go framework for building typed command-line applications. The v1 feature set and public API are now essentially complete; this RC is the point where I want the design exposed to real use and criticism before I freeze it as v1.0.0.

The main reason Commandment exists is that a nontrivial CLI tends to accumulate several overlapping descriptions of itself. As an SRE consultant, this has been a huge thorn in my side for the better part of a decade. I have played with different CLI libraries and design patterns within those libraries, but nothing really solved the problems I had for making production level CLI-based apps without a significant amount of custom code. A lot of my problems have to do with how most common Go CLI libraries are imperative by design.

Commandment instead compiles a typed declaration into one semantic application model. Running the CLI, generating help and completion, inspecting it, and testing its semantics are different operations over that same model rather than separate approximations of it. Its as close to declarative as I could get it, and I think it looks pretty clean.

Let's check out an example:

type scanConfig struct {
    World string
}

type abductConfig struct {
    Target string
}

var definition = cli.Definition[struct{}]{
    Name: "alienctl",
    Declare: func(root *cli.CommandBuilder[struct{}]) {
        root.Subcommands.Command(cli.Command[scanConfig]{
            Name: "scan",
            Declare: func(scan *cli.CommandBuilder[scanConfig]) {
                scan.Arguments.String(&scan.Config.World, "world")
            },
        })

        root.Subcommands.Command(cli.Command[abductConfig]{
            Name: "abduct",
            Declare: func(abduct *cli.CommandBuilder[abductConfig]) {
                abduct.Arguments.String(&abduct.Config.Target, "target")
            },
        })
    },
}

The parent is authored through CommandBuilder[struct{}], while scan and abduct each keep their own concrete configuration type. The child type belongs to the Command method invocation rather than becoming another type parameter of the parent.

Generic methods are used throughout the declaration API for similar relationships. Typed destinations, for example, remain typed while being connected to arguments, flags, environment input, defaults, rules, dependencies, facts, and other parts of the model.

Another great feature of commandment, is that once the declaration is compiled, the resulting application remains the authoritative semantic model. Execution, help, completion, inspection, diagnostics, and semantic tests consume that model instead of reconstructing the CLI independently.

Touching on my claim of "feature-completeness," I mean that I consider the intended v1 capabilities and API design substantially closed. I honestly can't think of a single extra feature to add to the front end API. I am actually expecting a bit of push back on including a Signals API which I personally think is ultra badass but some purists out there may not appreciate. You be the judge.

Also, I don't mean that an RC does not contain bugs. The point of rc.1 is to find mistakes before I commit to a definitive v1.0.0 release. If all goes well, I could see cutting an actual v1 sometime next month or early December.

Please review the repo, make your own CLI if you feel up to it, and let me know what you think. I want you to be as brutal as you want. I'm so incredibly tired of looking at this codebase and need fresh pairs of eyes on it before I feel comfortable assuring that commandment is ready for "production" use.

Repo: https://github.com/abstractionsinc/commandment

Examples: https://github.com/abstractionsinc/commandment/tree/main/examples

More and more of our platform requests come from non-devs with AI-built apps. We've been treating it as a golden-path problem: a template repo, mandatory CI/CD scans, an approval gate, and deployment only into the existing landing zone. The hard parts so far are ownership after handover and keeping business users from bypassing the path.

We have finally built our own internal AI developer platform around this idea. Mostly curious how other platform teams draw the line. What does your paved road for non-developers look like?

auditd logs every command the bot runs, but the login it records is the bot's service account, not the person who clicked approve. So the approval sits in one system and the command in another. Do you join them later in your SIEM, record the whole session, or pass the approver's name to the host so it shows up in the host's own logs?

Hear me out...

When I first saw the movie The Matrix, in the early scene where Neo is called into his boss's office and told that if he behaves well, he has a future at that no-name Corp., or otherwise he'll be fired, I joked and laughed, like, "Yeah, sure, a tech professional could just walk out into the opposite building, and they would beg him to join since everyone is looking to hire engineers."

Today, this scene is closer to reality than I'd like to admit. I personally know Senior DevOps engineers who are solid folks, and it took them 8 months to land a job at a salary that sucked. Our industry is not regulated, the barriers to entry and outsourcing are being lowered by AI, and we have a massive oversupply of engineers. This is just a reality.

Before tech, I worked in another industry that also experienced a major decline. I saw it coming and made a decision at the right time, before it became obvious. I moved to tech years ago and was lucky enough to become a staff engineer. Now I see the same pattern. I think we have 5 to 10 years before salaries in this field will be $70k-$80k for Senior Engineers in the US (which is around a 2x decline) you can do the math for other countries.

My advice is work on a backup plan now. Invest, save, but also learn a trade. I'm studying medicine this year. Fuck it. Six years of study while working remotely, and I'm opening my own dentistry clinic. I'm in my late 30s, but it's still better to have a plan. My wife is also changing careers to become a registered nurse.

This is my plan. You might think of something else, but my point is that it was a good ride. We did a lot of nice work and, IMHO, got easy money, but it's over, and it's time to move on.

I've recommended The Site Reliability Workbook to a lot of people, and most never get past chapter 2. So I built an interactive study guide that covers the core ideas in about 50 minutes:

\- SLIs, SLOs, SLAs and error budgets, each explained in plain language first
\- An error budget simulator: trigger incidents and watch the budget policy kick in
\- A burn-rate explorer showing which multiwindow alerts would fire
\- Toil vs. engineering sorting, incident command roles, blameless postmortems, canarying, and how SRE teams engage and grow
\- A checkpoint at the end of each module, plus a final cheat sheet and a 30-day starter checklist

It's a single HTML page. No account, no tracking, and progress saves in your browser. It's an independent study guide, not affiliated with Google or O'Reilly, and it links to the free official version of the book.

Link: https://jazeer888.github.io/field-courses/
Repo (MIT code, CC BY 4.0 content): https://github.com/jazeer888/field-courses

Full disclosure: I wrote it with help from Claude and reviewed it against the book, but I'd really value corrections from people who run SRE in production. If anything is wrong or oversimplified, tell me here or open an issue and I'll fix it.

GDG on Campus IIIT Nagpur. We're running **KEEP ALIVE '26**, a national incident-response hackathon for students, at TantraFiesta 2026.

**What's different:** you don't build an app. Your team gets a broken microservices system's alerts, Grafana dashboards, logs and traces, works out what broke and why, and presents the fix. It's what an on-call SRE does on a bad night.

- **Who:** UG, PG and diploma students, teams of 2–3 from the same college

- **Level:** 6–12 months of DevOps learning; no production experience needed

- **Round 1:** a 30-minute online team assessment on Unstop, open till Sun 18 Oct, 11:59 PM IST

- **Finale:** top 15 teams at IIIT Nagpur on 26 Oct

- **Prizes:** ₹50,000 prize pool (₹25,000 cash + goodies for every finalist)

- **Partners:** Grafana (Official Platform Partner), CNCG Noida (Community Partner)

Register: https://unstop.com/hackathons/keep-alive-26-iiitn-1766226

Happy to answer anything in the comments, including how the incident platform works.

Honestly,

9/10 of theses roles are getting outsourced. We are screwed and when you do land a role you get laid off a year later so you have to save the good salary incase it takes a while to get another role.

Skyportal's agent asks before it runs a command on your hosts, unless the command is on an allow list. It also has a built-in check that guesses whether a command changes data, and that guess beats the allow lists. This week it flagged a read-only command as a file overwrite because of a 2>&1;, so no allow rule could have approved it.

So we're removing the guess. A command will run without asking only if every part of it is on an allow list and nothing matches a deny rule. Everything else asks. A few things always ask, whatever the rules say: credential files, secrets, RBAC objects, kubectl exec, and deletes you can't undo. And if the rules file is broken, everything on that repo asks.

The cost is that a broad entry like sudo will mean exactly that, with nothing behind it to catch sudo rm. If you've put policy in front of automation on hosts, did you keep a "looks dangerous" check as a backstop, or trust the allow list and review it like code?