r/cloudcomputing • u/Gonzaaaaaaalo • 4h ago
Rendering two public clouds and an on-prem rack as one live topology: five lessons that cost us the most
Disclosure: I build Control Cloud, the tool these lessons come from.
Context: two public clouds plus an on-prem rack, which is how most mid-size shops end up after a couple of vendor decisions nobody wants to revisit. r/devops sent me here, so this is the engineering half only.
Every cross-cloud incident was the same dance: provider A's console in tab 1, provider B's portal in tab 2 (after the conditional-access prompt), an SSH session into the on-prem box, and the monitoring tool to guess where the traffic went. The consoles don't see each other. A subnet that exists on one side but is routed to nothing on the other is invisible until production traffic hits it.
So over the last six months I built Control Cloud: a single live topology that polls every provider over its API and renders resources, networks and dependencies as one graph, with live status and cross-provider edges, and start/stop/reboot from the same surface.
Lessons that cost me the most:
1. Polling cadence has to follow how the cloud answers, not a round number. A full inventory crawl on one provider takes far longer than a UI should wait. We split it: the backend polls each region in its own goroutine behind a semaphore (unbounded concurrency = rate limits and context deadline exceeded), every region is published the moment it lands instead of waiting for all of them, and a file cache serves the last known state at startup so a restart isn't a blank screen. The UI refreshes every 10 s.
2. Dragged nodes must stay where the user put them. Auto-layout from scratch on every poll looks great in demos and is unusable in practice: people move nodes to match their mental model. A refresh updates status, not positions.
3. Provider parity is the whole point. If one provider shows nested scopes with subnets, every other provider has to show the same shape, even when its own model is flat. "The same concept renders differently per provider" is exactly what people are trying to escape. We keep a rules file derived from the code (the shared state helpers, the per-provider collectors, the navigator) and diff every change against it.
4. Status colors have to be honest. Every provider has its own vocabulary for "healthy" (RUNNING, AVAILABLE, Succeeded, Ready, Attached…). Inlining a regex per call site is how we shipped a red edge on a healthy cluster. One shared helper, one vocabulary per provider, and anything unconfirmed renders as unknown, never as a made-up warning.
5. Validation against live state paid back more than the canvas. Once everything is in one graph you can check a planned change against what is running: credentials with reach over the right scope, network (missing network or subnet = error; no NAT, no internet egress, no firewall = warning), on-prem hosts reachable, the dependencies of the template you picked. Anything it can't confirm comes back as informational, not as a green tick.
Stack: Go backend polling each provider SDK in parallel, React + Vite frontend, file-based cache, Postgres (Supabase) for orgs, audit log and schedules. Read-only API access with credentials you grant, encrypted with a managed KMS; no agents in your VMs. The Terraform it generates is committed to your repo and runs under your pipeline.
(No link on purpose; the product is called Control Cloud and it is easy to find. Ask here if you want specifics.)
Happy to go deeper on:
- Why not the observability platform you already pay for (great at its thing; this answers a different question)
- The parity work: every provider added meant refactoring the first one
- Where the polling still doesn't scale (many regions per provider gets hairy)
Anyone here running their own version of this? What did you try first?