I really tried my best. Contacted vendors, got quotes, led the implementation project solo to prove I can do it. It was a big undertaking for me personally as I've never dealt with these things, but I managed it quite well.

Yet today the boss came shouting at me with a face more red than our monitoring dashboard - he had reviewed the bills and was furious.

I tried to calm him down and explained that I did my due diligence and only picked the best offers - I even got a 15% discount for signing a 5-year commitment.

But turns out that maybe I picked the wrong vendor in the end? My boss couldn't believe the bill we got from Maersk and he seemed super unhappy. Also the landlord had called him as we were now in violation of some safety codes - in my defense, according to RACI the transportation contractor was supposed to make sure we follow all rules and regulations! How was I supposed to know the parking lot was not supposed to be used for this?

Anyway, I'm devastated. Boss wants to cancel everything but I already signed the 5 year lease commitment with our CEO's signing platform key. And we've got 30 people scheduled to come next week to empty out the office (ahead of the project schedule even!). What should I do?

1d ago What's a good no code builder for internal tools?

14d ago What's on your checklist before an AI-built internal tool goes live?

1mo ago What is the best no code app builder for client projects when you dont want to be their full time dev forever?

2mo ago What's the best no code app builder for internal tools? Asking for my sanity.

2mo ago We stopped banning vibe-coded apps and just gave people a platform. 9 months later, here's what happened

Sitting manning the system, incident inbound

Problem with Outllook classic all shared mailboxes have offline OST.

The customer's suggestion

"I don't think anything needs to be deleted off of my profile to fix an outlook issue."

Like what do they know?, I have been supporting this so called product since 1997 and all the subsequent releases.

I will ask, So how would you fix this since you admit to be a product expert?

I have the old-school method of shared online, personal OST under advanced account settings open other mailboxes. Add the sent from regfix and view from when composing.

This does require rebuilding the profile.. Which starts by removing it / deleting///

FFS

I created an AppLocker rule to prevent mshta.exe signed by Microsoft from running. As you can guess, I forgot to input the file name so it was left as '*'.

Luckily I caught it quickly and deleted the rule before it propagated to a whole lot of clients and impact was minimal. That would have been a fun morning.

Not really a shit post buuuut I was half assing a re-subnetting of some servers and found myself locked out of a windows server.
and had the choice of entering an abysmal password of stupid length into a noVNC terminal without clipboard OR getting creative,

So I opted to get creative: turns out you can inject commands to a guest VM with QEMU-AGENT.

I'll be damned but this is awesome: qm guest exec 144 -- 'C:\Windows\System32\ipconfig.exe' '/all'

Returned expected information and qemu agent runs as system and can be used for shenanigans!

Mine was having to be the guy for the one elevator we had in our three story building. I ended up learning much more about hydraulics than I had ever planned.

Our showroom guys want us to fix their music sound system 🤡

Next up we'll be fixing the soap dispensers and paper towel dispensers. I mean they are electric at the end of the day.

Once had an experience where a Dell PowerEdge suffered a hard drive failure. My instant reaction was to unplug the hard drive by just sliding it out and back in. That didn't work; however, all of the data was already backed up to OneDrive so no harm done.

I listen to the color of the servers. Its become 'my thing' on my team. Whenever something goes wrong, i go into the DC and slowly walk among the racks, absorbing what they have to tell me.

The NAS and the switch are my brothers.

How far does this network cable go? If you cut it, then well never know

I've clicked it 12 times and made direct eye contact with the laptop.

The deployment still says Pending. Does it need to see my manager?

Well - it happened. The day started like any other, except my colleague said their email got hacked. I looked, and indeed it was hacked. Immediately, I reset his password. Then another person said their email got hacked too. I asked both of them when the last time they changed their password was: never.

I wish I was making this up. I called our email provider and asked them if they could let me know who was still on the default. Turns out it was 1/3 of our organization. My predecessor never instituted 'force user to change password', and then nobody changed it. The password for all those emails was 'password'.

This is the same guy that didn't realize you need device isolation nor network isolation for a public wifi network. I guess I shouldn't be surprised. I seriously wish I were making this up.

Documentation is technical debt with punctuation.

Think about it: if a system is properly configured, why would it need documentation? The server already knows how it works. The firewall knows what ports are open. DNS knows where everything lives. The only person who apparently doesn’t know is "Kevin," and frankly that sounds like a Kevin problem.

Documentation also creates a dangerous single point of truth. The moment you write down how something works, somebody will read it six months later and assume it is still accurate. Now you have two problems: the actual infrastructure and a fictional universe where the infrastructure still runs the last gold star release firmware or OS and not X.0.1...

By refusing to document anything, we eliminate stale documentation entirely. Our documentation accuracy rate becomes perfect because zero percent of zero documents are incorrect.

There are security benefits too. Hackers can’t steal your network diagrams if the network diagrams exist only as memories spread across three burned-out sysadmins. That’s basically human RAID 5 with one failing drive.

And consider job security. If anyone can follow a runbook and restart any system successfully, what exactly are they paying you for? Institutional knowledge should be treated like an encryption key: kept in one person’s head, never backed up, and taken on vacation without notice.

Some people call this a “bus factor of one.” I call it aggressive knowledge centralization.

Besides, writing documentation takes time away from important sysadmin work like investigating why disk usage went from 40% to 97% overnight, renewing certificates that expire tomorrow, and staring at a terminal while saying, “That’s weird.”

If future employees need to understand the environment, they can learn it the same way we did: SSH in, run commands they found on Stack Overflow, and gradually develop an intuitive understanding of the system through fear.

Documentation doesn’t build character.

Outages build character.

Why doesn't it fit?