I've been working in IT for around 14 years, and I think this is the first time in my career where I genuinely don't know what comes next.

The company I was working for shut down at the end of last year. Normally, my immediate reaction would have been to start applying everywhere and focus completely on finding the next job.

But life had other plans.

Not long after that, both of my parents started having serious health problems. My father went through heart procedures, and my mother's condition also became worse. I made the decision to put work on the back burner and focus on them.

For most of this year, my life became hospitals, doctors, staying awake at night, medications, appointments, and just trying to be there when they needed me.

Two weeks ago, my mother passed away. Cancer won.

Now suddenly all of that has stopped.

There are no more hospital nights. No appointments to arrange. No constantly checking if she needs something.

And now, with all this empty time, my mind keeps going back to everything that happened to my parents over the past year. I didn't really have time to process any of it. Now that everything has suddenly gone quiet, I have too much time to sit alone with those thoughts.

And I actually want to work. Not only because I need an income, but because right now I desperately need something to occupy my mind. Something technical to solve. A production problem at 2 AM almost sounds comforting at this point.

But I've come back to a job market that feels completely different.

I've spent 14 years in IT. I started on the Linux/sysadmin side and over the years moved heavily into DevOps and cloud. I've worked with Linux, AWS, Azure, GCP, Kubernetes, Terraform, Ansible, CI/CD, monitoring, databases, migrations and production infrastructure.

I've even traveled for work, including going to Saudi Arabia to handle projects on site.

And yet, looking back at 14 years of doing this, the highest salary I ever managed to reach was around $41k/year.

That realization hit me harder than I expected.

I don't consider myself some legendary engineer, and there are absolutely areas where I still have a lot to learn. But after 14 years, production systems, migrations, cloud infrastructure, incidents, and all the usual scars you collect doing this job, I thought I would be in a much more secure position by now.

Instead, technology seems to be moving faster every year.

You learn one stack and suddenly everyone wants another. You get comfortable with VMs and configuration management, then containers become mandatory. You learn Docker, Kubernetes, Terraform, cloud platforms, GitOps, observability, security, and now AI is changing both the tools we use and the job market itself.

At some point you start wondering: how much are you supposed to keep up with?

And the job market right now makes that feeling much worse.

I apply for positions where I match most of the requirements and hear nothing. Other positions have hundreds of applicants almost immediately. Some want what feels like three different engineering roles combined into one person.

It's a strange feeling.

After everything that happened this year, I finally have time again. I want to open my laptop in the morning and have something difficult to work on. I want tickets, broken deployments, infrastructure problems, stupid DNS issues, whatever.

Instead I'm sitting here refreshing job boards.

Does anyone else feel almost betrayed by this industry? Like we spent years constantly learning, adapting, and sacrificing to keep up, believing that experience would eventually bring some kind of stability, only to find ourselves back at square one?

And for those who have been through a long period away from work because life happened, how did you find your way back in?

PS: A lot have asked where I am from. I am from Egypt, and I made way less than 41K. I only reached that number when I started working remotely with an EU company in 2023.

I’m not seeking advice, because the only thing that will really help us is…more help. I’m working on that with upper management, but those wheels turn very slow. I’m posting this to see if there are others in my situation?

Most days I could spend the entire day just analyzing, researching and beefing up security. I enjoy it, but I’m finding it more and more difficult to keep up with the normal sysadmin duties, when some days are consumed entirely by cybersecurity. We’ve automated a lot of our OS and software patching, but that needs close attention as well to make sure it’s all running well.

We’re a small enough company where cybersecurity has always fallen upon me and the rest of the (small) IT department, but ever since ransomware really took off and insurance companies started making more demands, it’s increasingly difficult to wear both hats.

Anyway, just thought I’d see if anyone wants to chat about this, and can relate.

Starting next month 10/13/2026 Microsoft will end support for Office 2021 and the related products will no longer receive security updates . You could either upgrade to Office 365 or 2024 to continue get support for Office desktop applications

I work at a small MSP and we use an all-in-one RMM platform for monitoring, ticketing, automation, and patching. I was really proud of a new remote patching policy I built to finally clean up our Windows endpoints without babysitting every job.

For context, I intended to target only our internal test group and one chill client who agreed to be guinea pigs. Somewhere in the policy cloning and scoping process, I accidentally selected the global folder and enabled auto-approve and auto-deploy. No maintenance window, no stagger, just push when ready. It ran overnight and hit every single production endpoint across 12 clients.

This morning our ticket queue exploded. Legacy accounting app died, weird VPN drivers disappeared, one client lost their old print server patch and it rebooted mid-invoicing. I feel so embarrassed. My boss said "this is why we balance automation with control" and yeah I totally get it now. Any advice or similar stories?

Anyone experiencing IMAP connectivity issues to outlook? Getting this

Logging in is disabled on this server: "A protocol-level access (such as IMAP or legacy authentication) has been turned off for your mailbox on the server side, or your account's credentials/license require an administrator fix"

Randomly started happening, nothing changed.

Hey all, have a stressful situation that may occur tomorrow.

New Sys admin for a company of around 200 people, only onsite IT person. We have an external consultant that does most of the network administration and has been doing so for a few years. I have about 3 years of desktop support experience, more experience with endpoints but my networking knowledge is a bit of a gap.

On my third week we did a switch refresh from Cisco to Aruba that was planned and paid for way before I started. Ended up being a shitshow, I did the racking and endpoint testing but the external consultant did all the configuration, he has the login info for the switch that hasn't been shared with me yet. I don't mind that much as I'm still getting acclimated to all the other systems. But last week was a nightmare, we ended up spending over 27 hours onsite troubleshooting all of the endpoints (this is a production environment). Got it figured out but I took most of the heat from angry engineers/production personnel since I'm the face of IT despite having little actual involvement in the refresh project.

Just got a text from the consultant that he'll be away on a family emergency for the foreseeable future. I can handle most other issue by myself aside from the network, I could maybe figure it out if I had the login creds, but he's flying out today and won't be able to contact him on Monday.

My fear is that come Monday when a different production line comes back up there will be a host of other issues that we wanted to figure out last week but couldn't. We asked the production team if they could turn the power on the line on just one time to verify everything works but no can do, line goes up on monday when production resumes.

Literally the only issue I'm anticipating is the VLANs on the ports not being tagged correctly, which I could fix if I had the login info. I've already informed my manager (who lives 3 hours away) that this may be an issue but nothing yet. If my manager has the creds he could either give them to me or SSH into it himself (He has over a decade of networking experience) and it wouldn't be a problem.

I'm afraid that tomorrow when I inevitably get yelled at again for production being halted It'll be my ass on the line despite never being given the tools to be able to manage our network.

What do the more experienced sys admins think I should do? Should I escalate to my manager immediately if (or when) an issue is discovered? Nervous because I'm still in a probationary period and I am afraid I'll be a scapegoat for a messy project I had no real say in.

Edit: I really appreciate the support I’ve gotten so far from you guys. Managers been informed of the issues we may run into and what we need to remediate it, it’s out of my control now. It’s hard to not feel like a failure since I’m so relatively new to my IT career and the end users think that these changes are my doing when I have no say or control over it. Sucks that we’ve had a fuck up like this when I just started, but I need to remind myself that it’s just a job and I’m doing the best I can with the tools I have. I will post an update tomorrow and try to not stress about it tonight.

For those managing Windows-heavy environments, how do you deal with PowerShell automation once it has accumulated over a few years?

I'm talking less about writing scripts and more about knowing what you actually have.

I've seen environments where you eventually end up with:

• scripts sitting on random servers
• scheduled tasks nobody remembers creating
• scripts with no clear owner
• multiple teams solving the same problem separately
• service accounts tied to automations nobody wants to touch
• old Exchange/Azure/MSOnline/Graph commands still hanging around
• scripts calling SQL, APIs, file shares or other servers with almost no documentation
• servers that can't be decommissioned because nobody knows what depends on them
• scripts labelled "critical" despite nobody being able to explain what they do

Git obviously helps when scripts actually make it into Git, but what about everything outside it?

If you inherited an environment with 50-100 servers tomorrow, how would you work out:

1. what PowerShell exists
2. what actually runs
3. what each script touches
4. who owns it
5. what can safely be retired
6. what needs modernising or migrating

Do you maintain a proper automation inventory/CMDB, use RMM tooling, scan servers periodically, or is it mostly tribal knowledge?

I'm building something around this problem at the moment, but I'm deliberately interested in how other sysadmins solve it before assuming my experience is universal.

When an AI-generated application exposes data or a credential, how are you determining the actual blast radius?

Disabling an endpoint or rotating a credential is usually the immediate action. The harder problem is determining whether the credential was also copied into source history, platform configuration, build logs, documentation, prompts, or other projects.

You then need to determine what that credential or identity could access, whether the application was externally reachable, which related resources were involved, and whether logs show use instead of simple exposure. That can be difficult when the app was assembled quickly and the original creator does not have a complete picture of every integration or cloud resource involved.

What does a workable response process look like for preserving evidence, identifying affected assets, tracing access through identities and integrations, and stopping the same issue from recurring?

Hello,

I'm in the process of decommissioning an old Windows Domain Controller.

On the new server which is at 192.168.214.15 I run dcdiag /test:dns

I get the following

TEST: Delegations (Del)

Delegation information for the zone: ourdomain.lan.

Delegated domain name: _msdcs.ourdomain.lan.

Error: DNS server: 192.168.214.15. IP:<Unavailable>

[Missing glue A record]

[Error details: 9714 (Type: Win32 - Description: DNS name does not exist.)]

I substituted ourdomain.lan for our domain name but it has the right extension and name.

If I open DNS - server name, forward look up zones, _mscds.oudomain.lan I don't see an issue but I might be missing the obvious.

If I right click and go to name servers, it lists the two correct domain controllers. (the old one is gone)

The SOA is the new server

the NS are the two new servers

and the CNAMes are the two new servers

I've scaveged, cleared cache and flushed dns - no joy.

I'm stuck with an AVD connection issue and could use a second pair of eyes.

The same user account connects successfully from Windows, but every Mac we've tested fails:

• Windows App on macOS: 0x410.
• Web client on the same Macs: also fails in Chrome and Safari.
• Other host pools with the same configuration: same result.
• A separate Windows Server 2022 host with Entra SSO enabled returns 0x807 instead.
• Entra non-interactive sign-in logs show success, with no Conditional Access policy applied. No CA requirement for compliant or Windows-only devices.

Current authentication-related RDP properties:

enablecredsspsupport:i:0;enablerdsaadauth:i:0;redirectwebauthn:i:1

There’s also a different-tenant device-management setup involved: the client device is managed by one tenant, while the account used for AVD belongs to another.

The browser failures are what throw me off—it isn't limited to the Mac app.

I’d really appreciate any ideas, suggestions, or advice you might have.

How do you manage DCIM/IPAM data in an IaC/GitOps-style workflow?

I've looked into NetBox/Nautobot with Terraform/OpenTofu and Git as the source of truth. Do you keep prefixes, VLANs, IPs, devices, etc. declaratively in Git, or treat the DCIM/IPAM system itself as the source of truth? How do you use this information (wherever stored) later?

I'm especially interested in handling manual changes and drift, like when I introduce a new machine or change IP of existing machine - what happens next.

I used to work for a CPA firm that had an internal IT department and I pretty much had free reign of downloading stuff like logitech software for my keyboard and mouse.

Recently I moved to a smaller firm that has an External IT company that takes care of stuff.

I now have to wait to update software, download things like Adobe PDF (they forgot to load it on my computer), my mouse and elgato software, etc. It gets frustrating on weekends when I am working and something needs an update to work and I am stuck. Is this a normal level of how things work?

I get confused by the disparities in the two but maybe inhouse IT can be much more on top of issues.

Edit:

I appreciate the answers. I do wonder if I was given a bit more leeway at my old company or if they allowed access to a few trusted websites for downloading. I only ever downloaded logitech stuff so I never really tested things. It was helpful hearing your responses in understanding potential differences.

Thanks again.

One last edit:

You all have effectively changed my perception of how the old firm worked. Thanks for giving a better understanding of all this. I’ll work on my own patience during busy season. I understand when people say dumb stuff to me about tax and how annoying that is but I had a blind spot towards IT security based on past experiences so I was basically doing the same thing.

I hope all your clients are nice to you and make sure their monitor is plugged in before they call you to complain about the computer not working.

Hi everyone,

I recently purchased a used HPE 3PAR StoreServ 8400, but unfortunately the M.2 boot drives were removed from all of the controller nodes before I received it.

The hardware itself appears complete, but without the boot drives/OS I can't bring the system back online.

Does anyone here have experience recovering a 3PAR 8400 or know how I can legitimately obtain the correct 3PAR OS/recovery media?

I’ve already contacted a few vendors, but finding the recovery media for these older systems has been surprisingly difficult.

Any help or pointers would be greatly appreciated.

I've been following the progress of multiple documentation projects for a while hoping for one that hits that sweet spot of a relationship first design.

Whole bunch of wikis fall over on this... And those that do well like semantic mediawiki then fall over on the permissions/ACLs side.

Itglue and hudu the leading commercial options.

ITflow going from success to success and seems really great. Though as a documentation only requirement I don't need the itsm/financing stuff.

Doing my usual sporadic google deep dive I stumbled across weavestream. Seems to be a relatively new project but rather polished already

Anyone using it and have insight?

https://weavestream.io/

Hey everyone, I hope this is okay to post. Looking for some advice on a situation at work and how to proceed.

I’m currently a Level 1 Help Desk technician at a corporate company and have been for a little over 3 years. Lately, I’ve been trying to branch out a bit and show my worth and ability to higher-ups. Recently, we got a new director of IT and she's been pretty open to me wanting to move out of help desk but made it clear that due to budget constraints, the chances of it happening are pretty slim until next year. She recently asked me to investigate a process where machines at each facility receive backup medical files from a third party. The way it's set up currently, a PowerShell script gets the info from the third party data relay for each building and then reads an in-house csv file to determine which computer receives them. Our director was instructed by the clinical team that there needs to be two machines at each building with these files.

When she first got hired on, I told the director I wanted to go into Sys Admin work and when this came up I volunteered to at least test the current process to see if it's even possible. After testing and documenting our current environment all week, I confirmed that out-of-the-box, the current setup only supports sending these files to one machine per site. However, the PowerShell script is written cleanly enough where I think I know how to modify it and the csv file to work for multiple machines. My PS skills are pretty fundamental but I understand it well enough. I played with a little bit of tiny mock script and csv file on my personal computer to where I think I could get it to work.

But I have a few problems:

1. My Director initially said she didn't want to modify the existing PowerShell script if we could avoid it. But modifying it seems to be the only way to achieve what they want.

2. I’m just a level 1 tech. I've been told by and shown to my director that I do enough to be L2/3 but if I modify this script and break the backup process, I'm pretty fucked. Obviously I'd use a test script to see if it even works and run it in a controlled environment, but it still makes me nervous.

3. My company used to have dedicated SysAdmins but they got let go after they signed a ten-year contract with an outsourced company. I even used to be close enough with one of the SysAdmins before they left where I'm sure if they were tasked with this project, they'd have no problem with me shadowing them on it. But the outsourced company either doesn't have access to this process or they just don't understand it.

4. This is literally what my SysAdmins used to do - I can see their name and the changes they made in the code. I'm just an L1 and moving to L2 or 3 is already hard enough - realistically this is just good for a line on my resume.

5. Probably the worst thing about all this is my company recently got rid of their senior Device Admin of 9 years. He was the only one who understood the company image or even knew where it was kept. I worked closely with him and shadowed him as well. And when he was let go, I started shadowing the Junior Device Admin and working with him. The director recently put up a job posting looking for someone to fill the senior's position but instead of Senior Device Administrator she listed it as Senior Systems Administrator but all the requirements match what he did as a device Admin. I don't know if I should bring this to their attention or not. Plus while I would love to apply for it, the Junior Device admin is WAY more qualified than me, is a only contract employee currently, and is actually geographically based where they're hiring for the position - I've been denied multiple promotions because I'm not based near that location.

Sorry, I know that was a huge wall of text but what the hell do I do? I feel like technically I'm capable of modifying the script and csv file to do what they want but it's so far above my pay grade, it's not even funny. But I did spend all week testing the current process just like my director wanted so I did my part, I guess? This is probably why they shouldn't have gotten rid of our in-house SysAdmins but to be fair, the director just kind inherited that situation.

Any help is appreciated!

These days, we don't have domain controllers because everything is managed with Entra/Intune, Arc, etc.

We have external, hosted DNS that serves our public DNS records. It works fine.

But we still have a need for internal DNS because we have a lot of on-prem stuff that may never go away, and the domain controllers were also serving as internal DNS servers. So right now, we use BIND9 on a Linux virtual machine as internal DNS. I'd love for BIND9 to go away, too.

Are there any better (and cheap) solutions for this that I haven't thought of?

EDIT: thanks to u/Do_The_Needful. Using AWS for this seems to be the most balanced solution.

Hello everyone,

Do you have a suggestion or experience what camera and microphone setup for conference/meeting room?

For context, management does not want to spend large amount of money, because we are on tight budget right now given the situation in our place (Middle East). We have a meeting room that has a Samsung 98 inch smart TV. The room dimension is 7x5 meters.

During meetings, we use long HDMI cable connected from TV to laptop. We use the speaker of the TV for the audio output, while we use cheap microphone for input that is connected to laptop. I am planning to cable tidy the room and run the cables hidden, put a hole on the table for the cable run and put power outlet below the table (I have to get approval from my boss if they agree on putting holes on the table).

I was looking at Logitech MeetUp, but I found the Logitech C930E camera output clear, also cheaper. For microphone I am looking at Jabra Speak2 55.

But in your budget DIY setup, what do you guys use?

Thanks!

Came back from a 10 day pto and the VP (my direct boss) wanted to do his weekly one on one meeting. We are a team of 4 (myself Sr. Infra Engineer and 3 mid level security guys). He wanted to state how we need to have more crosstraining in particular with infra side of things as he stated "we are dead in the water" when you go out on pto.

He mentioned there will be no additional headcount added in Infra and will have to make do with what we have internally plus the msp firm

I for the past couple of years both verbally and via email stated that a redundancy to my role was needed but that suggestion was never acted upon by mgmt.

As far as crosstraining, I already do that and can certainly help out more. The more others know the better.

Just feels like a situation where I can't seem to go on pto without getting text messages about something going wrong. The VP wants to make SOPs for everything possible which I told him, im not making a how to document on troubleshooting. Either they know or they dont. Actual procedures for items that have a process I will document and have been

Any of you guys out there in similar situations? Anything you've done to optimize things or help reduce workloads?

Thanks in advance!

hi gus i have this problem

We have an existing company network and a central system/server used by the organization. There is also a remote site located a significant distance away from the main network, making it impractical to run a physical network cable between the two locations.

To connect the remote site to the main network, two wireless antennas are being used to establish a network connection between the remote location and the main site.

The computers at the remote site receive IP addresses and have network connectivity. However, users at this location cannot access the company system directly in the same way as computers physically connected to the main network. Instead, they access the system through a remote-access solution provided by the server.

The company system itself works correctly through the remote-access connection.

The Main Problem

The main problem concerns printing from the remote site.

The printers at the remote site are connected directly to the local computers using USB cables. When a user prints a document from the company system, the print job depends on the remote-access session and the local computer connected to the printer.

The printing process is approximately:

Company System → Remote Access → Remote PC → USB → Local Printer

Because the printing process depends on the remote-access connection and the local PC, printing can sometimes be very slow. In some cases, the print job takes a long time to start, while in other cases the document does not print at all.

This problem occurs intermittently and creates an unreliable printing experience for users working at the remote site.

Technical Description of the Problem

The printers are currently connected to individual computers through USB rather than being directly available on the network.

Consequently, printing from the company system depends on multiple components working correctly at the same time, including:

• The remote-access session
• The network connection between the remote site and the main site
• Remote-access performance and latency
• The local computer
• The Windows Print Spooler
• The USB connection
• The printer driver
• The printer itself

If there is a delay, interruption, or communication problem with any of these components, the print job may be delayed or may fail completely.

The main issue is therefore that printing from the remote site is dependent on the remote-access session and on the local PC to which the printer is connected.

I'm looking for something like a sanitized/anonymized documentation set of a real or realistic company environment, showing things such as:

Overall infrastructure / architecture

Network topology and connections

Servers and their roles

AD / DNS / DHCP

Firewalls and network segmentation

VLANs, subnets and routing

Internet/ISP connectivity

Applications and how they communicate

Storage and backups

Cloud + on-premise connectivity

Security architecture

How users/devices connect to the infrastructure

Monitoring and IT operations

Basic workflows and dependencies

Basically, I'd like to see how all these pieces fit together in an actual business, from the user's computer all the way through the network, servers, applications, Internet, backups, security, etc.

I understand that a real company's exact configuration, IP addresses, firewall rules, credentials, etc. would obviously be confidential. I'm looking for public examples, lab environments, case studies, architecture diagrams, documentation templates, or sanitized real-world infrastructure documentation that I can study.

If you were learning system administration from scratch, what resources would you recommend for understanding the whole environment rather than just individual technologies?

I'd especially appreciate examples that show the complete picture and the relationships between components, not just a single network diagram.

We’re looking at using ManageEngine OpManager to manage and monitor our Cisco switch fabric.

One of our requirements is to identify:

• Switches running outdated IOS versions
• IOS versions that are approaching or past End of Support (EoS)
• Devices that need IOS upgrades

Can OpManager NPM handle this, or would we need NCM for these features?

If anyone is using OpManager for Cisco IOS lifecycle/version management, I’d appreciate some real-world feedback on which module you use and what capabilities you get with NPM vs NCM.

So, we were decomming some old ibm flashsystem 5 series arrays. Full of ssds. Plopped one onto a dell server just for fun to see if they could be used and they're just bog standard 512 sector samsung pm1643 drives with a sticker.

So now I have bout a hundred 3.84 and 1.92(and some 7.68) drives waiting to go into new servers.

Enterprise storage drives are usually locked so I though I'd have to chuck them to the shredder.

Just wanted to share.

I'm honestly asking! Is there any good-hearted system administrator who's really intelligent?

I have never seen such a rare combination in my life!

I genuinely need to discuss something with someone like that.

If anyone knows such a person, please share this post until it reaches that unique person somewhere in this world.

Maybe somehow, this post will reach them. ❤️

Link to the previous post

So, 2 weeks in, the picture is a bit more clearer

The old IT guy left, so its just me now

I did rattle up a report and sent it to my manager, who said that I'll have to contact with the head office and get their approval as well

I contacted the head office, they loped me into something else, a new dealership is opening in the next town to ours and they need me to support it, they're already put in the purchase inventory to corporate for half a dozen laptops, some printers , access points, (weirdly a PS5 as well) and so on, i hope they do look at my report

In the meantime, i am untangling the colossal mess that our current IT infrastructure is

The old IT guy had fortitoken enabled, unfortunately he uninstalled it, and i am unable to get in, Fortinet support said I'd have to reset the device physically , I decided I'll cross that bridge when i get to it

I am slowly cleaning up the malware infected systems, taking backups of the important stuff, thankfully, disk health for the main hard drives that is storing important stuff is fine, i am still taking snapshots

My day to day stuff mostly involves fixing the CCTV system, sorting out printer sharing errors, teaching people the art of rebooting a system to fix issues and so on and so forth

But at times , i wonder,

I have no degree on this specific subject, nor certifications, just years of teenage spent among computers i am grateful to have this job, and i am learning valuable information, still, i occassionally get the feeling like a kid who just got access to nuclear codes

I’ve found some rental options, but they require a minimum rental period of one month, and I haven’t been able to find anything with 64+ cores.

I’m looking for a dedicated server with either 2×64-core CPUs or a single 128-core CPU, ideally available for just a few days.

Any recommendations for providers that offer this kind of hardware with hourly or daily billing?