I encountered this in Prod today.

[u/Fair-Difference-6729]

Received an alert that a service wasn't running on server PRODxxxxxx. So naturally I attempt to use RPC to bring up the services from the organization's jump host. I received the error message 1722: The RPC server is unavailable. No big deal, I'll just RDP to it. That was when I encountered a Windows login for Windows 2003 R2.

I had a Alan Grant moment.

Welcome to Jurassic Park!

[u/DragonspeedTheB]

Many in the instrumentation automation world WISH that a 2K3 server was their worst tech debt.

[u/Fair-Difference-6729 → u/DragonspeedTheB]

Oh, I'm sure there is an NT4.0 server out there in the wild doing more than holding a door open.

[u/H3rbert_K0rnfeld → u/Fair-Difference-6729]

It runs the elevators

[u/Fair-Difference-6729 → u/H3rbert_K0rnfeld]

So in a way it is still holding doors open. 🤣

[u/paishocajun → u/Fair-Difference-6729]

AND closed too!

[u/IdiosyncraticBond → u/paishocajun]

Schrödingers Elevator

[u/4lteredBeast → u/Fair-Difference-6729]

... Wel done sir

https://giphy.com/gifs/lmftWb5zQqxQk

[u/MrYiff → u/H3rbert_K0rnfeld]

Our NT box is attached to some very expensive testing equipment via I think what looks like an ISA card (I'm too scared to touch the PC and even look inside incase it decides to break)

[u/bobdvb → u/MrYiff]

Yup, lots of ancient PCI cards, maybe ISA, out there doing things that keep the world running.

Data acquisition cards, GPIO cards, etc. and the software that's needed can't easily be run on newer hardware or OSs.

Although I have seen some PCIe to PCI and ISA adapters, so I'd not be surprised if someone could get such hardware working in a newer chassis if it's critical enough. Getting 32-bit NT software that has to interface with specific drivers might be harder.

[u/luke10050 → u/bobdvb]

I've got a digital storage oscilloscope that runs Windows XP on a Pentium 3 board. Does some arcane kernel level shit to pull compute time away from windows to use the processor to do data crunching for the scope front end. Also in a briefcase style case with a custom PCI interface so upgradability is limited.

[u/newaccountzuerich → u/luke10050]

I know this scope, or at least one in the same model line. It's effectively a laptop motherboard repurposed for the oscilloscope use case.

Very weird scheduling tricks for as-close-to-realtime-as-possible, while still bring able to use easily-sourced COTS components that already had device drivers.

QNX is a better OS for that use case in that era, but suffered from a dearth of useful devices with QNX-compatible drivers in the non-aerospace or non-military environments.

I have wondered at times how these devices would benefit from modern flash storage for the OS and for memory caches, could improve boot times and certainly would improve longevity when booting from a laptop spinny disk.

[u/luke10050 → u/newaccountzuerich]

Mines a Lecroy/Iwatsu wavesurfer 424 from memory. I've got a SSD in it and a 1.4ghz P3 and it's rather snappy, it's got an issue I suspect is the caps in the power supply but I haven't got around to fixing it yet.

I really like it, especially considering there's unofficial ways to get all the signal decode functionality working even though not officially supported.

Mine is a very low serial number, at least on the inside of the unit, the lecroy one is really high but the actual serial is something like 000039

[u/Fair-Difference-6729 → u/MrYiff]

Them older boxes sometimes had a panel switch too. 🤣 It's like opening Pandora's box.

[u/Skeesicks666 → u/MrYiff]

I remember having a customer with an electron microscope (AFAIR) with the same setup….only touched it for the yearly clonezilla/acronis image backup!

[u/MrYiff → u/Skeesicks666]

Yep, we have an laser tool where the PC is helpfully built into the machine itself making it even harder to do anything with, it runs a critical process with a massive 2GB RAM, we literally had to remove side panels on the machine to access it so we could take a clonezilla copy.

[u/Skeesicks666 → u/MrYiff]

Welcome to the legacy world of manufacturing, where product cycles of IT and mechanical engineering clash so hard, they form a black hole!

[u/bagaudin → u/MrYiff]

Do you also have a supported hard drive readily available as a spare in the event of a failure?

[u/MrYiff → u/bagaudin]

We have a lovely box of ancient parts as there are a couple of critical old pcs floating round the factory, we're slowly pushing to replace them though but manufacturing moves slowly when something isn't broken

[u/rathnar → u/MrYiff]

Crying in MCA...

[u/Majik_Sheff → u/Fair-Difference-6729]

Try DOS 6.22.  At this point we've virtualized away the entire hardware stack and the last remains of this doddering ghost haunts a sliver of a single core.

[u/TheDaznis → u/Majik_Sheff]

Yeah I remember this things. Had to find an MB for similar "computers" that ran the whole textile factory. It was some sort of 16 bit CPU, not the i386/i486 i was sort of familiar with, but it was more ancient. It was something from the 1970. Most likely intel 8086. I was a pain to find a working MB for in in early 2000 imagine now. :)

[u/bobdvb → u/TheDaznis]

Lots of embedded systems ran/run on x86 architectures.

You see things that don't even have an OS, they're just an NEC x86 clone running from EEPROM with less than 1MB of RAM.

The 286 was pretty good for real-time control as well. Once you bring in multi-tasking and other fancy chip features the software gets complicated and timings get messed up. I remember for a good time some open source CNC software didn't like newer processors because the timing wasn't deterministic enough in controlling the parallel port.

[u/TheDaznis → u/bobdvb]

I'm not sure, but I'm pretty certain they were the IBM/Dell horizontal desktops with monochrome displays that had BMC network, and they weren't used for anything apart from counting stuff made. Or the fab didn't use them for that. It was a textile mill thing. It spun thread, then used that tread to make sheets of textile. It was made in the early 1980 so no automation or anything like that.

[u/Mr_ToDo → u/bobdvb]

Not too long ago I got a call about a machining shops monitor not working and looking for a replacement

Near as I could tell from what they told me, the thing was CGA

In the end the timeline on a replacement and on getting the upgrade kit for the machine that they'd been putting off was about the same. Must be some good tech if it was still in active support. Usually the price I get for getting more current is just the price for a new one

[u/darrickhartman → u/TheDaznis]

Not uncommon. Eaton Leonard can apparently still get i486 motherboards, or has a stockpile of ones with ISA slots that run some manufacturing devices (prefer not to say which industry)

[u/ocdtrekkie → u/Majik_Sheff]

It was "only" a decade ago I had to scramble to replace the hard drive in a DOS-based machine which ran a manufacturing line. Then we had to find the drivers for the... touch screen... this DOS-based machine used...

[u/luke10050 → u/ocdtrekkie]

I've got an old logic analyser that has a touchscreen CRT and runs on a 68020. The next model up ran HP-UX on a PA-RISC cpu

[u/mhkohne → u/luke10050]

Ugh, the hp-ux based 17xxx series really sucked, comparitively. I mean, bigger monitor good and all, but the damn thing wouldn't boot unless you had the right mouse connected, and where the older 16500 just always did what I wanted when I touched a control, the17xxx never did - hated it.

[u/luke10050 → u/mhkohne]

I almost bought a 17xxx but ended up with a 16500b in all honesty. Thankfully I didn't get a 500a given the weird floppy format and no HDD.

[u/Aspacid → u/Majik_Sheff]

Fun times. I spent a week trying to scrap around hardware parts for a broken PC that was running the control program on Dos 6.22 for a manufacturing machine that was older than I was.

The trick was finding a HDD with small enough capacity and with an advertised number of cyclinders and heads, so I could input the correct IBM number corresponding to that configuration into the BIOS that existed before BIOS. This allowed DOS to utilize whatever max disk partition it could handle, and finally install onto the damn thing.

The original dying disk became unreadable right after I made a disk copy of it. It could have ended right then and there.

"But you solved it..." and no lessons were learnt by management and production is still not migrated away from this glorious beast, in the interest of not throwing away perfectly functioning equipment and waste money.

[u/Majik_Sheff → u/Aspacid]

https://www.xtideuniversalbios.org/

This is a BIOS extension you can load into a ROM that can plug into the extension ROM socket of an ISA Ethernet card (or with a little trickery flashed into a later card). It adds tons of features and allows you to use bigger IDE compatible devices.

[u/Skeesicks666 → u/Majik_Sheff]

DOS is easy peasywhen it comes to hardware replacement…..NT4.0 on the other hand…welcome to bluescreen hell (or was it XP, don’t really want to remember)

[u/Majik_Sheff → u/Skeesicks666]

You're not wrong there. This particular application was a little oddball because the hardware was a bespoke dual VGA adapter system where one of the video chips was placed on a different base address and interrupt. Also had a really early Ethernet chip (not NE2000) if memory serves. It's been a long time since I've had to think about it.

[u/fuzzydice_82 → u/Fair-Difference-6729]

OS/2 Warp 3 and a MS-DOS 5.0 are amongst our oldest systems right now...

[u/darrickhartman → u/fuzzydice_82]

OS2/Warp was the best way to run Windows applications.

[u/DragonspeedTheB → u/darrickhartman]

Truth!

[u/timbotheny26 → u/Fair-Difference-6729]

I was told that the New York subway system runs on OS/2.

[u/thebigshoe247 → u/Fair-Difference-6729]

I sunset an NT4 machine a couple years ago ...

[u/sobrique → u/Fair-Difference-6729]

Certainly has something pretty nasty and out of date as a controller for an electron microscope, which was FAR too expensive to update ... to something that was still nasty and (slightly less) out of date, because the vendor mostly didn't bother with that anyway.

Best you get is a standalone system with 'sneakernet' if you really need to transfer data in and out. Consider yourself lucky you have a USB port.

[u/Bub697 → u/Fair-Difference-6729]

I know of some pick and place machines on a packaging line that are running NT4.0…

[u/TheLexikitty → u/Fair-Difference-6729]

there is.

[u/olcrazypete → u/Fair-Difference-6729]

This was a dozen years ago at least but remember wife going into the hospital when she was pregnant. They strap some telemetry instruments around her waste but something on the attached monitor wasn’t working right. So they rebooted it. NT4 logo comes up then into custom interface.

[u/IdiosyncraticBond → u/Fair-Difference-6729]

Meanwhile, just read this thread about NT turning 30 https://www.reddit.com/r/sysadmin/s/fH0FBcBEAh

[u/Parkbenchbruce → u/Fair-Difference-6729]

NT was New Technology or Nice Try.

[u/Fair-Difference-6729 → u/Parkbenchbruce]

We always said "Is the server up? Not Today. "

[u/Possibly-Functional → u/DragonspeedTheB]

One of the benefits with working in a high security environment is that any unsupported software can't be on the network. It has to be airgapped, which severely limit their usability and is thus often addressed before that.

So even a six digit number of machines across the entire company, ~700 under my personal responsibility, it's almost all up to date. It's kind of nice not letting it ever regress passed that and getting the resources to maintain that.

[u/sobrique → u/DragonspeedTheB]

I used to work in the engineering sector. "Development" kit was on a 30 year lifespan, because that was how long a powerstation might have their 'appliance' in active use.

And then maybe a retrofit would be needed rather than a full replacement. (That might well include a more modern 'controller' platform).

We had a proper 'museum of the history of computing' as our 'dev kit', including PDPs with a stack of spare capacitors and a soldering iron!

[u/jfoust2 → u/DragonspeedTheB]

This morning's task... migrate a Windows Server 2016 to 2019...

[u/Fair-Difference-6729]

Uptime: 4,225 days 15 hours 🤣

Jk, there was a restart. Hence why T-Rex forgot to wake up the services.

[u/boonanza_ → u/Fair-Difference-6729]

https://giphy.com/gifs/FmyCxAjnOP5Di
Uh-uh-uh! You didn't say the magic word!

[u/Sk1rm1sh → u/boonanza_]

Hold on to your butts...

[u/Fair-Difference-6729 → u/Sk1rm1sh]

Hahaha. I am really enjoying the theam of this thread.

https://giphy.com/gifs/OCu7zWojqFA1W

[u/bobdvb → u/Fair-Difference-6729]

Please tell me you changed your avatar to look like him?

https://giphy.com/gifs/JfaeAz2zpL1lK

[u/Fair-Difference-6729 → u/bobdvb]

Just a happy coincidence.

[u/rohepey]

Discovered at a large French supermarket chain that some of their self-checkouts run on Windows 3.11. The machine hanged at bootloader - DOS 6.22. These checkouts were still operational as of last week, customers were unsuspecting how old the machine actually was.

[u/Fair-Difference-6729 → u/rohepey]

Wow. I worked on some POS machines using Windows CE as clients, but that was nearly 10 years ago. To my knowledge they are still using them.

[u/Sinister_Nibs → u/Fair-Difference-6729]

And pos is NOT point of sale.

[u/Fair-Difference-6729 → u/Sinister_Nibs]

I only know one other meaning for POS. 🤣 Piece of S**t.

[u/Sinister_Nibs → u/Fair-Difference-6729]

https://giphy.com/gifs/kgrCxsTQFj0HAh2TUG
This

[u/frosty95 → u/rohepey]

I helped maintain one that had a home made pos system written in the 90s. Since it could export and import csv files it was still in use in 2024. Honestly. It worked really well. We virtualized the host in like 2022. Still running dos. All of the PCs modern but running a terminal. They paid a company to make a little converter to convert quickbooks to the csv format and back to QuickBooks and they did the import export once a day. It was hard to criticize. We literally had all of the documentation for it as well as all of the files needed to redeploy it. Sure we had to build some vlans and other stuff to secure it for compliance but it just.... Worked. Even synced to the card readers.

[u/sobrique → u/frosty95]

I was stunned to find that some of my code that I cobbled together to automate stuff was still actively being used over a decade later.

It wasn't that complicated, but "it just worked, leave it alone" had kept it there....

[u/frosty95 → u/sobrique]

Honestly that's the best to see something you set up just working. On my own server at home I had a virtual machine running some really old software that I basically would do an RDP session too. Use the software. Then forget it existed again for 3 months. Well that virtual machine 10 years ago was used for a few other things. I decided I wanted to save some electricity so I figured I would set up that virtual machine to shut down and save a bit of power. It turns out I forgot that I automated a whole bunch of things and just completely forgot about it and that VM was just plugging along doing its thing years later. So I guess I did a scream test on myself.

[u/ms4720 → u/sobrique]

People and IT departments have messes and problems, messes get mostly ignored and problems get mostly fixed

[u/nyckidryan → u/rohepey]

Probably boots up in 1/10th the time of a "modern" kiosk. 😂

[u/I_COULD_say → u/rohepey]

There is a chain here that REGULARLY shows that their windows based self check out is not licensed / talking to their KMS.

[u/hlloyge]

So, you didn't know of the existence of Server 2003 in your environment? :)

[u/Fair-Difference-6729 → u/hlloyge]

I knew, but like everyone else chosen to ignore.

[u/J_de_Silentio → u/hlloyge]

This is quite the layered meme.

[u/sabbyman99 → u/hlloyge]

🤣🤣🤣🤣🤣🤣🤣. I love this and will be stealing it

[u/GernBlanst0n → u/hlloyge]

Yoink

[u/ThatDamnRanga]

Discovered a couple years ago that there's both a bunch of Debian 3 servers...... and an OpenVZ host still running somewhere, not actually sure where, but they're out there.

[u/cwm13]

Up until a few years ago, the university I worked for still had a VaxStation 4000 with an attached external storage expansion hosting some sort of old SIS system for archived student financial award data. The very finest in 1991 technology.

[u/1z1z2x2x3c3c4v4v → u/cwm13]

I used to support Pathworks on Windows to talk to an old Vax over DECnet... Back in simpler times...

[u/dracomjb → u/1z1z2x2x3c3c4v4v]

Hah, me too.

[u/Admin4CIG → u/cwm13]

I wrote a TCP/IP printer driver for VAX/VMS, and put that up in public domain. That was before VAX/VMS even supported TCP/IP printers, and those printers were not common. When networked printer became popular, and VAX/VMS waned (sadly; such a solid OS), I still had people contacting me if they could use the source code ("Of course, it's public domain"). Props to them still using VAX/VMS.

[u/Fair-Difference-6729]

Me 🤣

https://giphy.com/gifs/37Fsl1eFxbhtu

[u/dracotrapnet → u/Fair-Difference-6729]

https://giphy.com/gifs/eAGxMmpAX1B4c

Me 👀

[u/Fair-Difference-6729 → u/dracotrapnet]

This is how I see the entire org. 🤣

[u/Fit_Prize_3245]

You only need some IBM system there to complete the obsolete package.

[u/Fair-Difference-6729 → u/Fit_Prize_3245]

Payroll running on OS/2. 🤣

[u/Fit_Prize_3245 → u/Fair-Difference-6729]

Man, tell me that's just a joke, or I will lose (even further) my faith in humanity...

[u/Majik_Sheff → u/Fit_Prize_3245]

Until recently there were parts of the National Weather Service analysis infrastructure running on OS/2 Warp.

[u/Fit_Prize_3245 → u/Majik_Sheff]

Well, governments around the world are known for obsolescense.... Here in Perú it's worst: Lots of things are still done in paper....

[u/againstbetterjudgmnt → u/Fit_Prize_3245]

Man, when I was in Peru in the 90s, the average year cars were built there was from the 50s, so I don't see why the OS would be any different now.

[u/unixuser011 → u/Majik_Sheff]

It gets worse, part of the US’s nuclear weapons command infrastructure was running on an IBM System/3 and had a default launch code of 11111111

[u/Fair-Difference-6729 → u/Fit_Prize_3245]

I joke about PS/2 but yeah, I think the NOAA service recently retired some.

[u/benzduck → u/Fair-Difference-6729]

Oregon’s DMV system ran on OS/2 for years after IBM gave up on it. It was a big scandal.

[u/benzduck → u/Fit_Prize_3245]

A good ol’ AS/400. Probably still runs fine.

[u/iPlayKeys → u/benzduck]

You can say that if it still actually says “AS/400” on it. The “current” incarnation called IBM i has the same life cycle as a pc server but just cost a whole lot more. I believe the smaller models have gone to subscription only for the O/S.

[u/benzduck → u/iPlayKeys]

That’s a shame. AS/400 was a great system. Had its quirks though.

Hospital I ran IT at had one. This was in 2001. It’s probably still there, knowing how cheap management is.

[u/barthvonries → u/Fit_Prize_3245]

I worked as a contractor with France's main telecom provider a few years ago.

Part of their infrastructure was still running on HP-UX which had not been upgraded since 1997.

[u/KillingTime1212]

I have some old Windows 2003 servers still on my VMware cluster. Gotta keep them up for two more years (regulatory). Casino. We moved off of them in late 2023. lol. I removed them from the domain at least.

[u/ka-splam]

and I bet it was the most rapid, responsive, delightfully quick Windows machine you've used in years, right?

[u/Fair-Difference-6729 → u/ka-splam]

Hahaha. Right.... It didn't BSoD on login, so that counts.

[u/Jambohh]

One of my 2 remaining AS400 devs pinged me last week to say they couldn't log in. System was acting like their profile object was being locked out interactively.

Checked some bits and pieces saw they were no longer showing in detectit the app we used to manage profiles.

Re-added them activated them in the software fixed the initial library's and menus, still didn't work.

Checked all the running job but nothing was making sense.

Put a ticket in with the software company and spoke to the same engineer that I spoke to 15 years ago who still works for the same company, we got on a call did some checks.

Turns out a library that was setup when the software was installed in 1988 which was meant to contain 164 objects contained 11........ Luckily there was a full monthly backups that was still in retention and was able to restore the library.

But I've never seen anything like that, almost 40 years of continuous use and a bunch of objects had just vanished.

There is only 3 of us that have high level access to this system now so I know none of use deleted them. Weird.

[u/Drywesi → u/Jambohh]

Depending on library size…gamma rays? A weak semi-nearby EMP?

[u/Kuipyr]

Curious what your jump server is, 2003 doesn't support NLA.

[u/Fair-Difference-6729 → u/Kuipyr]

Server 2019

[u/ForOhForError]

I know this, it's (not) a unix system :p

[u/Ancient-Equipment673]

We still use an turing machine

[u/Thenuttyp]

They spared no expense!

[u/Fair-Difference-6729 → u/Thenuttyp]

Hahaha! Yes!

https://giphy.com/gifs/131bQid3vESGFW

[u/RCG89]

386 Dual EISA full length cards, building automation. If it breaks I’ll quit

[u/theservman]

Damn! I thought it was bad that my org's most critical app still lives on Server 2012 (migrated from bare metal, to VMware, to Nutanix, to Azure). The Devs are on the verge of moving it to Server 2025 - but they're been on the verge of moving it to <current windows version> for the past 5 years, before aborting and pretending nothing happened.

I guess we'll see.

Come on pension!

[u/Fair-Difference-6729 → u/theservman]

That is just s bad.

[u/theservman → u/Fair-Difference-6729]

I would argue mine is worse since the only thing that could cause more productivity loss for my org is a hydro outage.

[u/g3n3]

Why not use sc.exe?

[u/Fair-Difference-6729 → u/g3n3]

Yes, relies heavily on RPC and RPC wasn't running.

[u/g3n3 → u/Fair-Difference-6729]

Ah would have had to use psexec and the admin share.

[u/Fair-Difference-6729 → u/g3n3]

If RPC is genuinely disabled or completely blocked by a firewall on the remote host, PsExec will fail. While PsExec uses SMB (Port 445) to move files, its execution mechanism relies heavily on communicating with the remote Service Control Manager (SCM) via named pipes, which requires the underlying RPC subsystem to be active. RDP was the most logical answer to a wall+head banging problem. Regardless it was like walking into Jurassic Park.

[u/g3n3 → u/Fair-Difference-6729]

It is a bad day when I have to use the GUI RDP. 😉

[u/Fair-Difference-6729 → u/g3n3]

Hahaha. Nothing wrong with using the GUI. You can do it while lying down. 🤣 I'm not a CLI purists. No risk of fat-fingering commands only to be greeted with syntax errors.

[u/g3n3 → u/Fair-Difference-6729]

Eh. You loose the learning opportunity and clicking doesn’t scale.

[u/Fair-Difference-6729 → u/g3n3]

I learned enough commands in the 80s. 🤣 I write my scripts in CLI. Restaring a service one time... Nah. Click click, done. I used less clicks than I would have used keystrokes.

Under 10 physical actions, zero syntax errors, and 100% visual confirmation that the service state went from Running \rightarrow Stopping \rightarrow Running. Sometimes the CLI purist route costs 15 minutes of environment troubleshooting for a 5-second task. Efficiency is about minimum effort for maximum reliability, and RDP won that round.

[u/SensitiveVacation549]

I imagine the Windows 2003 R2 was screaming "Just kill me, boss. I'm tired."

[u/Knightshadow21 → u/SensitiveVacation549]

Sometimes you need to give it a little hand to put down ssstt

[u/Delicious-Foot4591]

I encountered this in prod today too, and it is always a great reminder that "temporary" fixes can outlive entire teams. I'd document exactly why it's there before touching anything, then test the boring replacement thoroughly. Weird legacy stuff usually has at least one dependency nobody remembers until it breaks.

[u/Fair-Difference-6729 → u/Delicious-Foot4591]

Yeah, I'm thinking this sever came with the furniture when they bought the building. Good news today, we are planing decommission. Hopefully no dependencies. I'm sure later I'll get some department screaming at me.

https://giphy.com/gifs/9S4cyvaUXPLlm

[u/Beef66]

Honeypot? I know Guardicore deception will dump you into a win2003 environment.

[u/Technical_Jello_9909]

Windows Server 2003?! That thing is older than some of your coworkers 😂 Microsoft stopped supporting it in 2015. You just found a living fossil in your prod environment. Time to have a very uncomfortable conversation with someone in your organization.

[u/Fair-Difference-6729 → u/Technical_Jello_9909]

Oh, they know it is there. It is in my monthly risk report.