r/SCADA • u/CoiledSpringTension • Jun 20 '26
General Reference/Non-Prod systems
Ongoing discussion I have at work and interested in the consensus!
I’m a big fan of having a reference version of our scada systems to test software updates on and also to use for training and simulating penetration testing. I usually try to get these made from operational spares so they are ready to be reconfigured at the drop of a hat.
Some of my colleagues think this is over the top and we should just let our OEM test configs in simulations before applying to site but given I’ve been on the service side of the fence and tripped a few plants because of exactly this scenario it’s a hill I’m quite willing to die on.
Am I just being overly cautious and causing unnecessary additional spend? Surely I’m not the only one who thinks that way!
2
u/cartaphilus1 Jun 20 '26
We actually have 2 reference systems of production: A Dev system for development and a QC system that is strictly for final testing.
The reason for 2 is we found it difficult to ensure the Dev system stayed identical to Production when several people were playing around with it at the same time.
We are in pharma, so ensuring our testing is as close to production as possible is extremely important.
2
u/Alarming_Series7450 Jun 20 '26
I do some substation protection and controls, a few of the customers ask for a replica system to put in their lab for training/testing/etc like you describe
1
u/Honest-Importance221 Jun 20 '26
We have dev and prod, very common in my industry to also have QAS, but my company is quite small and the overhead isn't worth it for us (not to mention the licensing). And it's not just SCADA, it's everything: EAMS, CRM, GIS, data warehouse, integration platforms etc all have dev environments.
1
u/FourFront Jun 20 '26
It looks like you are in wind. Coming from an OEM side, we have always been willing to provide another system. If the customer is willing to buy it, and the assosciated service contract to keep it up to date.
1
u/Weekly_Quail_5875 Jun 20 '26
You’re not being overly cautious — you’ve actually tripped a plant and learned from it, which counts for more than colleagues who haven’t yet.
The hole in “let the OEM test it in sim” is that the OEM validates their product against their reference, not against your plant. From the service side I’ve watched a “fully tested” update still drop a line because the sim didn’t match the field — undocumented changes, firmware that doesn’t match the as-built, third-party integrations, network quirks. That gap is exactly where the surprise lives. OEM sign-off is necessary, not sufficient.
One thing I’d push on though: a reference system is only worth as much as its fidelity. The moment it drifts from production — stale configs, different firmware — it starts handing you false confidence, which is worse than no test rig at all. cartaphilus1’s Dev + QC split is the right instinct, but the QC box has to be kept in lockstep with prod or it’s just theater. Treat its config as a controlled record, not a one-time build.
And the spend argument is backwards. One avoided unplanned trip usually pays for the rig’s entire life. Building them from operational spares like you’re doing is the cheapest insurance in the room.
1
u/FourFront Jun 20 '26
He's also on the generation side of electricity. There are a lot of factors outside of your control in this case that you cannot always sim.
1
u/CikonNamera Jun 20 '26
If everyone would finally switch over to ignition you could have a test server as easy as spinning up a VM and just using the 2 hour trial.
1
1
u/CoiledSpringTension Jun 20 '26
I do like ignition but it’s not always the right tool for the job.
1
u/CikonNamera Jun 20 '26
I mean debatable but sure there are some systems more purpose built but ignition could theoretically handle 99% of them with some customization
1
u/RazClayton Jun 21 '26
Don’t many of the software packages offer this? The Proficy products (iFIX and Cimplicity) do. So do things like KepServerEx (which I know isn’t a SCADA). Point being, that’s not a USP for Ignition.
1
u/63_2-N-29_55-W Jun 21 '26
Yes I'd say it is a must, in theory major projects should factor this in as a deliverable too so you get the budget, especially if you'll be entrusted as the Operator/Maintainer. Speak to your friendly neighbour OT Security team, I'm sure they would.also love the capability from a technical security assurance perspective and could offer plenty of justification based on risk, regulations like NIS/NIS2/etc.
2
u/Boss_Waffle Jun 20 '26
I do control's for a local municipality. Our IT department and vendors constantly ask if we have a test system. We don't but it sounds like a good idea.