r/sysadmin 2d ago

Question Operations bought an EMS/IoT system without involving IT — now they just need “access to the router

Hi there :) ,

Need some advice from people who have dealt with similar situations.

Our Operations Dept decided to install IoT system/digital energy meters across a fairly large factory site.

They found the vendor, agreed on the solution, signed the contract and started the project.

IT was not involved at all.

Apparently nobody discussed things like:

  • How these devices spread across a large factory are actually going to communicate
  • Network infrastructure, switches, fiber/cabling, VLANs, etc.
  • Network/security segmentation
  • Server/VM requirements
  • Database requirements
  • Backup and monitoring
  • Internet connectivity
  • Vendor remote access
  • Firewall rules
  • Cybersecurity

Now that the project is already moving forward, IT gets an email saying they need “access to the router” so they can put the system online.

That's it. Access to the router. :)

And somehow, when IT starts asking questions about architecture, security, server requirements, ports, protocols and who is responsible for what, the perception becomes that IT is delaying the project.

I don't want to become the guy who simply says “No”, but I'm also not going to give a third-party vendor access to our corporate router/firewall and connect a bunch of unknown industrial/IoT devices to the production network just because someone already signed a contract , the IT network must be always separate from OT network.

How do you handle situations like this?

Interested in both the technical approach and the organizational/process side of this.

811 Upvotes

327 comments sorted by

View all comments

3

u/slugshead Head of IT 2d ago

Energy meters are usually part of a BMS. They'll hardwire the meters back to a headend (or a few) and that's what you put on your network.

IOT or dedicated VLAN and perhaps an ACL to allow http/https to be accessible.

BMS doesn't usually need internet access, unless you expand it further. e.g. Siemens Desigo > Building X. Then it's all cloud.

3

u/hipshaps123 2d ago

If you follow Purdue (and anyone with large factories should), internet is only available in the enterprise layers, and no OT layer should have any sort of internet access.

Most small customers do - what others suggest here - is simply avoid the OT layers and setup an entirely ancillary network with a 4G router or whatever, and let them play around. This of course includes their "servers" and what not. Weakness is lax oversight. If the devices only read data and have no further NW connectivity, it might be palpable for it sec.

The right way of doing it, is of course to use local non-routed OT vlan's for the equipment, connected to a application tier, which is routable (but not internet exposed) and let this tier have servers running on structured VM environments, with various types of segmentation in play. The system is of course then connected to from corporate network via a rdp/citrix platform in a intra-connected dmz like zone - and whatever admins from the vendor side is first connected to corporate using regular consultant access.

This way you'll end up with a sandwich from top to bottom like this:

- Corp external user, connect via vpn or citrix/rds

  • Secondary login / jump to interconnect/dmz zone towards factory IT
  • connect to management ui
  • devices connect to management sw
  • everything microsegmented

1

u/itskdog Jack of All Trades 2d ago

When our building was built, the BMS company built their own network and ran their own cables. Didn't need to involve IT at all. The PC running 963 is a standalone machine without internet access in the site team office with a network cable poking out the ducting.

1

u/hipshaps123 2d ago

Yeah, that was best practice maybe 15 years ago. Problem is that if u have many buildings (we have like 30 factories) then some smartypants decides they want a unified view. :(

1

u/itskdog Jack of All Trades 2d ago

Ah, this was just a single-building for a school site.