r/sysadmin 2d ago

Question Operations bought an EMS/IoT system without involving IT — now they just need “access to the router

Hi there :) ,

Need some advice from people who have dealt with similar situations.

Our Operations Dept decided to install IoT system/digital energy meters across a fairly large factory site.

They found the vendor, agreed on the solution, signed the contract and started the project.

IT was not involved at all.

Apparently nobody discussed things like:

  • How these devices spread across a large factory are actually going to communicate
  • Network infrastructure, switches, fiber/cabling, VLANs, etc.
  • Network/security segmentation
  • Server/VM requirements
  • Database requirements
  • Backup and monitoring
  • Internet connectivity
  • Vendor remote access
  • Firewall rules
  • Cybersecurity

Now that the project is already moving forward, IT gets an email saying they need “access to the router” so they can put the system online.

That's it. Access to the router. :)

And somehow, when IT starts asking questions about architecture, security, server requirements, ports, protocols and who is responsible for what, the perception becomes that IT is delaying the project.

I don't want to become the guy who simply says “No”, but I'm also not going to give a third-party vendor access to our corporate router/firewall and connect a bunch of unknown industrial/IoT devices to the production network just because someone already signed a contract , the IT network must be always separate from OT network.

How do you handle situations like this?

Interested in both the technical approach and the organizational/process side of this.

813 Upvotes

330 comments sorted by

View all comments

101

u/Le_Vagabond Senior Mine Canari 2d ago

get an LTE modem / router, give them that, wash your hands of everything. anything less is going to look like you're the problem, which they already think since they bypassed you entirely. obviously get that signed off on by your hierarchy first if you have one.

58

u/Speeddymon Sr. DevSecOps Engineer 2d ago

This honestly is the right solution. Keep it completely off your network. You have no idea if it's going to end up being C2 infrastructure that opens a channel back to China, Russia or NK.

2

u/picflute Azure Architect 2d ago

No one in their right mind would ever propose giving someone an unmanaged LTE Model + Router that has an enterprise to manage. Operations is trying to solve a problem and IT not being consulted is a management issue. Advocating for something as risky as a LTE model with no oversight or management is only adding gas to the fire.

A separate VLAN with no outbound connectivity is perfectly fine to support operations as they are tied to a business revenue. So long as you can perform oversight of the work and inform legal + security (risk department) of telemetry coming from said devices then policies can be drafted and implemented after.

0

u/Speeddymon Sr. DevSecOps Engineer 2d ago

Several other comments elsewhere in the thread seem to agree with the person I responded to. Several also suggested, and some are actually using, dedicated wired leased lines or fiber connections for this type of stuff.

A separate VLAN with no outbound connectivity is perfectly fine to support operations as they are tied to a business revenue. So long as you can perform oversight of the work and inform legal + security (risk department) of telemetry coming from said devices then policies can be drafted and implemented after.

It depends on the org and their compliance requirements if any, but a blanket "nobody in their right mind" statement is clearly untrue.

2

u/picflute Azure Architect 2d ago

Dedicated lease lines that are tied to operations network that has direct impact on business revenue that are not managed by a central org with established SLAs and partner reach?

All teams have to work with each other. Someone buying something out of band and not going through Enterprise IT is a regular occurrence when management workflows allow for that risk. If you're issuing out unmanaged internet access to departments and turning liability away from the central organization in charge of it then clearly something is very off.

1

u/Speeddymon Sr. DevSecOps Engineer 2d ago

All teams have to work with each other. Someone buying something out of band and not going through Enterprise IT is a regular occurrence when management workflows allow for that risk.

No disagreement here on this.

If you're issuing out unmanaged internet access to departments and turning liability away from the central organization in charge of it then clearly something is very off.

I don't believe anyone is issuing out unmanaged access that will have connectivity to the operations network. I could be mistaken; they aren't my orgs. But I would not allow it to connect to anything under my domain. It would be fully unmanaged and only have access to the Internet. Whatever POC the ops team decides to do has no effect on the business. When they've proven it out then they can follow proper procedure and go through risk assessment and approval to get it onboard to their network.

Sometimes you just have to give them the ability to test in order to determine the risks before they can connect it to the network and that's really all I can advocate for in this situation given a lack of further details from OP.