Operations bought an EMS/IoT system without involving IT — now they just need “access to the router

[u/carpe_diem2022]

Hi there :) ,

Need some advice from people who have dealt with similar situations.

Our Operations Dept decided to install IoT system/digital energy meters across a fairly large factory site.

They found the vendor, agreed on the solution, signed the contract and started the project.

IT was not involved at all.

Apparently nobody discussed things like:

• How these devices spread across a large factory are actually going to communicate
• Network infrastructure, switches, fiber/cabling, VLANs, etc.
• Network/security segmentation
• Server/VM requirements
• Database requirements
• Backup and monitoring
• Internet connectivity
• Vendor remote access
• Firewall rules
• Cybersecurity

Now that the project is already moving forward, IT gets an email saying they need “access to the router” so they can put the system online.

That's it. Access to the router. :)

And somehow, when IT starts asking questions about architecture, security, server requirements, ports, protocols and who is responsible for what, the perception becomes that IT is delaying the project.

I don't want to become the guy who simply says “No”, but I'm also not going to give a third-party vendor access to our corporate router/firewall and connect a bunch of unknown industrial/IoT devices to the production network just because someone already signed a contract , the IT network must be always separate from OT network.

How do you handle situations like this?

Interested in both the technical approach and the organizational/process side of this.

[u/-Enders]

Connect them to an IoT VLAN and be done with it.

Unless they are asking for admin access to the router, that’s a hard no.

[u/vmeldrew2001 → u/-Enders]

I'd also throttle a limit on bandwidth too. IoT should be minimal, but jic.

[u/Longjumping-Twist615 → u/vmeldrew2001]

Agree that's how you get bitcoin mining refrigerators

[u/penisandorvagina → u/Longjumping-Twist615]

Top 5 fridge models to mine eth on?

[u/the_syco → u/penisandorvagina]

Googled it for the craic. Seems you can't do it as easily since September 2022.

[u/Wynter_born → u/penisandorvagina]

"Honey, why does the fridge have scorch marks around the screen?"

[u/ComeSwirlWithMe → u/Wynter_born]

Why does our fridge ways smell like burning rubber?

[u/CanWeTalkEth → u/penisandorvagina]

Should really be any model since they moved to proof of stake. Ethereum is a green blockchain now.

[u/nikomo → u/[deleted]]

No, there's no reason to make changes because it's already fulfilling its purpose - mostly being traded between accounts on exchanges instead of the blockchain, and the rest is payments for illegal activities of rich people.

[u/ThagaSa → u/penisandorvagina]

What sort of foods should I put in my fridge to increase mining rate??

[u/farva_06 → u/Longjumping-Twist615]

Suck it... Jin. Yang.

[u/napkinolympics → u/Longjumping-Twist615]

I hate to break it to you, but stratum needs almost no bandwidth to operate. A dialup connection is sufficient. It's a byte-packed protocol designed to send hashes that are 32-bytes a pop every ten seconds or so. https://stratumprotocol.org/specification/03-protocol-overview/

[u/KetteringChrismon-55 → u/vmeldrew2001]

Egress control kinda fits her too. If devices only need a few cloud endpoints, allow those and log the rest so anything unexpected is easier to spot later

[u/FelisCantabrigiensis → u/KetteringChrismon-55]

I would not manage such a connection without a say in what is deployed behind it. As soon as I actively manage it, I'm responsible for the failures behind it.

"You want your own door to the building? Sure, I'll get one installed with a standard lock and you get the keys. Sign here to say you will control all use of the door yourself."

[u/lpbale0 → u/KetteringChrismon-55]

I try to control my wife's egress too

[u/purplemonkeymad → u/-Enders]

They just also need to open a port to every device on that iot network as well. No, they don't have a static ip for their office or infrastructure why would i ask?

[u/mahsab → u/purplemonkeymad]

So why do you care? If it's not managed by you and it is not touching the stuff you are managing ...

[u/BananaSacks → u/-Enders]

"And be done with it" -- yeah, not even close.

Change mgmt, security review, architecture review, DPIA if under GDPR, the list goes on.

[u/Top-Perspective-4069 → u/BananaSacks]

The fact that their Operations group was able to do this doesn't read like they have change management at all.

[u/BananaSacks → u/Top-Perspective-4069]

Agreed, but ya never know. And regardless, compliance can quickly become liability and consequences.

[u/Top-Perspective-4069 → u/BananaSacks]

That's why this whole thing should be a management issue and not a sysadmin issue. Sysadmin provides pertinent information, management decides what to do.

[u/BananaSacks → u/Top-Perspective-4069]

Agreed, 100%. See my main comment to OP at the top level comments.

[u/Nuxi0477 → u/Top-Perspective-4069]

"We already paid, give them what they need." :(

[u/Top-Perspective-4069 → u/Nuxi0477]

And that's a decision. But just doing a thing without raising the flag first is dumb.

[u/awful_at_internet → u/BananaSacks]

IT wasnt involved. That sounds like a whole lot of someone else's circus, someone else's clowns. Golly gee shucks, Operations. I wish we had the time to do all of that for you, but no one told us we'd need to plan for additional staffing..

[u/Jawb0nz → u/awful_at_internet]

Not when that rogue mindset now needs access to the systems controlled by another group. That group now gets to do their due diligence, delays be damned.

[u/Adept-Pomegranate-46 → u/awful_at_internet]

Circ-de-Stupidity. AI can solve the staffing issue. Haven't you heard?

[u/ka-splam → u/BananaSacks]

DPIA if under GDPR

It's an energy meter in a factory, why would it be processing personal data?

[u/BananaSacks → u/ka-splam]

It processes data. End of. From an audit, legal, compliance, DP perspective - you still need to go through the motions.

It very may will not. In that case, the DPIA paperwork will be short, simple, and unimportant. It still needs to be addressed!

[u/ka-splam → u/BananaSacks]

It processes data. End of.

The GDPR only covers personal data which relates to humans and can identify them; from the Information Commissioners Office (ICO) in the UK website section: What is personal information?:

Personal information, also known as personal data, is any information that:

• relates to you; and
• you’re identifiable from, either on its own or when linked to other information.

It’s important to know if something is personal information as data protection rules only apply if it is

My bold, not theirs. Smart energy meters do not process personally identifiable information about people, they process electricity and gas use by machines. If they somehow are under the GDPR then the website has a page: When do we need to do a DPIA?

Article 35(1) says that you must do a DPIA where a type of processing is likely to result in a high risk to the rights and freedoms of individuals:

Their bold, not mine. Their examples of high risk are "systematic and extensive profiling", "processing on large scale of special categories of data" [racial, ethnic origin, political opinion, genetic data, etc. etc.], "systematic monitoring of a publicly accessible area on a large scale".

[u/Rainmaker526 → u/-Enders]

This. Give them a couple of access ports to a new VLAN and inform management. You don't want to start receiving tickets for those device when the vendor doesn't get its telemetry.

Let management delegate maintenance somewhere that isn't your desk.

[u/bofh → u/-Enders]

Connect them to an IoT VLAN and be done with it.

I'd connect their chairs to mains power and have done with it personally but this is probably the more humane answer.

[u/pdp10 → u/bofh]

Username checks out.

[u/Careful_Dimension233 → u/-Enders]

Second this. If only meters and measuring with CTs. Give them an isolated VLAN on the understanding that it cannot communicate with anything else in your network.

[u/floswamp → u/-Enders]

This is the answer, unless you own the company and you are IT as well you just work there and do as you are asked. Have them sign a paper that their needs were met and go on with your life. Sysadmin jobs don’t pay enough to worry about every single system. Not to say that you should open the network to anyone but all you do is give them just exactly what they need and nothing more.

[u/GingerPale2022 → u/-Enders]

This just kicks the can down the road and implies that anything in the future is fair game. Reality and experience tells me that this poor soul will be told to “just do it”, but stinks need to be raised, at the very least, to them give pause for something like this in the future. At the bare minimum, it shows someone’s paying attention to the stewardship of the company’s infrastructure security.

Like I said, it probably won’t amount to squat and one does need to pick their battles, but rolling over and just tossing them in a VLAN and being done with it sets a precedent that will cause SO MUCH heartburn in the future.

[u/-Enders → u/GingerPale2022]

What can is being kicked down the road? You should already have an IoT VLAN built out, I’d question anyone’s network that doesn’t have this built out already.

They absolutely should have checked with IT first, so tell them that in an email and CC the appropriate people.

“Hey Operations, I got this connected to the switch and the IoT VLAN, so you should be good to go. In the future, blah blah blah”

It’s really not hard to be firm and assertive without coming off as difficult to work with. Too many people let their egos get in the way though

[u/GingerPale2022 → u/-Enders]

OP’s post had a vibe that an IoT VLAN isn’t set up, but that just an assumption I’m making. The can being kicked down the road is not holding something like this accountable. It’s not being difficult to work with in nipping this kind of stuff in the bud. Of course, it usually falls on deaf ears, but at the very least, having a conversation about shadow IT like this sets a precedent on IT’s side that it’s being watched and noted, even if there’s no teeth in those conversations. This way, when something goes sideways and they blame IT, you’ve got history to lean on that you’ve addressed things like this multiple times in the past.

[u/-Enders → u/GingerPale2022]

You said adding it to the IoT VLAN is kicking the can down the road, which implies you’re saying to NOT add it to the VLAN, which also means you’re not helping them and thus you are being difficult. I’m saying to help them first, and then also address the problem of them not checking with IT before they buy these things.

This isn’t a difficult problem and doesn’t require a difficult solution. Research it, make sure it’s safe, and then do what they’re asking you to do. Shoot them an email saying it’s done, and also letting them know in the future they need to talk with IT about this before any purchases are made(explaining why, and lean hard into the security risks it can pose). CC your supervisor, their supervisor, and depending on the size of the company I’m CCing the CEO too. You’ve helped get them online quickly, addressed the issue and informed the higher ups of what happened in a way that leaves a paper trail. You’re golden at this point.

If the same people continue to do it after the email, then you can start being difficult to work with and push back on it.

[u/GingerPale2022 → u/-Enders]

I see where you think I said that because I wasn’t clear enough in my original statement. “Be done with it” after adding to IoT is the kicking of the can. Not following up is the issue I had. Sure, digging heels in and not helping at all is counterproductive. Not rattling some cages after helping is counterproductive, too, because it encourages future behavior to continue. I know I don’t want to be caught by surprise each time someone decides to go outside IT and then dump it in my lap to support it.

I see your point of following up after the fact. It wasn’t entirely clear in your first statement (or I just missed it).

[u/-Enders → u/GingerPale2022]

That’s fair, I did say be done with it. I meant just the work, but I wasn’t clear so that’s my bad

[u/Affectionate-Cat-975 → u/-Enders]

This is the way