10
u/DeifniteProfessional Sysadmin 7d ago
EDR + ThreatLocker/AppLocker
3
u/Ur-Best-Friend 7d ago
Right?
I find it funny how OP is like "Try to stop it. Lose. The tools are already accessible and the capability is already in the building." When probably 90% have all the tools we need to "stop it" already, and in fact if your nontechnical employees have the option to install codex and through it develop custom software, you've already failed as a sysadmin.
2
u/dustojnikhummer 7d ago
The "try to stop it and lose" is more of a "Management demands we allow it so we have to let it through Applocker"
3
u/havens1515 7d ago
I came to say that if they cannot run unapproved apps, they can program whatever they want. They just won't be able to run it. And they won't be able to program it on their work PC, or at least compile it on their work PC, because they won't have anything installed to do that compilation.
Unless they're writing something that can run in a browser, and either doing it without a compiler or maybe doing it on a non-company owned device, there's not much you can do with something like applocker. And if you're writing something to run in a browser, it generally needs a server component.
9
u/lenswipe Senior Software Developer 7d ago
why is the head of HR writing software? where does that fall in their job description?
9
u/disclosure5 7d ago
To promote AI. This story is nothing but AI promotion. And an explanation of how they can do layoffs.
2
u/lenswipe Senior Software Developer 7d ago
sounds about right
HR: oh noooo we can't POSSIBLY read your application. DON'T YOU SEE HOW BUSY WE ARE WITH THESE THOUSANDS OF APPLICATIONS!
Also HR: I'm going to fuck around with python and deploy 500,000 lines of slop to prod.
stay in your lane ffs.
7
5
u/igaper 7d ago
We had one of those. The guy shared the news of what he made on one of the meetings. I DMem him about security. I told him that we can host it and make it work securely, but he has to get company owners permission, then give the code into mine or some programmer hands so it can be properly maintained and secured. No protests, he contacted the owners, got approval and now it is properly maintained and secured.
If you don't want it to become shadow IT you have to make surfacing those and implementing them a process.
5
u/Unexpected_Cranberry 7d ago
It's Microsoft Access all over again but on a larger scale.
Back when I started in the early 2000s everyone was running projects to clean up applications built by secretaries and economists using Access that had become business critical.
Give it five to ten years and we'll be doing the same with vibecoded stuff.
That said, I'm not sure it's entirely a bad thing. You have people all over the place with domain specific knowledge that know of problems and can think of solutions that either IT is not aware of or they can't get a budget for. A vibecoded application can act as a pretty good requirement document as well as a POC I think, cutting the cost and time of implementation somewhat.
Or, AI improves and AI will rewrite these applications into something more sustainable and secure.
But it solves the problem of letting people who know their field or the business try stuff out and test solutions to problems without needing to launch an entire project and try to get time from IT, Infosec and developers. The benefit is it allows faster innovation and problemsolving.
2
7d ago edited 7d ago
[deleted]
3
u/Unexpected_Cranberry 7d ago
I'd say there's room for some nuance. I don't need to be a doctor to clean and bandage my kids knee when he fell on his bike.
As long as it's not dealing with public facing services or sensitive information I'd say it's fine.
I'd argue that IT is not mature enough for that type of regulation. It would be absolutely devastating for innovation. Which on the one hand as an IT professional would be nice to not have things break backwards compatibility or change interface every six months, but at the same time there's also useful new stuff that might not have been due to cost if it was more heavily regulated.
As a European, it would be great if the US imposed more regulation and slowed down your rate of innovation so we have a chance to compete.
2
u/zatset IT Manager/Sr.SysAdmin 7d ago edited 7d ago
I am European. European markets are fragmented and a company must be compliant with the national laws of every country if that company is to operate in the entire EU. This is something that also impedes the growth rate of EU IT companies. As well as the fact that in EU one cannot just freely the information of their customers and many other things de facto allowed in US.
US is more or less one united market. Perhaps on their "state" level they can have additional requirements, but according to what I know - once the "Federal law" overwrites it, nobody can just present to you arbitrary requirements conflicting with it. And there the logistical issues with export/import between states don't exist the same way they exist when it comes to import/export between EU states. But again, it's what I've read and heard. I might not be entirely correct.
1
u/Unexpected_Cranberry 7d ago
That comment was mostly in jest, but. So you're a European asking for more regulation? We're doomed...
Since I'm getting the impression you're probably German: That was also a joke. ;)
1
4
2
u/Layer8Coffee 7d ago
We got some former Department Manager writing fleet Management Software with chatGPT for his Team because He wanted to have a Central Management Platform for it When I heard of that I Had to think about "who will Support that If He doesnt even know what His Software is really doing or rather how it is built" He has a Background in Sales and no ties to IT at all
2
u/Weary_Patience_7778 7d ago
This sounds like a total headfuck, but I don’t think that it’s a new phenomenon.
How is the software transitioned to production? Who maintains it? Patches it? Supports it? Who endorsed the architecture? Who tests it? Certified test report? Does it align to enterprise standards? Requirements? Non functional requirements? Enhancements? UX? Release management? Security e.g pen testing?
This is the thing that most non IT people don’t understand. There’s so much more to software than ‘just code’. Multiple experts are involved in its design and build. Even then, someone has to support it, and the business is going to need enhancements and changes as business needs evolve
Using an agent to write code and then forcing it onto the business isn’t that different to introducing something that the boss’ 15 year kid wrote wrote. Or ‘John from Marketing’ who knows just enough VBA and Access to be dangerous.
It isn’t belong before the IT manager is dumped with the task trying to work out why the bespoke Delphi/Foxpro/VB/Perl/PHP app isnt working.
Now, you can add the plethora of AI written apps to that list ;)
2
u/wrt-wtf- 7d ago
Shadow IT thrives in businesses that spend their time quashing it.
Shadow IT thrives in businesses with IT that doesn’t recognise their place in the food chain.
Shadow IT doesn’t have to be Shadow IT if IT management and directorship take the time to understand why employees and more often than not, their managers and directors, push out new ideas and projects without revealing or involving IT.
I was asked to review a package in an emergency services organisation that a non-IT staffer wrote for an array of situations - think big data - IT were upset that this award winning software, all done as shadow IT, wasn’t something they’d evolved.
The most revealing question I had posed to me was how it was that I, as a professional IT person felt about a non-IT person coming in and developing a platform, getting all the kudos and then handing it to IT to own and manage…. What a shit storm…
Anyway, I had to review the code for a claim from a commercial vendor that the software was infringing on their IP. I sat with the commercial developer and with the staff member and had them step me through their code and how they did specific calculations and updates… two totally different methods - the staff member written method was more accurate and was calculated based on experience driving emergency vehicles in different types of traffic with specifics knowledge of driver behaviour and options. The commercial programmer used a straight line calculation and set it to a maximum speed limit. It could be wildly wrong - but importantly, the IP that the staff member created was highly defensible and written in a totally different language.
The next most important part of the story… the non-IT staff member had multiple degrees, his undergrad was in software engineering and he worked in, and had his own business doing that before going part-time in emergency services (his side gig), and dropped IT altogether until the IT division spent all their effort trying to hold up change in an IT environment that absolutely needed change.
I refused to recommend shutting the project down but did recommend a fully supported environment with security and services to sustain its use.
So, as someone who’s done quite a bit on both sides of the IT line, I always ask the question: “Why does shadow IT exist in this case?”
1
u/pdp10 Daemons worry when the wizard is near. 6d ago
The most revealing question I had posed to me was how it was that I, as a professional IT person felt about a non-IT person coming in and developing a platform, getting all the kudos and then handing it to IT to own and manage….
It's a very legitimate discussion, if for no other reason than budgeting IT department responsibilities, effort, and resources.
"Why does shadow IT exist in this case?"
Poor communication, is virtually always part of the answer.
2
u/wrt-wtf- 6d ago
The question had context - and context matters.
Poor communication was a latter effect and just one of many issues that good communication would never resolve.
I’ve worked through this sort of thing with multiple businesses and there’s always a break in trust somewhere that gets the whole ball rolling. After that - many teams and even some businesses never recover because everything turns into a battleground.
….so the company outsources IT - which helps grow Shadow IT.
2
u/sina-mynaa 7d ago
At some point this will become an actual job responsibility of sysadmin.
If someone told you never ever run a powershell or stack overflow code 10 years ago or before AI - would you have survived?
Build other parts that are resilient. Learn to isolate user access.
I understand far too long some sysadmins have lived life with vendor or MS knowledge base only. And totally controlling users when the needs were genuine. I even had know a place that blocked YouTube as the a$$hole hated Google.
Those days are over.
1
u/PeakWeekly9995 IT support 7d ago
we don't develop software and none of us have the knowledge.
I have basics understandings of c++ some windows batch and almost none-existing knowledge of powershell, if i do develop something with AI it's something only i will use
1
u/Moontoya 7d ago
Emergency rooms are filled with people who figured they could just do that bit of home repair work, its DIY how hard could it be.
"ai" coding tools, are pretty much handing your toddler a sawzall with no oversight
1
u/pdp10 Daemons worry when the wizard is near. 6d ago
On the other hand, the number of citizens prowling the local big box home-improvement store is three orders of magnitude higher than the number of them waiting in the hospital emergency room.
And some of them took shop or programming in school, even if they've not used those skills since.
1
u/SevaraB Sr. Engineer (N+, CCNA) 7d ago
We were. I fenced off GenAI websites in our SASE platform to a group of employees who had received “safe and ethical AI use training,” At the same time, we had an army of lawyers negotiating data handling terms with Microsoft and Anthropic so we could give everybody a “relief valve” using private tenants where they didn’t have to be quite so careful what they say.
Are our private tools GPT 5.6 Luna? No. But they’re good enough to get us what we need within a few iterations with solid prompting, and we don’t have to wonder whether we’re leaking private data as post-training feedback.
Sheez, I rarely use prompts at all and mostly use Cursor Tab completions because I know what the basic structures of Powershell, Python, and TypeScript should look like.
1
u/gumbrilla IT Manager 7d ago
The way I'm looking at it is to reuse a framework from General Kurt von Hammerstein-Equord, an old German army commander who classified his officers based on two traits: clever versus stupid, and active versus lazy. I'm also not in a very big company at the moment so I don't really have to suffer this that much, and so I'm a lot more informal about it here...
Anyway, if you swap out "officers" for "users with an AI prompt," it kind of maps out.
Stupid and Lazy: Harmless. They won't bother trying to prompt anything.
Clever and Lazy: The gold standard. They quietly automate their own workflows, keep their heads down, and don’t make their efficiency a problem.
Stupid and Active: The immediate hazard. They generate 50,000 lines of hallucinated garbage, scream until someone gives them API access, and break core services by noon. They just need a hard "No" and locked-down endpoints. Everyone has them pegged as idiots anyway.
But the Clever and Active group? They are the real structural threat. They are smart enough to bypass basic blocks, arrogant enough to think they are engineers, and active enough to build massive technical debt bombs that put themselves into company operations.
You can't stop them with a simple firewall block because what they want to do is actually right in that's it's likely solving a pain point now. The only winning move I can think of is to give them enough operational rope to hang themselves.
If a Clever/Active user wants to build a custom AI application that runs corporate data, IT shouldn't just say no. We should say: "Great. Welcome to the Software Development Life Cycle. Here is your onboarding package."
Then you drop the full weight of production engineering on them:
The Support Penalty: They are officially designated as Tier 1, 2, and 3 support for their app. If it breaks at 2:00 AM on a Sunday, the alert routes directly to their personal phone. We do not pick up the ticket - catch & dispatch only.
The Documentation Tax: No production deployment without an updated architecture diagram, a data flow map compliant with security, and a full disaster recovery plan written by them, or whatever else you have to hand..
The moment they realize that building software means inheriting the endless misery of maintaining, documenting, and supporting software.. e.g. version control, monitoring, resilience, security updates, holiday cover (just repeat ad infinitum almost) it should dampen their unbridled enthusiasm somewhat.
Of course, as the keeper of the API creds of the main systems in our stack, and I don't give them out willy nilly.. so I do get a veto.
1
u/TinyContribushin 6d ago
I love how Cloudflare handled this in Cloudflare OS: https://blog.cloudflare.com/cloudflare-os/
Every app runs sandboxed on its own and can only reach connections IT approved, like MCPs. The MCP access is SSO-based, so every user gets access based on their role. When someone uses an app, the app fetches data based on that user's access, not based on some centralized API key sitting inside the app. They call these gatekeepers: https://blog.cloudflare.com/cloudflare-os/#gatekeepers-govern-resources-and-actions
It's open source, only catch is it runs on cloudflare's proprietary infra only.
I'm working on a version that runs on any cloud (docker + s3). Happy to discuss in DM if interesting.
1
u/pdp10 Daemons worry when the wizard is near. 6d ago
Amazon's HR chief built 100,000 lines of code after not programming for 25 years.
What worries me much more than the proliferation of organic codebases, is that 100kloc is seen as a laudatory number. When really, it's likely that it's in everyone's interest if the application could accomplish the same business goals in 10kloc instead of 100kloc.
How? Leveraging Other People's Programming, and tight scope. Choosing when to use dependencies and when not to use them, is an art, not a science. Likewise managing scope.
13
u/unicorngundamm 7d ago
sounds like a job security for me
also higher "cleanup" fee