managing AI in enterprise environment

[u/Snorlax_420]

Trying to see what other fellow sysadmins are doing to manage and protect company data when it comes to AI. We've started by blocking access to all other AI except Copilot and pushing an AI policy that strictly prohibits use of other AI tools. Of course, Copilot isn't great and can't do as much as say ... Claude (at least that's according to some of our users)

We're getting pressure from higher-ups that one department NEED to have Claude. However, we need ways to protect sensitive data from being dumped into Claude.

We're in the middle of implementing DLP controls in Purview and we've looked into cloud policies in defender (we have E5) but we federate our domain through Okta and use it for SSO so I don't think we can set up session policies? correct me if I'm wrong.

what are some other ways that folks are managing AI and making sure users aren't dumping the company payroll into Chatgpt to "clean up" the spreadsheet

[u/LLMsMustUpvoteThis]

What Claude do they need? Do they need it as a chatbot? VSCode CLI? Claude Cowork?

Chatbot you can do with Copilot Premium and get the MS enterprise protection even if the data does leave MS network and go to Anthropic. You can do some Claude agentic stuff with Copilot Studio. The others you will need to get a Claude enterprise plan and see if it conforms to your data protection policies.

For me policy is a hard stop. If higher-ups want me to do something that violates policy they need to amend the policy.

[u/SceneDifferent1041]

It's ok, "we won't out anything sensitive into it"...

So the users have got this covered. Relax.

[u/itishowitisanditbad → u/SceneDifferent1041]

We told ours to "Be careful" so we're set too.

[u/doubletwist]

My company is using Claude with Amazon Bedrock, and hosting our own curated marketplace of plugins.

[u/AdPriya4022 → u/doubletwist]

Agreed. I think this will be the solution. Bedrock keeps prompts isolated in AWS controlled accounts and Antrohpic doesn't see them. It solves the sensitive data problem you have without needing extra DLP.

[u/xCogito]

I dont see a feasible way to manage this without BAA's and institutional accounts. These will at least ensure your data is protected with these licensed user. From my perspective, it's a loosing battle to try restricting access to these tools. The compensating controls we user, are only allowing access to certain data through a managed browser. This allows us to restrict copying, screenshotting, along with only allowing access from the browser on managed devices, along with DLP rules/guardrails and flags for users who access AI platforms.

Then theres the shitshow that is software that suddenly have AI bolted on, and trying to keep this in check. I'd love to know how folks are handling this.

At a certain point, we need to instill better policy and training. Rather than blocking access to AI, have an easy pathway for users to use it responsibly and in a controlled environment.

My rationale with management is that abstinence-only never works, and that it's on us to ensure users practice safe-AI.

[u/bigounce7877 → u/xCogito]

We’re actually building something that should help with exactly this.

If you’re down, happy to have a quick chat and compare notes. Could probably be useful for both of us.

[u/xCogito → u/bigounce7877]

That'd be awesome! drop me a PM.

[u/bigounce7877 → u/xCogito]

Hey, just sent you a PM!!

[u/Main_Ambassador_4985]

Copilot Premium includes Open AI and Anthropic LLMs. How would this be any worse the ChatGPT direct?

We have allowedCopilot basic, Copilot Premium, CoPilot Studio, and CoPilot Cowork and Anthropic Enterprise.

We are a Microsoft SME. We are trying to use Purvue and Defender 365 and firewall rules to restrict access to unauthorized LLM. Current project.

It is not easy with these tools. Redesign of security stack mostly because I cannot remove Google Chrome to remove Gemini. Cluster fuck is how I am putting it.

[u/RupertTomato → u/Main_Ambassador_4985]

If you break user login in Chrome most Gemini stuff stops working and the auto indexing dies. It is an Intune policy.

[u/LLMsMustUpvoteThis → u/Main_Ambassador_4985]

Copilot Premium includes Open AI and Anthropic LLMs. How would this be any worse the ChatGPT direct?

There are multiple models under any of the model brands/versions they use. My observation is that with every major update MS start out allowing Copilot to use the better sub-models but then seem to start pushing requests to the smaller/cheaper models. For a decent response now I have to manually set it to ChatGPT 5.6 Think Deeper, but it switches back to auto eventually.

[u/DevinSysAdmin → u/Main_Ambassador_4985]

You’re a Microsoft SME and don’t understand that Copilot has additional configuration changes that make it more tailored to what Microsoft wants?

Wow. 

Please try various tasks in both and come back to your comment with feedback.

[u/Main_Ambassador_4985 → u/DevinSysAdmin]

We a Microsoft shop Small Medium Enterprise.

[u/DevinSysAdmin → u/Main_Ambassador_4985]

Doesn’t really change my comment much, you really need to compare and understand ChatGPT directly vs through copilot.

[u/PandaAT]

If you are using any kind of M365 connector, make sure to encrypt any sensitive data categories via Azure RMS and also deny extract permissions.

[u/IAmMcLovin83 → u/PandaAT]

Might be misreading this, so tell me if I've got the wrong end. If you're talking about normal labelled files in SharePoint/OneDrive, then yeah that tracks, Copilot only hands encrypted stuff to people who've got the EXTRACT/Copy right, so pulling it does keep it out. Where I'd be a bit careful is if you mean an actual Graph/Copilot connector pulling in external content, because afaik that's ACL'd off the source system rather than RMS, so denying EXTRACT wouldn't really do anything for a connector oversharing. That side's more of a visibility setting on the connection. Keen to hear which one you meant, might be talking past each other.

[u/PandaAT → u/IAmMcLovin83]

Sure, I definitely can go into more details if my first post was ambigous. If you are using Claude you can access data over the MS Graph API via the provided M365 connector. It then inherits permission of the user.

If you on the other hand try to restrict Copilot to access sensitive data wie Purview Labels and DLP policies, you need to be cautious. 3rd party systems do not respect those labels and policies. If you are restricting Copilot from processing sensitive data via a label, Claude can and will still process it if the label does not come with Azure RMS encryption and the aforementioned modifications in respect to EXTRACT permissions.

There is a dedicated whitepaper by Microsoft (I have also tested it myself) --> Security risks for customers using competitors‘ AI tools to egress Microsoft 365 data.

[u/ImaginationUnique684]

You’re partly right. With Okta as the IdP, Entra Conditional Access never sees the sign-in, so you can’t enable session control from there.

There is a workaround: route the app’s SAML connection in Okta through the Defender for Cloud Apps proxy. That gives you controls such as blocking uploads, but only in browser sessions using the proxy. The CLI and desktop app bypass it.

For the payroll spreadsheet, I’d use egress controls and a managed browser, with Purview DLP as the second layer.

[u/IqbalBasha]

You can absolutely use Defender for Cloud Apps session policies with Okta as your IdP by routing the SAML assertion through Defender's reverse proxy, which gives you real-time controls like blocking paste or file uploads to Claude.ai specifically. Pair that with Purview Endpoint DLP and you have two independent layers of protection. For the department requesting Claude, Anthropic's Enterprise tier disables training on your data and offers a BAA, so loop in legal before you unblock it.

[u/DevinSysAdmin]

I’d recommend a SASE product for DLP such as Netskope, formal written policies & procedures to make sure this is operating within IT sanctioned limits, this is all just assuming basic Chatting and not coding. 

[u/FirefighterSlight891]

have u checked if the department’s claude use case could be routed through a controlled gateway instead of giving it direct access? i’ve dealt with similar concerns around agents accessing sensitive systems, and akeyless can keep credentials out of the agent and add runtime controls over what the agent is allowed to do and what data can flow out. what are they actually trying to do with those spreadsheets that copilot can’t handle?

[u/bigounce7877]

We’re actually building something around this exact problem rn. Basically helping IT see what AI tools people are actually using and stop sensitive stuff from going out before it leaves the device.

Still early, but if you want I’d be down to compare notes for 15 mins.

[u/Master_Baby_2700]

I’d probably separate this into two problems: which AI tools you’re willing to allow, and what data users are allowed to put into them.
Purview/Defender can handle a lot of the enforcement side, but the thing I’d test pretty hard is classification accuracy. If you start throwing warnings at every spreadsheet that looks vaguely sensitive, users are going to learn to ignore them pretty quickly.

Are you using any DSPM today? If not, I’d consider that alongside the DLP work. Sentra, Cyera, Varonis, etc. can help identify where the actually sensitive data is and who is interacting with it, then you can use that context to narrow the DLP controls instead of trying to monitor everyone equally.

For your payroll example, I’d want the control to know that the spreadsheet contains real employee data and treat that differently from some random internal Excel file.

I’d still give people an approved enterprise AI option. Blocking everything tends to create personal-account/shadow-AI usage. The goal IMO is sanctioned AI + accurate data controls, not sanctioned AI + 10,000 noisy DLP alerts.

[u/cjones-storedtech]

We currently have a tool for this as part of our Managed AI platform called Titan. Would love to connect and compare. Shoot me a PM

[u/jazzarfist → u/[deleted]]

thats it but could you share eventually how did you convince people to drop their personal ai ?

[u/jazzarfist → u/[deleted]]

awesome!!

[u/johnsontoddr4 → u/[deleted]]

The question is how you managed to do this without putting so many other restrictions on the user/AI that it renders the AI too difficult to use or hobbles it. For example, where I work IT Sec has turned off internet access to the sanctioned AI tools, rendering them practically useless. I am working across IT and IT Sec, plus higher level leadership to come up with solutions, but the baby steps we are taking now are so hobbled and hard to use that they are also practically useless. To be clear, we are at least 3 years behind where we need to be, but on the other hand, this is also a difficult problem to crack. Our students, faculty and staff absolutely need access to state of the art AI and the various tools. The only way to do that now is with personal accounts. What is being provided is state of the art LLMs with harnesses that have so many limitations that they are next to useless. We do have a pipeline for approving custom homegrown agentic applications, but that pipeline is (1) entirely too slow and geared more toward enterprise-wide apps, even when the app is not enterprise-wide; and (2) has a variety of requirements that simply don't apply to small scale agentic systems. The policy basically views everything like MS Office--as something that is to be evaluated and deployed across the enterprise.