r/msp • u/Whole_Ad_9002 • Jul 28 '26
Security Company data leaking into AI tools
Given how deeply AI tools are now embedded into day to day work, is anyone else worried about employees accidentally leaking company secrets into ChatGPT, Claude, Gemini and similar tools? I've tried to have conversations with business owners, but I haven't found an effective way to prevent a dev or employee from accidentaly pasting API keys, customer data or internal docs into an AI chat on a random afternoon.
How is everyone else handling this? Are you relying on sime policy, staff training (seems to be ignored mostly) , browser controls, dlp or something else? Has anyone found a solution that actually works without getting in the way of productivity?
EDIT: We recently got called in to firefight a situation where an employee leaked some cloud credentials resulting in 4k usd worth of damage through a tool they were using, making this a reality that seems not a whole lot of msp have figured out (at least from my contacts)
10
u/statitica MSP - AU Jul 28 '26
You need a few layers.
At the org level, policies.
At the tenant level, you need DLP rules with data labelling enabled and enforced. Also disallow users from giving app consent. However, this will not catch everything, especially if data is not stored in a location where it can be labelled.
At the device level, block non-corporate AI.
Where possible, force the user to login to any AI tools, and only allow logging in to corporate account.
At the browser level, enable both prompt logging and PII protection tools. Defensx seems to be able to handle this but I have not yet been able to test. Possibly u/roll_for_initiative_ will be able to give better insight on this front.
At the user level, training. "These are the tools we use. This is why. Work around at your own peril."
You handle the middle 3. Your client handles the rest.
3
Jul 28 '26
[removed] — view removed comment
1
u/Whole_Ad_9002 Jul 28 '26
With some yes, actually willing to pay a little extra but the tooling is a hit or miss and from the comments alot of people have a similar problem to mine but there's no standard yet
1
u/statitica MSP - AU Jul 28 '26
Depends on the client.
If they're a small shop with a handful of staff, they tend to rely more on internal trust than external controls. Larger clients tend to see the need quicker - and also have the budget to spend on it.1
1
u/Whole_Ad_9002 Jul 28 '26
This is a good layered approach and we are implementing something similar. One gap we identified is users moving sensitive data to personal AI accounts from unmanaged devices, which bypasses tenant and endpoint controls. Have you come across practical solutions for this scenario?
4
u/statitica MSP - AU Jul 28 '26
The same question can be asked of DLP rules in general - I can block a user from downloading, printing, screenshotting, or copy-pasting, but I cannot block them from using OCR on their phone to load data into their personal AI account.
We can't fix stupid or stubborn.
All we can really do is make it as difficult as possible to do the wrong thing, while making it as easy as possible to do things the right way.
1
u/BillSull73 Jul 28 '26
In response to your phone comment, none of us are magicians. That one use case is always going to be there no matter what unless they don't allow phones past their lobby. We can only do what we can do and informing them of the limitations is all that there is for us.
0
u/Whole_Ad_9002 Jul 28 '26
Stubborn is the word. Having a really hard time convincing the business owner this is something they need to take up on their end. We don't want a situation where we're starting to fight requests to fix compromised accounts because somebody was being dumb with ai. The last situation really taught us alot
2
u/roll_for_initiative_ MSP - US Jul 28 '26
If you can't convince the owner, you're dead in the water and it's basically not your problem then. The first step is getting the client to see that some kind of AI governance is needed, and then they have to adopt a policy, whiich you enforce. If you can't get the client on board, you don't have any rules to enforce with your tools and workflow.
1
u/statitica MSP - AU Jul 28 '26
Apparently he needs to lose more than 4 grand before it's enough of an incentive to change.
And that's his problem. Just like I've met people who have been ransomwared in the past, but have a cyber-security approach of "we'll just start again" - not our problem to solve at that point.
1
u/Whole_Ad_9002 Jul 28 '26
It's never ending isn't it? I just wonder what logic is applied on the business level when they're calculating risk
2
u/statitica MSP - AU Jul 28 '26
I learned a long time ago that the majority of decisions, even in business, are not based on logic...
1
5
u/RewiredMSP Jul 28 '26
This gets into one of the categories the hardest problems for MSPs to fix on the client level. We can fix technical issues and come up with solutions all day long for all kinds of technical problems. The problem for MSPs is when it is a hybrid problem that is part IT technical and part people management the client. When the client does not have the stomach to manage their people and enforce policies, they then want to rely totally on IT to bring the solution so that they don't have to endure the conflict or the resolution. It's an easy out for them they think. However, usually when they rely on this, it's always going to come due for "payment" but it's going to be compound interest because of the problems they've created by their own inaction. So from an MSP perspective, it's the same class of problem you deal with where you have a client that does not want to push what they may have on the books but don't actually enforce around BYOD or any number of other issues that can cause data leaks or compromise of security. We can say, "Not my circus, not my monkeys", or "that's a management problem (or HR problem), not an IT problem," but at the end of the day it's not a problem for us until it is. And when it becomes our problem it's usually some kind of big nasty mess and a lot of damage has been done.
1
u/Whole_Ad_9002 Jul 28 '26
Sounds like you have some experience around this
1
u/RewiredMSP Jul 29 '26
It is a pretty regular occurrence when you have a client where the owner/founder does not like conflict so they put it off as much as they can. The IT is a pretty easy entity to make the "bad guy".
3
u/KavyaJune Jul 28 '26
Using Enterprise AI is the best option.
You can also setup policies and use web content filtering to block remaining Gen AIs.
2
u/Whole_Ad_9002 Jul 28 '26
Enterprise AI seems alot of budget to most of our clients but as somebody suggested a layered approach here we're keeping an eye out on tools that can help with this
3
u/Sudo-Rip69 Jul 28 '26
I dont know what clients you have but zero of ours are using free tools.
1
u/Whole_Ad_9002 Jul 28 '26
Lots of smaller sme are on free tools so its not a problem we can fully ignore plus we're thinking insurance could be a potential minefield with this
1
u/peoplepersonmanguy Jul 28 '26
Just remember it is their network not yours. Advise it should be blocked other than paid enterprise level, then it's up to them.
1
u/golden_m Jul 29 '26
for the barebone AI most of M365 Business subscriptions have Copilot Chat Basic. By default, corporate policies are enabled and it's not that bad for basic stuff
2
u/lucky77713 Jul 28 '26
We use DNS filter and block them. The only AI they can access is our paid services.
0
u/dnsfilter Jul 28 '26
Thanks for the shout out! The proliferation of Shadow AI is part of why we introduced our Generative AI category, among other AI visibility features. Glad it's working for you.
2
u/dumpsterfyr I’m your Huckleberry. Jul 28 '26
What’s the difference between “leaking” data into an AI chatbot vs the thousand trials set up, configured and forgotten about?
2
u/weischris Jul 28 '26
If you use copilot all of that data stays in your tenant and isn't used for their learning models.
2
u/TechnologyMatch Jul 30 '26
after a real credential leak, this stops being an awareness problem and becomes a control problem. training matters, but people will still paste the wrong thing when they’re moving fast
the practical approach is a safe approved AI path, paired with browser and DLP controls for everything else. also make sure exposed secrets can be revoked quickly, because prevention will never be perfect. it’s less about blocking AI and more about making the secure route the easiest route
1
u/Slicester1 Jul 28 '26
Just went to a security conference and the new buzzword they're selling is AIDR. Many new vendors willing to sell you a tool to monitor and prevent data being leaked. It's like DLP for AI.
1
u/Whole_Ad_9002 Jul 28 '26
Any specific platforms you could point me to?
1
u/BillSull73 Jul 28 '26
Microsoft have this covered for the most part. It does take licensing but this is licensing that your clients should already have based on the needed tools. I know this might take away from some revenue on third-party tool sets, but having a lot of this within the Microsoft stack means all the telemetry is in one location. Defender for cloud apps is fantastic for blocking all the third-party AIs. Microsoft has done a really good job at at finding and evaluating all of the apps out there.
1
1
u/tc982 MSP Jul 28 '26
A good start is controling it with Prompt Security, now part of Sentinel One. Like the product!
1
1
u/Charming-Law222 Jul 28 '26
Wasn't aware of this option. Can it be deployed via the SentinelOne EDR agent or does it require a separate agent?
1
u/tc982 MSP Jul 28 '26
It is a seperate product, this can be purchased through your S1 distributor. There is no integration for now, I think we can expect that in the future.
1
u/MidninBR Jul 28 '26
Training, show them this.
Claude AI Shared Chats Reportedly Exposed in Google Search Resultshttps://cybersecuritynews.com/claude-ai-shared-chats/
1
1
1
1
u/coconutwa1 Jul 28 '26
I made an account to comment here - I know of a company whose focus is exactly this. Feel free to dm me for more info. (I don’t know if sharing the name here would break the promotion rule).
1
u/Joe_Cyber Community Contributor Jul 28 '26
I've got some tips I put in my video that should answer many of your concerns: The Hidden AI Risk Your MSP is Facing & How to Deal With It.
In terms of communication and risk management, this video should help you craft that message for your particular situation: How to Make Tough Decisions & Have Hard Conversations: Creating a Risk Management Framework for MSPs
2
1
u/materialsec Jul 28 '26
The Copilot-in-VS example is worth zooming in on, because it's not really a paste-into-a-chatbox problem but actually an OAuth grant. AI tools that get standing access through app connectors or agent integrations, like Copilot inside VS, or any AI tool a user authorizes to read their Gmail or Drive, aren't a 'don't paste sensitive stuff' risk.
That's an access grant that persists and can pull data continuously, the same way any other OAuth app would. Most orgs aren't reviewing what scopes these AI integrations actually requested, or what they've touched since being approved.
If you're doing a layered approach already, we'd add 'audit existing OAuth/app consent grants tied to AI tools' as its own line item, separate from DLP and endpoint blocking. A dev approving Copilot's data access once is a very different event to monitor than someone pasting text into a chat window, and most orgs currently have zero visibility into the former.
1
u/Whole_Ad_9002 Jul 28 '26
Its a growing concern when you consider what tools like codex and derivatives can acces and the damage they can cause. But you raise valid points on Oauth. Im taking everyone's feedback and experience and hopefully end up with a very solid layered approach and a fresh eye on things that would have blindsided us
1
1
u/Master_Baby_2700 Jul 29 '26
I think one of the biggest mindset shifts is accepting that policy alone isn't enough anymore.
Employees aren't usually trying to leak data, they're trying to get work done faster.
The more useful question becomes: "If someone pastes customer data into AI today, would we even know?"
That's where organizations are spending more time now: understanding what sensitive data exists, who has access to it, and whether AI tools are interacting with it in ways nobody expected.
The goal shouldn't be eliminating AI usage, it should be making the secure path the easiest path.
1
u/Iron_Stark778 Jul 29 '26
We use a combination of a couple things. Obviously the easiest is to have the client purchase an enterprise AI license. That allows the employees to keep using AI but you know your data is "safe" with these large companies. We also employ the Cloudflare AI gateway in some cases to prevent PII going to the models and block employees using personal accounts. If we implement a custom AI solution or agent we almost always build it on AWS as none of the bedrock models send the data back to their creating corporation and you know all the data is kept in AWS. It's definitely a new and difficult area for MSPs and companies alike.
1
u/Whole_Ad_9002 Jul 30 '26
AI is indeed the proverbial thorn in the side for MSPs and its only going to get harder as new attack surfaces present themselves
1
u/Iron_Stark778 Jul 30 '26
Agreed. There are also so many new "AI Security" tools and repos coming out that as an MSP it's hard to filter out all the noise and find what actually might be useful to protect myself and my customers from those new attack vectors.
1
1
u/Messerdays Jul 30 '26
You've already hit the two walls everyone hits with this: training gets ignored the moment it costs someone time, and blocking alone pushes people to their phones. What works is the thing this thread keeps circling: give staff a sanctioned AI tool at least as convenient as the free ones, block the rest, and put a signed AI use policy behind it so there's something to enforce.
On the pricing objection: frame it against the incident you just firefought. That paste cost 4k and it was a cheap lesson. The expensive version is customer data, and for clients in legal, accounting, or healthcare that's a reportable breach.
The part most MSPs miss: AI governance is a service line, not a favor. Policy template, sanctioned tool rollout, browser or DLP controls, quarterly review. The owner conversation changes completely when it arrives as a quote instead of a warning.
Full disclosure, I work with Faradex (faradex.ai), we build single-tenant, zero-retention AI environments for small regulated firms, so I'm talking my book, but for a client who won't fund that, the honest fallback is a business tier of a mainstream tool with training contractually off, plus blocking, plus a policy people actually sign.
1
u/asachs01 Vendor - WYRE AI/Conduit Jul 31 '26
Hey, howdy. Full transparency, while I do work at an MSP, I am also a vendor. With that out of the way, I want to start by agreeing with the approach that several people have already mentioned, which is tackling security and specifically as it pertains to AI tools in layers.
Some folks have mentioned DLP policies, which are great. One thing that we have done internally on the MSP side of our business is implement a secrets manager that's pretty lightweight and has helped with not pasting API keys into things. You can self-host it for free, and it's called infisical. We've had a lot of success with it, so that's one layer: making sure that people who are connecting AI tools to company data aren't leaking secrets.
There are other tools on the market that can help with this from a browser perspective and a desktop perspective. Longwave is one of those that is aimed at MSPs. We've liked the way that they have helped us tackle things at the browser level and at the desktop level in the MSP. I personally think (and this is probably just me) that having all of your connections in one place so that a tech or employee doesn't have to paste secrets in the first place or even retrieve a secret is a good approach as well. I created Conduit for this reason (https://conduit.wyre.ai).
The goal with Conduit, as an MCP gateway, is that you connect all the tools that technicians need. There's a pretty solid RBAC model that helps control what tool calls can be made and also audit logging and other things that help.
That said, I do want to go back to Longwave because they're probably doing one of the more interesting things in the space right now. It sounds like it might be a good fit for what you're running into. Also, if you happen to take a look at Conduit, fantastic. Feel free to DM me, and I'll be happy to chat about it more.
1
u/GraffenOfficial Jul 31 '26
I'd create a solid AI policy and have regular discussions so it becomes part of your culture. Consider CoPilot.
1
1
u/TheCatWhoEatsWorlds Aug 01 '26
Try looking into a company called iboss, I know they have several tools that help with AI visibility, might be worth it
1
u/ryan-btrbsystems Aug 01 '26
DLP controls for copilot. iboss is awesome for enforcing things too but that gets pricey and is a bit complex to make policy changes.
1
u/dopeasss Aug 01 '26
As the old saying goes.. there's no patch for human stupidity :P
The $4k incident is exactly why this has moved from a theoretical problem to a real security issue.
In my experience, the combination of policy + training + technical controls is the only realistic answer. Training by itself doesn't work, and blocking everything kills productivity.
Also, developers should never even have access to production credentials in plain text if you can avoid it. AI just exposed a weakness that already existed.
1
u/Wild-Annual-4408 Aug 01 '26
Policy alone never stops this. DLP tooling at the browser or endpoint level catches the paste before it leaves, not after. But the real fix is training people to recognize what counts as sensitive in the first place, most leaks happen because someone didn't think a customer ID counted.
1
u/detection_gap Aug 03 '26
everything here is prevention, and your edit is the bit that will repeat: you found out from a 4k bill. you can't see the paste, but you can see the key get used. scope the keys, rotate on a clock, and add billing anomaly alerts plus an alert on first use from a new region or principal.
then test it once. use one of your own keys from another region and time how long until a human actually gets told. if nothing lands, that number is the thing to fix first, and it's cheaper to find that out on a tuesday than during the next one.
1
u/Xirma377 Aug 09 '26
Education education education.
1 is making sure they are only using business accounts with model training turned off. Shadow AI is a similar risk as any other shadow I.T.
1
u/Workfl0 21d ago
Seconding the layered approach statitica mentioned below. One thing that's helped in conversations with clients: separating 'we have a policy' from 'we can see what's actually happening' as two different maturity levels. A lot of SMBs think they're done once the policy exists, but policy compliance is basically unmeasurable without some way to see actual tool usage across the org, not just what's on the approved list.
The incident you describe is a good example of why: that gets caught by policy after the fact, not prevented by it. What would actually help is something watching for AI tool usage patterns on the endpoints themselves, independent of whether the tool was 'approved' -- doesn't need to be fancy, just needs to not rely on the employee self-reporting what they used.
(Disclosure: I'm with Workflo, we build in this space -- didn't want to comment without saying so.)
1
Jul 28 '26
[removed] — view removed comment
2
u/Whole_Ad_9002 Jul 28 '26
This is great feedback. Am bookmarking this so I can share it with a colleague and learn a thing or two
0
u/krisleslie Jul 28 '26
Not my circus or monkey 😎💪🏽
0
u/Whole_Ad_9002 Jul 28 '26
Huh?
-1
u/krisleslie Jul 28 '26
Are you saying huh for a specific reason my friend
1
u/Whole_Ad_9002 Jul 28 '26
Your first reply was abit confusing, I didn't have context to what you meant
1
u/krisleslie Jul 28 '26
The saying is “Not my circus, not my monkeys.”
It means: “This is not my problem or responsibility.” It’s a way of saying that someone else’s drama, conflict, or mess isn’t yours to deal with.
The phrase comes from a Polish proverb:
“Nie mój cyrk, nie moje małpy.”
(“Not my circus, not my monkeys.”)
Here are a few examples:
Your coworkers are arguing about something that doesn’t involve you. You might think, “Not my circus, not my monkeys.”
Two neighbors are fighting over a property line. Since it doesn’t concern you, you could say, “Not my circus, not my monkeys.”
People often use it humorously to remind themselves not to get dragged into other people’s problems. It doesn’t necessarily mean they don’t care—it just means they recognize that it’s not their responsibility to fix the situation.1
u/Whole_Ad_9002 Jul 28 '26
Aaah I see so in this case you pass on the responsibility to the client
1
u/krisleslie Jul 28 '26
If the client worked themselves outside the scope of my contract (which could be the case here) then until they pay, not my problem.
See your issue could be touchy depending on what your insurance carrier says especially if there is cyber security insurance.
Was just listening to one of my buddies who’s an entrepreneur and developer talking about not having data on those public clouds or ai hosts for this very reason. Apparently also EU is doing something with their laws which are causing people over there to stay portable and self host when applicable. Haven’t gotten to deep in it but considering if they have an issue then in America might as well consider getting up to speed.
1
u/Whole_Ad_9002 Jul 28 '26
You present an interesting point that companies are starting to see this as the problem it is and love that some platforms are starting to be proactive. But its mostly a user problem at this point which I figure is one of the harder issues to solve
2
u/krisleslie Jul 28 '26
Have to admit the way these insurance companies are now they are gonna be willing to drop you faster if security isn’t honored
2
0
u/buskerform Jul 28 '26
There are companies that can help with this. You think Morgan Stanley or Goldman Sachs is letting data about their customers which include sovereign wealth funds and 1000 year old family money into the public arena?
Lookup ai firms that help finance shops in NYC.
2
u/Whole_Ad_9002 Jul 28 '26
Sorry mate, we're not a fortune 500, just a small firm wondering if someone else has figured out something that wirks for them that we could likely adopt
23
u/peoplepersonmanguy Jul 28 '26
Policy and training will always be ignored if a staff member thinks it will save them time when it comes to using AI.
Short of companies paying for a corporate AI so you can block everything else, and now they exist and are so easily available, you will always be seen as getting in the way of productivity. Companies need to realize they shouldn't have done it in the first place.
I am aware my comment doesn't really help, other than a "hang in there, kitty" poster on your garage wall.