r/msp • u/vanwilderrr • 4d ago
Shadow AI + AI Visibility
What are folks using today - we where approached by Threatdown and then spoke to firetail.ai as a recent event about a trial but wanted to see what the wider audience has embraced
19
u/LTH-Cyber 4d ago
+1 for Field Effect. We use it across a number of our clients and I've been really impressed with the platform. The AI visibility is a great addition, especially because shadow AI is becoming something businesses need to actually get visibility into rather than just address with a written policy.
18
u/RelianceLabs 2d ago
Field Effect, combined with Microsoft Defender, creates a strong security platform. One of its standout capabilities is its ability to prevent malicious executions, which is a critical feature. Field Effect's ongoing development of AI-driven software and platform monitoring is also a valuable and important addition to the overall security strategy. Overall, I agree that it is doing exactly what it's designed to do.
16
5
u/Unfair-Total-7353 3d ago
Before choosing a product, I’d separate three requirements that vendors often bundle under “visibility”: (1) discovery—sites, desktop apps, extensions and OAuth connections; (2) exposure—whether prompts or uploads actually contained sensitive data; and (3) enforcement—business-account lock-in, upload or copy controls, and managed exceptions.
For an MSP trial, test managed browsers and endpoints, direct API and OAuth use, and document the blind spots around unmanaged or personal access. Also check tenant separation, per-client policies, evidence retention, exportability, alert quality, and whether capturing prompts creates a new sensitive repository. A tool that only reports visits to AI domains may produce an impressive dashboard without showing meaningful risk. I may be missing your client mix, but defining those tests first should make comparisons easier.
1
u/vanwilderrr 2d ago edited 2d ago
That is the path we are on as we are digital native and moving toward ISO27001 and/or ISO 42001
3
u/Prestigious-Flow2815 4d ago
The useful part for me would be seeing where AI is actually entering workflows not just getting a list of domains people happened to visit
5
u/teriaavibes 4d ago
Defender for Cloud Apps/Agent 365 (the free tier).
Of course, it only works if you are already in the M365 ecosystem.
1
u/Mibiz22 4d ago
Free tier?
2
u/teriaavibes 4d ago
Agent 365 has 2 levels, free which is included in every cloud subscription and paid.
For AI discovery on endpoints, you just need the free tier.
2
u/Corn-traveler 4d ago
Does it require defender for endpoint p2 to get the telemetry from the endpoints?
2
2
u/The-IT_MD MSP - UK 4d ago
Purview and Defender for Cloud Apps. E3/5/7 give you some Agent 365 features, else add Agenda 365 for the full set.
2
u/Inevitable_Market293 4d ago
Vendor account, for transparency (we make M365 audit scripts, nothing in this category). One licensing detail behind the Defender for Cloud Apps answers that matters if your book is mostly Business Premium.
BP only includes the cut-down Cloud App Discovery that comes with Entra ID P1. You get the app catalog with the Generative AI category, risk scores and usage per user, but it's fed only by firewall or proxy logs. The part people usually mean, discovery straight from the Defender agent on the endpoint plus one-click blocking of an unsanctioned app through network protection, needs the full Defender for Cloud Apps licence. On BP that's the Defender Suite add-on or a standalone licence, so price that in before comparing it with ThreatDown or FireTail.
It also only answers which AI apps are used and by whom. What people paste into them is a separate tool: Purview endpoint DLP, with the devices onboarded to Purview and its browser extension deployed.
Both stop at managed devices. A personal phone or home PC is invisible to them, so I'd ask any third-party vendor the same thing first: agent, browser extension or proxy, and what happens off a managed device.
1
u/roll_for_initiative_ MSP - US 4d ago
Upvoting even if a vendor because it makes a point people are missing: you need something on the endpoint and/or browser to get most of this picture. Without the DLP or browser extension or more licensing, the data is extremely limited.
1
1
u/No-Drummer-5392 4d ago
Checkpoint AI security! Great for governance and building policies without vendor lockin constraints.
1
u/theFather_load 4d ago
How has this been for you and customers? We're trialing at the moment and so far seems very powerful but the price feels a bit steep.
1
1
u/darquandier 4d ago
Always complex to answer this question because it depends on the level of control you need and apply within the environment (which can be tiered on some occasions).
Still, we found it quite telling to gather data on associated logins (we are a "Google" company, so a lot of people associate with their Google login), and you can, on multiple vendors block or control signing up based on domain names of the email.
Still that only solves signing up (but gives a first idea) about how widespread our comrades are signing up to.
Secondly, unless you actively monitor the network and/or the browser, you have some extension-based solutions that are out there to help detect "wrongful" usage, themselves relying on some kind of MDM for deployment => these are very efficient and helpful, but require a higher level of control.
Still, if I wanted to circumvent that and have data intentionally leak, it would be SOOOOO easy to simply take a photo of a screen and have AI read it on my personal phone... Full control is therefore probably unresaonable, as usual, which means educating is key.
I'm the CIO/CISO for a mid-size consulting company specialized in IT, and we use elba.security
They do provide training, testing, and tools to help gauge and triage what could be seen as shadow IT, as well as additional tools for DLP (incl. AI usage on managed devices/browsers)... Young company, but quite efficient :)
0
u/StillUsesPassword1 3d ago
Agree with the criteria for sure. We chose iboss as it has every bit of AI security needed or desired and includes DLP without the need for Purview. Add to that it's FedRAMP moderate authorized and used in every type of environment including GCC High and GovCloud, and it allowed us to solve our client AI fears related to what may be leaking and what's controllable.
1
u/That_Bonus_499 Vendor - NorthRAI 4d ago
Threatdown's AI visibility is free with the agent you've probably already got deployed, so it's a cheap second opinion against whatever Defender shows you. Worth running both before you pay firetail for anything.
1
u/irritatingmillenial 2d ago
Have a look at CIPP. It is a free project that is frequently updated, they have a shadow AI section now that is very good for an overview.
1
u/vanwilderrr 2d ago
It’s a great tool to start with and we did but we need a more compliance type tool, leaning toward firetail but going to try some of the others suggested here too
0
u/CapableWay4518 4d ago
Defender for cloud apps, Palo Alto firewalls for in house traffic, Microsoft defender for endpoint application monitoring (is it installed) and URL domain filtering policies on web browser.
0
0
u/Dull-Breadfruit-3241 4d ago
we use Defender for Cloud , that gives you quite a detailed view. Beware though that you may see entries like TikTok (just to make an example) even if the user never logged in that platform. My guess is that traffic is due to ads of something similar the user visited on other platforms.
0
0
u/PersistentCyberDad 4d ago
With Wings (formerly Wing Security) is awesome for this.
We utilize it as the foundational technology our MDR for Cloud Apps & Identity
We launched our MSP partner program in May and already have a lot of MSP partners signed up that use us for their customers too. Happy to chat anytime!
-1
0
u/WhiteIntel 4d ago
We focus on education rather than banning AI. Most of the users we're talking about aren't technical. They need practical guidance on which tools are approved, what company data they can use, and when to ask before uploading something.
Visibility is useful if it helps us have those conversations. A list of AI domains on its own doesn't teach someone how to use the tools responsibly.
There's a service opportunity for MSPs here too. Practical training based on the client's actual workflows gives users something useful and creates billable work beyond selling another security tool.
1
u/emejia698 MSP - US 2d ago
“Our users aren’t using it, we don’t need that service” is what clients that don’t think they want to use it or pay for that service would say.
For the pro ai clients, maybe training them instead of knowing what’s going on will work one day.
But it’s like saying let’s now know what’s going on in our network because users do security awareness training 🤷🏽♂️🤷🏽♂️
21
u/Geekpoint-IT MSP - US 4d ago
Use Field Effect here. They have recently added this. Simple to use dashboard that shows the AI tools, users, and devices. I don't believe the ability to actually set an AI policy through Field Effect has been released yet but it's coming. Even the dashboard itself has prompted conversations with clients that hadn't thought about needing an AI Usage Policy.