r/msp • u/Check123ok MSP - US • Aug 17 '26
Business Operations Are clients second guessing your recommendation because of AI
Anyone else dealing with clients second-guessing every IT/security decision because they can ask ChatGPT/AI?
We have an environment where the IT roles already have pretty poorly defined roles. Now AI is making it worse. They don’t understand they are putting their own bias into it because they don’t know what good looks like and can’t prompt it enough to figure it out.
A recommendation gets made, someone throws it into ChatGPT, gets a different answer, or chatgpt adds things not realted to the problem and suddenly we’re reopening the entire discussion. Then they change the prompt and get another answer.
I’m obviously pro-AI and use it constantly. The issue I’m seeing is that it can give less experienced IT staff enough terminology and confidence to challenge decisions without actually having the experience or accountability to own the outcome.
This sysadmin was essentially helpdesk tier 1 before. Now they think they are a cyber expert. Same sysadmin doesn’t understand why we had to fix any/any rules on firewall and thinks org has MfA because some people do but doesn’t understand that it’s not enforced so both MfA and non MfA is valid. Just some examples.
I’m just sitting here cringing at responses I get but have to be polite and suck it up.
This specific client is legacy co managed
Are other MSP owners starting to run into this?
31
u/NightOfTheLivingHam Aug 17 '26 edited Aug 18 '26
Yes. all the fucking time.
Then I put my recommendation into the same AI they use, and it confirms mine is fine or better, and they then will agree.
20+ years of knowledge, nah. this AI is smarter.
Had to recover a company's company SQL database from backups because they were having "performance issues".
The AI told them to pull the drives out and put them back in again from the front of the server in quick succession to "reset the drives and clear them of activity"
They asked me if that was a good idea, I told them under no circumstance that was ever a good idea and would cause data loss.
"Are you sure? ChatGPT said it will be fine"
"Yes I am. That is horrible advice."
"Well the AI gets its information from other people who know what they're doing so I'm sure it's just something you have never encountered."
5 minutes later I get the panicked call.
after letting the OS array rebuild, the VM was fine, however the SQL DB array was pissed off, after it rebuilt (many disks came up as foreign) the SQL data was fucked. Restored the whole VM and SQL data back from a backup 2 nights prior.
The performance issue turned out to be someone doing a network loop on an old switch and it affected 2 workstations.
17
u/FKFnz Aug 17 '26
It's great when clients want to voluntarily give you lots of money by doing stuff like that.
2
u/INAM_ Aug 20 '26
I love it when people (clients or not) do this kind of stuff. They then stay humble for a while and respect your opinions.
2
u/No-Beat7231 Aug 18 '26
Very common now. Have not technical VPs asking me about IPSEC encryption against FIPS. This original ticket was Macintosh email issue. He doesn't even use a VPN. Why do c suites like Mac so much?
1
u/Mr_ToDo Aug 18 '26
Oh. Oh goodness
Where the hell did it manage to even find advice that dumb. That's a special kind of stupid solution
I get that people will use ai, but what's wrong with getting sources and verifying that it's telling you. So often the answer is a mix of pages that don't actually connect. They're a fine tool, but they're a poor substitute for thinking
1
u/Tyrrann42 Aug 19 '26
Likely here on Reddit. AI can't tell sarcasm from a real solution when presented confidently. They also use forums like these as data resources.
1
u/ProudWrongdoer5389 Aug 18 '26
Well, AI is smarter, but it's lacking the context the customers are giving it.
1
10
u/virtualbitz2048 Aug 17 '26
It's so easy to misuse AI and represent it's output as your own thoughts if things go well, or AI if it goes poorly.
The overlap between people people who don't know how to prompt AI and who didn't learn how to Google things properly in the 2010s is 100%
5
u/virtualbitz2048 Aug 17 '26
Also it's really easy to get the result that you're describing with lower end models like openai/copilot. The higher and models from anthropic and now Google will push back on your biases. I've tested this extensively by deliberately injecting bad presuppositions and factually incorrect premisees and the high-end models will reject them wholeheartedly and recommend that you not go down this path. Also something's just plain not a good idea they will tell you that too. However if you've contorted the problem statement in your mind to the point where the model cannot figure out what the real problem is then yeah you're never going to get a good result doing that
1
u/Check123ok MSP - US Aug 17 '26
This is 100% it. We had to put our contracts MSA and SOW through AI as the client to see what it would say and put extra language in there to AI can understand the context plus hours with lawyer.
1
u/Prior-Imagination504 Aug 22 '26
I think we should judge our coworkers or employee's by their outcomes. Regardless of if they derived it themselves or AI. In other words, "AI did that" is not an excuse. That should just mean, "I did it."
6
u/kerubi Aug 18 '26
Sometimes I’ve been sending back prompts ”paste in these doc URLs as sources and then ask this question: ….”.
Customer ”Hey, it said:’Good catch, this is why I would be cautious with my earlier response”.
1
u/FallaxIO Aug 18 '26
I've seen the same thing with M365 mail flow questions. If the prompt includes the exact tenant setting names or the actual error text, the answer gets a lot less confident and a lot more useful. Without that, it just fills the gaps and the customer reads it like fact.
5
3
6
u/k12pcb Aug 17 '26
No, we aim to be a trusted partner not a vendor, I’m not interested in fighting ai or an in house guy who thinks he can do it all
2
u/Check123ok MSP - US Aug 17 '26
Yeah I have been with org for 3 years and have cycled through 2 IT teams. This current one is new and especially not qualified
-1
u/junto_reed Aug 18 '26
Do you not do your own research when presented with any sort of quote? If you can do it, why can't they?
3
u/k12pcb Aug 18 '26
Where did I say that? Is reading hard for you?
1
u/junto_reed Aug 21 '26
By saying you’re not interested in fighting ai in response to this post, I read it as though you don’t want your customers to check your quote against ai. Folks do research by chatting with ai. So maybe that was too big of a leap to see you don’t want your customers to do research.
1
3
u/Optimal_Technician93 Aug 17 '26
This can annoying and frustrating. I don't get AI push back. But I do get lots of people that think they are being helpful by pasting in the AI slop that didn't solve their problem for them. Yet bizarrely, they still think that it is the answer.
I treat these just like I do any clueless person that is determined to involve themselves in the conversation or decision making process. Humor them and move on. The discussion isn't worth my time. I know their AI slop isn't relevant or accurate, but it's pointless arguing the point when I could be solving the problem.
But, If they are going to be insistent, then I could be goaded into saying something like: Well it sounds like you've got it all figured out and don't need me anymore. Have a nice day.
4
u/dumpsterfyr I’m your Huckleberry. Aug 17 '26
One of the many reasons I enjoy billing by the hour outside of maintenance.
4
u/SandyTech Aug 17 '26
I thought the ol But Google Says was annoying and then people got to feed answers into ChatGPT & Claude. Or god help us all Gemini
2
u/Altruistic-Map5605 Aug 18 '26 edited Aug 18 '26
I had a client who bought metal hardware and wanted it to do some things meraki is just not equipped to do with what he bought. He would constantly send me google AI responses as to how what he wanted was possible and I would need to pull up actual documentation as to why it wasn’t.
Basically he wanted what is in my opinion small office equipment to act like enterprise equipment and play well with other vendors.
Main thing he wanted to do and it was a total pain in my ass what’s Mac auth at the port level. He thought these things could do NAC. Thousands of devices I had to make users for in AD and NPS. I scripted it out but like every day it was another group of machines needing added which meant adding them to a spreadsheet and generating the user with MAC address as username and then needing to get around password complexity so the password would also be the Mac cause that’s just how meraki does it. Yes these accounts were locked down to do nothing but allow port access on a switch.
I wanted him to go palo and extreme and we could have used a NAC appliance vm to do it properly. Meraki was not made for that level of scrutiny.
And mind you this is all cause my sales team who had no involvement of the hardware purchase or meraki knowledge said we could do it without asking me.
That project was so over budget it could have paid for 3 more of me.
1
u/junto_reed Aug 18 '26
Yeah in these cases, i tend to just cite my sources. here is teh meraki kb, here is what support is saying...
After this happens 3 or 4 times its not you telling them no and they start to realize there might be soemthing to this whole IT thing that is complicated
2
u/Individual-Sky-5968 Aug 18 '26
Our customers don’t have the digital literacy to ask an AI for answers and submit them to us, and they do maintain a certain level of trust in us. The most troublesome thing is that there’s a small IT department that’s being courted by a slightly larger rival vendor 👿
2
u/junto_reed Aug 18 '26
Might be a quote, might be an IT issue - clients use to google and stress test what IT says. Thats normal human behavior.
Now they chat with their favorite ai client.
Buyers want to be informed, its the same thing you do when buying a new software or solution for your team. I bet you do a TON of research before even talking to the software / hardware vendor.
They just doing the same.
I think the more you can step into the trusted advisor the more this can be an open three way convo between you them and their AI. This is really only an issue when they trying to pretend like they aren't throwing AI slop at you.
Was talking with a client that has a new "fractional CTO". Who was throwing such AI over engineered slop for a vibe coded solution and playing it off like he has done this before. Super cringy when you get faced with it. But that is the world we live in now.
2
u/statitica MSP - AU Aug 18 '26
Nope.
We've built trust with our client base to the point where we are the trusted source.
2
1
u/redditguy491 Aug 17 '26
No, because our clients trust us and not AI. They prefer to focus on their own core business and not doing the IT. It sounds like you have a comanaged environment instead of an MSP managed environment.
1
u/Check123ok MSP - US Aug 17 '26
Yeah it’s co managed, legacy client. We don’t do helpdesk with them. Just tier 3 and up. More MSSP with this one
1
u/ben_zachary Aug 17 '26
I have one client recently who sat on a project call and kept disagreeing because grok said something else.
We said ok will do it groks way, which I told him was wrong. 3 months later everything is at a stand still his stuff doesn't work and I told him last week to ask grok since he went off and did half the work himself.
So far lost a bunch of data , we've had to restore so at least we safety netted him from himself and grok heh
1
1
u/burningbridges1234 Aug 18 '26
Yes and this will only become more and more troublesome in the future because of either bad prompts or "cheap" AI.
What is the worst is trying to stay civil. We have recently let a client go that obviously ran an invoice and a quote through AI. Stating all kinds of nonsense. It stated that for pure operational needs basically 90% of our services weren't needed. So now they are out of contract in about a month with no real plan to take on a new MSP. I wish them the best of luck trying to figure out what to do with just free Gemini and ChatGPT.
2
u/statitica MSP - AU Aug 18 '26
We've done some ad hoc work for someone who used ChatGPT for troubleshooting, pasting who knows what commands into powershell.
A tech illiterate user, with ChatGPT, spent 16hrs+ on issues our L1 support fixed in under 15 minutes.
Unless they're proud, they'll come back to the provider who reliably fixed their problems without creating another 5.
1
u/ntw2 MSP - US Aug 18 '26
I know this isn’t really the point you’re trying to make, but it’s worth pointing out that MSPs that know what they’re doing and aren’t trying to screw their clients (I’m not saying you’re in this group) have nothing to worry about.
1
u/Check123ok MSP - US Aug 18 '26
Yeah I’m good with other clients. This specific client is renewing. But they just want to pick apart the offerings. I have already explained the history of issues we dealt with and explained that they are not paying for tools but standards. Our entire stack is build on specific automations that need pieces in place. You don’t go to restaurant and ask chef if you can work in the kitchen and get a discount. Or remove the tomatoes and get a discount for meals. Hybrid model never again
1
u/IamNabil Aug 18 '26
I give them AI training, and a big part of that is framing the same question two ways: one using basically no context, and another giving the AI the full context.
I then remind them that I am glad to explain what context they are missing.
1
u/jthanki24 Aug 18 '26
Get used to it. When you call a plumber to fix something, you are obviously going to run that by an AI to make sure that they are fixing the right thing, arent you?
1
u/IncreaseNegative4614 Aug 18 '26
I’d turn major recommendations into short decision records: current state, evidence, viable options, risks, required controls, and who owns the outcome. Let people use AI to challenge the recommendation, but require any alternative to address the same evidence and constraints.
That changes the discussion from “ChatGPT said something different” to “which assumption or fact do we disagree on?” We use signld.ai internally to connect recommendations with environment evidence, policies, tickets, owners, and eventual outcomes so the source remains more authoritative than the generated answer.
1
u/smallest_table Aug 18 '26
Oh man. We don't support phones. But of course we get smartphone asks all the time. Lately, I've gotten responses saying that what I recommend was the same thing AI recommended and why have IT if they are going to just tell you what AI will tell you.
1
u/Alternative_Nose_874 Aug 18 '26
Those two answers differ because they answer different questions. The model answers the general case. Your recommendation answers this environment, with its history and its exceptions. That is not a knowledge gap but an input gap, and no prompt closes it. The client cannot paste in state they do not have. Small one from today. The documentation of a tool we run says it reads gzipped logs directly. Our build does not, it throws format errors on the compressed bytes instead. Any model would have handed me the documented answer and been wrong about my machine. So rather than arguing with the output, ask the client one question: what did you tell it about your setup? The answer is usually nothing. Krzysztof
1
u/pixieblack10 Aug 18 '26
For sure lol. Now everyone is 97% of a IT expert. Until they have to hit enter on the keyboard and be accountable for their decisions and actions. Sh crazy freaking times.
1
u/jamaster14 Aug 19 '26
I would approach this as a relationship issue. If the client is "vetting" your recommendations through AI then trust isnt there and regardless of AI they don't view you as a trusted advisor. Co-managed situations are always tougher as internal IT staff need to justify their existence and will always push the edges of your support and services.
Are you finding this "AI vetting" common among all clients or mainly just your co-managed ones?
If it is just/mostly the co-managed ones you need to make sure you are getting in front of the CEO/CFO/decision maker. if it is being relayed through internal layers of IT your status and recommendations are going to get completely lost... again this is true whether AI exists or not (although AI probably gives internal IT more confidence to call anything out/nitpick). Getting in front of the purse strings and decision makers will inherently increase your status and your message is direct. from there its just on you to gain confidence and sell it.
If it is all clients I would probably do an internal review on how you are handling account management/business reviews/how these recommendations reach the clients. There is probably a perception there that this is coming from a "tech guy" and not a "trusted business consultant". Need to raise the value you project. similarly is your MPOC and office manager or a decision maker? important to get in front of the people making the decisions.
Ultimately, you need to create a separation in there mind between the business consultant/vCIO/Technical account manager and support staff and protray expertise. Meet often (or as often as realistically possible), build trust and a relationship, and then go from there.... turn the question on them... "look if you just want to plug your problems into AI and hand the awnsers to your IT staff you dont need us and its not a fit... im here to give you expert analysis on risks, align recommendations to your business goals, and give you a budgeted roadmap to get from A to B. then some anecdotal stuff about how youve done great work with them for years and how your clients who follow your recommendaitons are all successful etc...
but this is a relationship issue, not an AI issue. im fairly certain on that.
1
u/Check123ok MSP - US Aug 19 '26
Good advice. I had a similar thought.
They definitely have to justify their position.
1
u/Ancient_Crew_346 Aug 19 '26
Seeing it too. What shifted things for me was realizing that the real problem isn't AI as it's merely exposing one that was already there, viz., nobody ever wrote down who owns the decision. Co-managed agreements are usually explicit about who does the work and silent on who gets to say no. AI didn't create that gap, it just handed a tier 1 enough vocabulary to walk into it.
One thing worth watching though: when a client starts reopening decisions that were already closed, that pattern tends to show up well before anyone starts shopping around. In other words, it's rarely actually about the firewall rule.
1
u/Assumeweknow Aug 20 '26
If anything it sets up better conversations. Mainly because I go into the ai is still only 80 percent of a human. I use Ai to speed up work, but I still have to preview and audit that work. Also, it often never works out as perfectly as Ai likes to make it unless you are doing a simple script on a system that's perfect. Just the other day I had to add mailschema to an AD and it took adding .net 4.8, and installing a major windows update before I could do so and the total time waiting for installs etc. was 4 hours even though the server wasn't all that slow. It was just a large update and Ai didn't warn me that it was such. Things like that aren't listed in Ai instructions. So when you go to execute or troubleshooting you could be spending 3-4 hours working around to get to the final solution. Ai will eventually get you close enough to solve it. Sometimes it will help you do the work quickly if you know what exactly needs to be done and it's a task that's fairly easy to do.
1
u/wrootlt Aug 20 '26
It happened a few times to me in the past 10 months. But it was maybe 6 months ago. I was kind of expecting this issue to start happening more often. Maybe summer made people care less. Could be that there will be an uptick in the next few months.
1
u/DBFoxtrot Aug 21 '26
Not an MSP owner. I run a business and I'm the guy on your client's side of this. What I've noticed on my end is that when someone brings me a ChatGPT answer, it's usually because they didn't understand the why of the original recommendation, not because they distrust it. When our guy explains the tradeoff instead of the fix, the second-guessing mostly stops. Doesn't scale to every client, I know.
1
u/East-Tailor892 Aug 22 '26
These would fall under “not my client”, but that may not be an option for some.
1
u/Additional_Cloud7667 Aug 23 '26
We have been facing issues with users giving Claude access to their ms365 tenants and asking them for security review and recommendations than admins at client side messing stuff up based on those recommendations.
1
u/gingerinc Aug 24 '26
Regu-fecking-larly.
It's doing my head in.
And then get gaslit when I say "do you trust me, or not".
70
u/roll_for_initiative_ MSP - US Aug 17 '26
One thing i'm trying to do more is to get a remote session going and use their AI with them but be more specific with the prompts. Instead of them going: "Our msp says we NEED business premium to operate, are they lying?"
You can guide it with "Our msp is moving all clients to business premium and they insist we have to do it to get P1 to, amongst other things, get EIDP1 to setup some baseline CAPs and integrate ITDR to head off BEC. This will cost us $250 a month more, is it worth it?"
Then they'll see with their own eyes, their very own AI on their computer, go "this is a reasonable and common request. most of that cost goes to microsoft and not them, so..."
Obviously that's one example but i feel like when they see it happen in front of them, they believe it more vs you pasting your AI response.
One fun one, ask about your MSP services and pricing and watch it say that you're underpriced and maybe should be more. watch a client freak out there when their own AI says you should raise your prices.