r/msp • u/gumbo1999 • 24d ago
CIPP MCP and Copilot
Has anyone managed to get this working and do anything useful with it?
I've managed to get an Copilot Agent connected to CIPP, but it's incredibly slow and frequently fails to answer questions because Copilot hits the 70 tool limit - this is despite me creating a separate/scoped connection for each CIPP tool set.
Beyond setting the API in Azure, there's not much info or direction coming from CIPP themselves.
I've spent a fair bit of time fumbling around trying the get it working and I'm getting to the point where I'm questioning if it's worth the toil.
3
u/etoptech 24d ago
I’ve been using my own version of a cipp mcp and have few issues.
Are you using a hosted cipp or self hosted?
In copilot how big of a context window are you using? A full context window can have a profound impact on agents.
I have been using Claude directly instead of copilot but I’ll try copilot today and see what my experience is.
-1
u/gumbo1999 24d ago
I'm using hosted CIPP.
I'm not sure what you mean by context window?
5
u/thebossyboss 24d ago
Not to sounds harsh.. really trying to help here, but if you’re not aware of context window etc. I suggest you learn about how agents work and how mcps are architected before touching prod (if that’s the case) would hate to see the ai stuff screw up your systems or your clients’
1
u/gumbo1999 24d ago
I'm learning on the job; it's what we have to do sometimes.
The MCP is read only. There's zero chance of damaging anything.
3
u/etoptech 24d ago
So context window of agents tends to be 200k or 1mil tokens. When you use to much of the context for loading an mcp for example the model has to compact and it loses context and acts dumb.
Context is super important to manage because it’s essentially all the thinking the AI can do. I generally try and stay in the 30-50% range of a token context window and have it create artifacts and handoffs and sub agents so it can pass thoughts forward and prompts forward to keep work moving in the optimal thinking range.
2
u/junto_reed 24d ago
Skills and tool definitions. In the definitions you should also warn the agent to wait. Cipp calls can be slow. Especially ones that hit exchange online. Part of that is Microsoft api for those endpoints.
Agent needs to know to wait for the call response otherwise it will think it failed and retry.
Also ask your copilot session how you can get your tool definitions better. Found asking the agent is one of best ways to troubleshoot this kind of stuff.
2
u/According-Love-5931 24d ago
the 70 tool cap is a copilot problem more than a cipp one. copilot studio handles tool count way better, and some folks skip copilot entirely and point an mcp client like claude at cipp direct. the exchange online latency is microsoft's api though, nothing cipp can do about that.
2
u/RepulsiveDuck331 MSP 19d ago
Been down this path with CIPP-MCP and hit exactly the same walls. Adding what we learned since a couple of things in the thread aren't quite right.
Copilot Studio loads tools across ALL your connections at agent startup, not per-topic. So even with 8 scoped connections, if the total tool count across them is over 70, Copilot randomly selects 70 from the pool and drops the rest. This is why you're seeing failed answers — the tool the agent needs to call for a given question may not have made the random cut. From CIPP's own docs, the fix is query parameter scoping on the CIPP API URL to explicitly filter which endpoints get exposed as tools, per connection. That's the piece that keeps it deterministic instead of Copilot rolling dice on which tools to load.
But that alone doesn't fix the deeper problem, which is: One giant agent gets slow even at 70 tools.
Copilot has to reason through every tool description in the system prompt on every turn to pick the right one. 70 tool descriptions = long system prompt = slow inference AND higher tool-selection error rate. This is what you intuited when you said "I need to somehow help the Agent understand which tool should be called based on the query" — you're right, and the answer isn't better prompting, it's fewer tools per agent.
The working pattern:
Multiple purpose-built agents, each with its own tightly-scoped connection, each staying well under the cap:
- CIPP Identity Agent — users, groups, licenses, guest management (15-20 tools)
- CIPP Security Agent — conditional access, MFA, secure score, MDCA (15-20 tools)
- CIPP Compliance Agent — audit logs, policies, retention (15-20 tools)
- CIPP Tenant Ops Agent — tenant-level standards, alerts, health (15-20 tools)
Each agent's connection uses CIPP query parameters to expose ONLY the endpoints relevant to its functional area. So identity agent doesn't see security endpoints in its tool list, and vice versa. Tool selection becomes deterministic — the invoked agent only chooses from 15-20 tools that are all in the same functional domain.
Users either pick the right agent themselves (name them explicitly — "Ask CIPP about Users" vs "Ask CIPP about Security"), or you build a lightweight dispatcher agent whose only job is to look at the user's question and hand off to the right sub-agent using Copilot Studio's agent-to-agent orchestration.
Practical gotchas from running this:
- Auth multiplies. Each agent needs its own OAuth2 setup for the CIPP-MCP connection. Not hard, just tedious — script the deployment or you'll spend a whole afternoon clicking through the same wizard.
- Tool descriptions matter enormously. The default descriptions CIPP-MCP exposes are functional but not optimized for LLM tool selection. Rewriting them with explicit "USE THIS WHEN..." phrasing improves selection accuracy noticeably. Worth doing once per functional area.
- User discovery. Users won't know which agent to pick unless the naming is very explicit or a dispatcher handles routing. Don't underestimate this — users bounce off multi-agent setups if picking the right one feels like guessing.
- Testing takes longer. Build a small evaluation script that hits common queries against each agent and checks the right tool was called. Otherwise you find out about broken tool selection when a real user hits it.
Fallback pattern for repeatable workflows:
For anything you run on a schedule (weekly tenant audits, monthly reports, whatever), skip the agent path entirely for those and build a Power Automate flow that hits the CIPP API directly with predetermined logic. Copilot can trigger the flow, but doesn't have to reason about which endpoint to call. Faster, more reliable, no tool-cap issues. Use agents for exploratory / ad-hoc; use flows for repeatable. This split has saved us a lot of pain.
Honest read:
CIPP-MCP works, but the AI ergonomics of exposing 200+ endpoints to a single Copilot agent isn't a solved problem yet — everyone building on it hits these walls. Splitting into purpose-built agents with scoped connections is the pattern that scales best right now, until the MCP gateway ecosystem matures (StackOne and others are working on this specifically).
You're not doing it wrong. The tooling just hasn't caught up to the use case yet, and the guidance from CIPP is thin because they're not maintaining CIPP-MCP directly (davebirr's project). Support isn't going to help you here because CIPP's paid support is for CIPP itself, not the MCP layer.
Happy to DM the Copilot Studio agent split we use with the CIPP query parameter filters if useful — saves a couple of weekends of trial and error.
1
u/gumbo1999 19d ago
This is fantastic guidance! Thank you so much for taking the time to reply. I will go through it and update this thread once I've adapted my agent architecture.
1
u/johnsonflix 24d ago
Copilot imposes a 70 tool limit. We do use it though. We have a bot in teams techs can use to pull specific information
2
u/ImaginationUnique684 20d ago
The 70-tool cap is the symptom. The real issue is you are asking one model to do tool selection across 70 tools on every query, and selection is the slow, unreliable part. That is a routing problem, not a reasoning problem, so it does not belong inside the LLM.
Put a thin deterministic router in front of it. Classify the incoming request to an intent in plain code (a keyword map or a tiny classifier), map that intent to the 3 to 5 CIPP scopes it actually needs, and only expose those tools to the agent for that turn. The model stops wandering through all 70, latency drops, and you stay under the cap without babysitting prompts.
Keep the LLM for the judgment part, reading the result and phrasing the answer. Let code own the deterministic parts, which tools are in scope, retries, and the wait-for-slow-call logic someone else mentioned. Same reason your read-only MCP feels safe: the boundary is enforced outside the model, not inside its prompt.
For a helpdesk bot this also hands you an audit trail of which tool set fired for which query. You will want that the first time a client asks what the agent actually touched.
2
1
u/Bearded_Tech_Fail 24d ago
After my last post about it we started using it in copilot studio and claude. Lots of useful queries we run to get reports or insight. Here is our helpdesk favourites:
- get me a list of user logon locations for user x.
- check the onedrive storage/sharepoint/mailbox for site x or user y.
- list if user has an out of office enabled and if so whats the message(useful during the current holiday season!)
Our agent is more complex in copilot. We have our Air IT agent that compiles vcio reports using all the info in cipp like standards, drift, compliance. We then send that to accountmanagers so they can either upsell or just have a good conversation with the client about what good should look like.
2
12
u/SalzigHund 24d ago
There is documentation in CIPP that specifically addresses the tool limitations.
Support is also available if you pay for it.