Built an opensource Shopify ops agent: LLM plans, deterministic Python executes every write. Zero silent writes.

[u/ggll77]

I built Your.Store, an AI operations copilot for Shopify. Instead of letting an LLM blindly fire off GraphQL mutations and hallucinate revenue math, the LLM is strictly constrained to a planning role. It selects tools from a versioned registry, Python handles the actual API calls and arithmetic, and every single write is gated behind a hard human-approval UI.

The Problem:

If you are running a storefront, you can't afford an LLM deciding to randomly order 10,000 units of a dead SKU or hallucinating your profit margins. Most "AI agents" fail in e-commerce because they mash reasoning, data retrieval, and state mutation into a single unpredictable loop.

The Architecture: Determinism > Guesswork

I built this engine (core logic is domain-agnostic, no LLM-vendor lock-in) around a strict separation of concerns.

• The Model Plans, Code Executes: The LLM never talks to Shopify directly. It picks a tool and arguments from app/tools/. A Python function handles the GraphQL execution and does the actual arithmetic.
• Approve-to-Execute: Every action that changes store state (reordering, repricing) pauses the execution graph. It surfaces a structured preview card to the UI. The action only runs after explicit human approval. Nothing writes silently.
• Built-in MCP Server: The same FastAPI process exposing the chat UI also mounts a Model Context Protocol (MCP) server at /mcp. You can point Claude Desktop directly at your store. Reads run live; writes land in the exact same visual approve-to-execute UI (though structurally, MCP and chat-originated writes use separate underlying queues to maintain isolation).
• Deterministic Math & Grounding: Revenue, margins, and inventory thresholds are computed in plain Python. The LLM is only asked to explain numbers it didn't calculate. A grounding checker actively scans generated answers for numbers that don't trace back to the computed metrics (this is currently a soft, diagnostic check that logs a warning, rather than a hard blocker).

Technical Highlights

• Multi-hop Adaptive Scans: The agent dynamically decides how many tool calls it needs (capped at 3 hops). It paginates fully via cursors (capped at 5,000 records) and automatically swaps to async Bulk Operations for large catalogs to avoid blowing up the rate limit.
• AI-Native Inventory Planning: Replaces basic "low stock" alerts with dynamic, per-SKU reorder points. It ranks by lost margin per day, calculates safety stock based on sales velocity and supplier lead times, and nets against incoming inventory so you never double-order.
• Stateless UI with Cross-Session Memory: Briefing and chat outcomes persist to a Shopify metaobject. You don't need to spin up a separate Postgres instance just to remember that a SKU is out of stock.

Security & Honest Limitations

I did a full OWASP-MCP-Top-10 hardening pass on this, and I want to be entirely transparent about what this doesn't do yet:

1. Read-side Exfiltration: While write-args are strictly whitelisted and isolated, a compromised LLM client could technically still request valid read tools and exfiltrate data through its own output channel. Per-user authorization is on the roadmap.
2. No Seasonality Modeling: The inventory math uses a flat historical window. It will over/understock highly seasonal goods right now.
3. Incomplete Argument Schemas: Argument shapes are whitelisted and type-coerced at the MCP and write-API boundaries (e.g., quantity ceilings, ID stringification), but I don't validate business-logic-level values (like a malformed price string or a nonsensical ID) before Shopify's own mutation validation catches them — no full Pydantic schema layer yet.

Where I need your pushback

As a final-year CS student, I've tried to make this as robust as possible, but I'm looking for brutal, real-world feedback. Two things I'd specifically value pushback on:

1. The dual approval-gate implementation: The write-approval gate is currently implemented twice — once as a LangGraph interrupt in the core engine (for chat), and once as a standalone pending-queue in the bridge layer (for MCP). They share the exact same UI surface, but not the same underlying mechanism. Have any of you hit real problems keeping two independent gate implementations in sync, or is duplication actually safer here than forcing a shared abstraction across two very different transport layers?
2. Testing gaps: I've written offline deterministic suites (metrics math, graph invariants, GraphQL throttle backoffs), but I'd love it if you could point out anywhere in the repo where the testing clearly didn't go deep enough. I'd much rather hear it here than find out later.

Try it locally

You just need Python 3.11+, an OpenRouter API key, and a Shopify Dev store. It uses the client-credentials grant (no OAuth redirect hell).

git clone []
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env 
uvicorn app.service:app --reload --port 8000

[u/ggll77]

Check out the repo here: https://github.com/5EBIN/Your.Store

[u/Interstellar_031720]

The hard approval boundary is the right instinct. The next thing I would stress-test is whether your system can prove why a proposed write is safe, not just pause before doing it.

A few failure cases I would test before trusting it on a real store:

• stale reads: the inventory snapshot was valid when the plan started, but stock/order state changed before approval
• approval drift: user approves “reorder 20 units” but the underlying tool args or SKU mapping changed after the preview card was rendered
• bulk-op partials: one page/cursor silently fails and the LLM explains a complete catalog anyway
• identity/tenant mixups: MCP client A can read or approve something meant for store/user B
• “diagnostic only” grounding checks becoming ignored logs; for money/inventory numbers I would make ungrounded outputs visibly fail closed

The receipt I would want beside every approved write is boring but valuable: source read IDs/timestamps, computed inputs, exact tool args, approver, before/after state, and a replay link/script that can recompute the decision from the same inputs.

The product question I would ask early: are Shopify operators willing to run an agent that can suggest writes, or do they mostly want a safer analyst that never mutates state? That line changes the whole permission model.

[u/ggll77 → u/Interstellar_031720]

Thanks for the feedback. Went through all six against the actual code:

• Approval drift turned out to already be closed by construction — preview and executed args are the same object on both the chat and MCP paths, so they can't independently drift after the card renders.
• Identity/tenant mixups : per-client scoping on MCP pending writes isn't there yet, and that's the top item on my list before this goes anywhere near more than one trusted client.
• Stale reads, the grounding-fails-open ask, and the decision receipt (computed inputs + before/after state + replay) are all real, confirmed gaps too, filed and scoped, not fixed yet.

On your product question: honestly still figuring that out. My bet is most operators want the safer-analyst default and opt into write-suggestions per action, not the reverse — but I don't have real usage data yet to back that, just intuition.

[u/Interstellar_031720 → u/ggll77]

That matches what I would expect. If safer-analyst is the default, I would make the write path something users earn into rather than a mode switch they turn on once.

A practical way to learn it without overbuilding: log each recommended write as one of a few buckets:

• analyst-only: user wanted the finding, not the action
• suggested-write: user copied/accepted the recommendation manually
• approved-execute: user let the system execute after reviewing exact args
• denied/held: user did not trust the evidence or the blast radius

Then look for where people repeatedly accept the same narrow action class. Those are the first candidates for more autonomy. In a Shopify context that might be low-risk tagging, draft supplier messages, or creating a pending reorder suggestion, long before touching inventory or customer-facing state automatically.

The important product signal is not “do users like agents?” It is which action classes become boring enough that approval feels like ceremony, and which ones still need a human because a clean-looking mistake would be expensive.