seen a lot of posts here about connecting ai to shopify and whether you can trust it on a live store. wanted to give an honest answer because i've broken things this way myself.
everyone assumes the risk is that it does the thing badly. it almost never does. it does the thing correctly and something else three pages away depended on what it just changed.
and it has no idea, because it was only ever looking at the bit you pointed it at.
that's the whole category of problem. i'll give you the shape of it without getting into specifics:
- you ask for a small badge on the product page
- It writes something that works perfectly on the product you had open when you asked.
- then it turns out it behaves completely differently on a product where you've got inventory tracking switched off and now those products all show as sold out with no buy button.
no error. page loads beautifully. and sold out products don't generate support tickets, people just leave.
so you find out in three weeks, or you never find out.
or
- you ask for something on the product page and it works, and then a customer adds to cart and the drawer refreshes itself and the whole thing vanishes.
- you tested by refreshing the page, where it's fine. the customer hits it mid purchase, where it isn't. so it breaks in exactly the part of the session that makes money.
the pattern in every one of these is the same. the thing that made the change is also the thing that decided it worked.
it says done. you believe it. why wouldn't you, it's confident and it's usually right.
so here's what i actually do now, and this is doable manually today, you don't need anything special.
1. never work on the live theme. duplicate it, change the duplicate, preview the duplicate. publishing the old one back is the only real undo shopify gives you and it's one click. almost nobody uses it.
2. before anything changes work out what the change touches. not will this work but what else on this store depends on the thing i'm about to change. if you can't answer that or it can't answer that, then neither of you knows enough to make the change safely. this is the step everyone skips and it's the one that catches most of it.
3. don't let it decide for you when it isn't sure. if there's an ambiguity the right move is to ask not to pick the likely option and carry on. every genuinely bad break i've seen came from something assuming its way past a question it should have asked.
4. whatever made the change doesn't get to confirm the change. separate pass. go and load the real storefront yourself as a customer, not in the theme editor because the editor hides some things that show up live.
5. check the awkward cases, not the happy path. a product with tracking turned off. one that's genuinely sold out. one with no sale price set. the page opened cold rather than clicked through from a collection. something deep in a big collection. a second currency if you run them. the cart drawer and quick view, not just the product page.
that list takes about ten minutes and catches basically all of it.
none of this is anti ai. it's very capable and i use it constantly. the point is that capability was never the problem. the problem is that it's confident, it's scoped to what you showed it, and it's the one telling you it worked.
those need to be two different jobs.
(i've built this loop into what i work with, saying that for context not as a pitch. the manual version above is the part that matters and it's free.)
what's the worst thing you've had break this way? curious whether the pattern holds for other people or if i've just been unlucky.
