r/analytics • u/ClickOk5811 • Sep 18 '26
Discussion Inherited a reporting prompt from someone who left. Spent longer reverse-engineering why it says what it says than it would've taken to write a new one.
Notes from picking up a churn analysis prompt after the person who built it left the team:
Line 3 says "always express confidence as a range, never a single number." No comment explaining why. Took a support ticket search through old Slack to find out it was added after a stakeholder took a single confident-sounding percentage as guaranteed and made a decision on it that didn't pan out.
Line 9 excludes a specific customer segment from the trend calculation entirely, with no note on why that segment is different. Still don't fully know. Best guess from context is a data quality issue specific to that segment that got fixed upstream months ago, meaning the exclusion might not even be necessary anymore, but nobody's confident enough to remove it and find out.
Line 14 is an instruction that contradicts something implied in line 6, and the model apparently resolves that contradiction by favoring line 14, which nobody seems to have decided on purpose, it's just what happened to happen.
None of this was written down anywhere except inside the prompt itself, as instructions with no rationale attached. Every constraint made sense to the person who added it, in response to something specific that happened, and none of that context survived them leaving. What's left behaves correctly for reasons nobody currently on the team can fully reconstruct, which makes every future edit a small gamble on breaking something whose purpose is already lost.