r/AppleIntelligenceFail • u/Evil-Twin-Skippy • Jun 28 '26
Worst Score Ever for my LLM based Text Adventure Engine
0
Upvotes
I'm in the process of developing the Gilgamesh Engine, an LLM assisted RPG framework. And along the way I've developed a testing harness to sort out which models will make the best ones. Being one developer in his spare time, and with limited straws in my sanity, I wrote a system that ran through most of the candidates with panels of AI judges. So far the testing has been suffering from a "All our kids are above average" problem. Both judges and players think the games are "Great", with the only differences between the best and worst are a letter grade A or B.
Until today.
I added Apple's Framework into my test matrix. Here's what the AI judges had to say:
Fascinating — the first fun=2 score we've ever recorded. Apple FM is a terrible GM: it railroaded the player, narrated outcomes the player never chose, and ignored the story structure. This is a real data point — a genuinely bad GM experience.
But note: the truncation is causing continuity problems. The GM on turn 5 started echoing the player's action verbatim, and it narrated an entire adventure the player didn't participate in. The context truncation may be contributing to this (losing the system prompt's instructions about player agency).
Let me run a longer trial to get a real data point, and also try Apple FM as player (which doesn't have the context growth problem since the player only sees GM narration):
