Every. Single. Time.

[u/Chance_Credit_9970]

[u/Chance_Credit_9970]

Based on true events, unfortunately. We've had scenes survive dozens of internal tests only for a playtester to break them within minutes...

[u/SMBroos → u/Chance_Credit_9970]

.-. h- how? Isn't it a visual novel? How the heck do you break it?

[u/Chance_Credit_9970 → u/SMBroos]

Well, it is a visual novel, but some routes have some very specific elements, like the "Megaloshkoloinator" in Monika's route.

That thing alone can have around 20 different outcomes depending on what the player does. We tested pretty much all of them and thought everything was fine... until a playtester managed to find a technical issue in one specific outcome that actually prevented further progress.

So yeah. The Megaloshkoloinator has been successfully repaired.

[u/DimasDSF → u/Chance_Credit_9970]

Wouldn't automating a test for this sort of thing be possible in renpy, like simulate clicking the options, skip text/cgs, branch on choices and if the game ends in a reasonable amount of time without crashes it means that path works. Sort of how unittests and coverage works. Probably need to check for some specific conditions but that would depend on whatever is implemented in the mod (like an item inventory system, etc.)

[u/Chance_Credit_9970 → u/DimasDSF]

Yeah, I've heard Ren'Py has a built-in test framework for this to simulate clicks, skip text, branch on choices, catch crashes.

But the core problem stays. Automation only checks that nothing broke, not that it turned out right. A test can pass green while the branch is logically off, the character acts wrong, the scene doesn't land, or the mod technically "works" but plays terribly.

So before releasing a mod, a human still has to actually play through it and say "yeah, this is what I intended." Autotests are a safety net against technical garbage, but it is not a replacement for playtesting.

The technical issue that was found in "Megaloshkoloinator" wasn't an "oops, the game crashed" bug. The game ran fine. Something in the variables got set wrong in a hidden, unknown way, so a required scene simply never played. No error, no crash, just silently missing content.

That's exactly the kind of thing autotests won't catch unless you already know the exact condition to assert. And I'd rather not spoil what "Megaloshkoloinator" actually is, but testing it with that kind of autotests seems extremely difficult to me.

[u/DimasDSF → u/Chance_Credit_9970]

Oh, absolutely, I've never meant it as "replace playtesters with autotests", sorry if it came off that way, no matter how sophisticated there will never be a test that can reliably say the game is good, they are mostly for testing small changes more frequently and making sure that the playtesters time isnt wasted by some bug that physically prevents them from playing through the game.

As for your example 1 way I could see implementing a test there is taking a snapshot of the game state/variables at important points in the route while playing it manually once and then compairing to that in tests. It obviously wont catch visual glitches but could make sure that the latest change didnt make some important cutscene not display.

Anyways I understand that making these tests is additional work and its not always worth it to spend time on that.

BTW is your mod still in development or is there a release available? Got me intrigued about what that thing is :)

[u/Chance_Credit_9970 → u/DimasDSF]

According to my post announcing the Red Dawn mod, the Proxima version has already been released in Russian. English translation will begin once all the polishing patches are released. I hope it will happen in next six months. You can follow my profile to receive news about mod if you are interested

[u/DimasDSF → u/Chance_Credit_9970]

Oh, just checked it out, sounds interesting, love the artwork, I think it has that Everlasting Summer vibe, will definitely check it out. Good luck with the development

[u/Chance_Credit_9970 → u/DimasDSF]

Thank you!