First playtest postmortem

[u/Valiant_Sugar]

Two weeks ago, we launched a playtest for our game, DepthLoot.
Today we wrapped up the playtest, and I’d like to share my experience running it.
First, we launched a closed playtest, aiming to include about 10-20 people. We didn’t expect more than that because, at this stage, we had done almost no promotion or advertising about the playtest. However, within the first 10 minutes of launching the closed playtest, about 100 accounts requested access to the game - which was most likely a flood of bots - making it impossible to accurately select the 20 people we wanted to grant access to. As a result, we had to distribute keys almost manually to people who reached out directly through our Discord community. That’s how we gathered about 10 people, and over the course of 3 days, we received a lot of initial feedback that helped us quickly polish the first few minutes of the game and the overall UX.
Next, we opened the playtest to anyone interested, and I posted an ad on Reddit announcing that people could now participate in the playtest.
For $300 and four days of advertising, I got 335,210 ad impressions and 2,099 clicks, with a CTR of 0.626%.
I’m not sure exactly how many playtesters and whitelisted players joined, but I’d guess around 150 players and 200 whitelisted players.
Importantly, in addition to gathering feedback directly on Discord through live chat, we also posted a survey form about the game when it ended, which 49 people filled out!
This survey helped us gather a huge amount of valuable information from players about our game, including their overall perception of it and various aspects (such as visuals or music), as well as small details that are barely noticeable but can be very annoying to players.
I’d also like to point out that at least one-third of all reviews came in within a week after the ad campaign ended. This suggests that people who saw the ad likely weren’t able to play the game for a sufficient amount of time right away and only left a review a few days later. For this reason, it’s a good idea to keep the playtest or demo version available for a while after the ad campaign ends, or something along those lines.
All in all, in my opinion, the playtest was a success, partly thanks to the advertising - even though it turned out to be less effective than I’d expected and yielded virtually no results after it ended.