Glyph Quest Chronicles - The Post Mortem
In which Glyph Quest Chronicles comes out and I spend a disproportionate amount of time staring at analytics.
Leanne and Shin had both signed up to do a talk at the Develop Conference (11th - 13th July) all about the things we had learned from the launch.
Our launch date was the 20th July. Sure, it was supposed to be the 6th and everything was looking good and then we ran into all sorts of problems with the Android version. I disliked the Android version before and, even after release, I have yet to see anything to change my mind thus far.
Another issue involved one of our biggest features - the inclusion of Patron Mode. It's what happens when we think you've spent enough money on the game and it works like a soft cap that does away with adverts and the energy model. There are two ways of unlocking Patron Mode - the first and simplest is to buy the Patron Pack. Essentially, this transforms the game into the shareware model that we're already a fan of. The second way is to gradually build up to Patron Mode by purchasing the smaller items for sale - a gem pack here, a coin doubler there. They all add up and when you've spent enough, we turn Patron Mode on.
Buying a Patron Pack outright is all well and good. If you were to uninstall the game and reinstall it later, the purchase can be restored from your app store of choice. Building up to Patron Mode however is a different story. Since the only way of reaching the threshold was to buy at least some currency packs and those currency packs are consumable items, there's not way of restoring those purchases. An uninstall would mean that you had lost your Patron Mode status entirely.
This is obviously suboptimal, so the decision was made to try and implement a cloud save system that would let people play across devices and remember how much they'd spent. Once again, a really quite technical thing and something that we'd have to hand off to our tame coder friends.
So again, we spin up to the Develop Conference and Leanne and Shin do a wonderful talk about how we made the game and how we don't actually know anything about how well it's done or whether or not F2P is a viable concern for a tiny micro studio. Ah well.
But now. Now the game has been out for almost a month and we have a tale to tell. A tale of data. A tale of woe. A tale of almost insurmountable tech debt, terrible sales figures and plenty of firefighting one star reviews.
First, the good news. As this wasn't our first rodeo, the lessons we'd learned from the first couple of releases were put into effect. Namely, we spoke to both Apple and Google ahead of time and got as much of a promise of a feature as you ever get from those people. Happy days indeed.
Lo and behold, we were featured in most territories - some of which even had big banner things. So far, so good.
From this point on, my life seemed to consist of staring at the Unity Analytics dashboard. Specifically the livestream which would show me whenever people would fire up the game or buy something or complete a quest or... It would even ping a map of the world so I could see where the people were playing it from.
It was all very exciting - especially seeing the Revenue number going up...
But then it all started to go wrong. Very wrong indeed.
We started getting a raft of 1-star reviews on both Android and iOS. There were several complaints.
First, the ads were unskippable and the only way to get rid of them was to buy the Premium Pack which was deemed to be far too expensive.
Then we had people having issues on Android where the game wouldn't actually load at all. Did I mention that I hate doing Android things?
Some people didn't want to play the game because they couldn't choose a male character...
Finally, and this is a bit of a doozy - occasionally the game would reset the player's save game and dump them back at the start. It turns out this sort of thing is frowned upon by the playerbase. Who knew?
How the hell could we have got this so very badly wrong? Not once, in all of the testing, did the game every wipe our saves - not for us or the hundreds of beta testers and players in the soft launch. We made some tweaks to the save system without ever really understanding what was causing the problems.
As for the Android thing - well, it turned out to be an issue with the split APK. Basically, the game would just assume that everything had been downloaded already and was ready to go, but if a player had downloaded it over mobile data, chances were the OBB file wouldn't be pulled down. That would result in the game hanging on the loading screen as it tried in vain to load data that simply wasn't there. Some googling later and we managed to implement a system that would check to see if the data was there and, if not, download it first.
Far be it from me to get into an argument with people who complain about the price of something that they can play entirely for free, but we did make the cheaper purchases (Alchemist and Loyalty Card) also disable ads.
Version 1.1 was rushed through submission and we spent the time responding to the 1 star reviews - even the ones in Chinese*. This is important and something we'd already learned by hanging out on the Touch Arcade forums. The long and the short of it is this - if you get a bad review, engage with the reviewer. Find out what's grinding their gears. Explain the reasoning behind your decisions. You'll be surprised how often this is enough to turn the situation around. I'm not saying you should just jump through all of the hoops that they say you should - merely that you should acknowledge their grievances and be as transparent as possible in your process. As successful as this approach is, it's still about as soul destroying as it comes as you have to wade through a lot of bile. There's a reason why the phrase "Never read the comments" is as prominent as it is.
Everyone updated to the new version. We were still featured (except on Google Play where your feature tends to be tied in to the overall rating of the app) so it was time to see if we couldn't get a second wind.
Well the Android version would actually load now, but some people were still having their progress wiped.
The updates didn't stop there though.
The save system was completely overhauled. The previous versions used the PlayerPrefs to store data and, over time, just evolved and became quite unwieldy. To top it all, it turns out that PlayerPrefs aren't the most... secure way of storing data. That is, they don't really do any checks to ensure that what you want to write actually gets written.
Some people were complaining that the game was too hard and didn't like the idea that you had to buy potions to have a chance of completing quests. In our mind, potions are a staple RPG trope - well established and pretty universal. But that didn't stop people thinking that we were trying to gouge them out of their in-game currency. So we made sweeping changes to the balance of the monsters, making everything easier.
We also fixed a simple, but obscure, bug that would hammer the save if you left the game on any of the town screens for longer than 60 seconds. The theory was that this was the save game wipe problem...
For some people, it worked fine. For others... not so much. Now people were experiencing crashes on load or in the middle of quests - none of which had happened before. Well, not since the initial Android issues at least.
So there's a new version in the works that, whilst it doesn't actually fix any of the crashes, it does, at least, report those crashes to us with hopefully enough information that we can actually stand a chance of fixing them.
To date, here are some of the figures that I've been staring at on the analytics screen.
The Pyromancer is the most popular character, being chosen by nearly a quarter of the players.
It's something you should already know, but the importance of decent QA cannot be understated. Of course there will still be the issues that slip through the cracks, but the trick is to try and have as few of them as possible.
Opt-in Ads are pretty cool. People for the most part are happy to watch if they know they're getting something cool in return. But more than that, it's the idea that they get to make the decision themselves.
Android is a waste of time. Sorry Android users, but even though the install base is massive, the wealth of different machines, piracy and an almost complete unwillingness to pay for anything means that it's just not worth it. Not to us anyway.
* We roped one of the localisers in to crowd control.
** Perhaps it's her ridiculous hat?
Leanne and Shin had both signed up to do a talk at the Develop Conference (11th - 13th July) all about the things we had learned from the launch.
Our launch date was the 20th July. Sure, it was supposed to be the 6th and everything was looking good and then we ran into all sorts of problems with the Android version. I disliked the Android version before and, even after release, I have yet to see anything to change my mind thus far.
Another issue involved one of our biggest features - the inclusion of Patron Mode. It's what happens when we think you've spent enough money on the game and it works like a soft cap that does away with adverts and the energy model. There are two ways of unlocking Patron Mode - the first and simplest is to buy the Patron Pack. Essentially, this transforms the game into the shareware model that we're already a fan of. The second way is to gradually build up to Patron Mode by purchasing the smaller items for sale - a gem pack here, a coin doubler there. They all add up and when you've spent enough, we turn Patron Mode on.
Buying a Patron Pack outright is all well and good. If you were to uninstall the game and reinstall it later, the purchase can be restored from your app store of choice. Building up to Patron Mode however is a different story. Since the only way of reaching the threshold was to buy at least some currency packs and those currency packs are consumable items, there's not way of restoring those purchases. An uninstall would mean that you had lost your Patron Mode status entirely.
This is obviously suboptimal, so the decision was made to try and implement a cloud save system that would let people play across devices and remember how much they'd spent. Once again, a really quite technical thing and something that we'd have to hand off to our tame coder friends.
So again, we spin up to the Develop Conference and Leanne and Shin do a wonderful talk about how we made the game and how we don't actually know anything about how well it's done or whether or not F2P is a viable concern for a tiny micro studio. Ah well.
But now. Now the game has been out for almost a month and we have a tale to tell. A tale of data. A tale of woe. A tale of almost insurmountable tech debt, terrible sales figures and plenty of firefighting one star reviews.
Features
First, the good news. As this wasn't our first rodeo, the lessons we'd learned from the first couple of releases were put into effect. Namely, we spoke to both Apple and Google ahead of time and got as much of a promise of a feature as you ever get from those people. Happy days indeed.
Lo and behold, we were featured in most territories - some of which even had big banner things. So far, so good.
Analytics
From this point on, my life seemed to consist of staring at the Unity Analytics dashboard. Specifically the livestream which would show me whenever people would fire up the game or buy something or complete a quest or... It would even ping a map of the world so I could see where the people were playing it from.
It was all very exciting - especially seeing the Revenue number going up...
But then it all started to go wrong. Very wrong indeed.
Reviews
We started getting a raft of 1-star reviews on both Android and iOS. There were several complaints.
First, the ads were unskippable and the only way to get rid of them was to buy the Premium Pack which was deemed to be far too expensive.
Then we had people having issues on Android where the game wouldn't actually load at all. Did I mention that I hate doing Android things?
Some people didn't want to play the game because they couldn't choose a male character...
Finally, and this is a bit of a doozy - occasionally the game would reset the player's save game and dump them back at the start. It turns out this sort of thing is frowned upon by the playerbase. Who knew?
How the hell could we have got this so very badly wrong? Not once, in all of the testing, did the game every wipe our saves - not for us or the hundreds of beta testers and players in the soft launch. We made some tweaks to the save system without ever really understanding what was causing the problems.
As for the Android thing - well, it turned out to be an issue with the split APK. Basically, the game would just assume that everything had been downloaded already and was ready to go, but if a player had downloaded it over mobile data, chances were the OBB file wouldn't be pulled down. That would result in the game hanging on the loading screen as it tried in vain to load data that simply wasn't there. Some googling later and we managed to implement a system that would check to see if the data was there and, if not, download it first.
Far be it from me to get into an argument with people who complain about the price of something that they can play entirely for free, but we did make the cheaper purchases (Alchemist and Loyalty Card) also disable ads.
Updates
Version 1.1 was rushed through submission and we spent the time responding to the 1 star reviews - even the ones in Chinese*. This is important and something we'd already learned by hanging out on the Touch Arcade forums. The long and the short of it is this - if you get a bad review, engage with the reviewer. Find out what's grinding their gears. Explain the reasoning behind your decisions. You'll be surprised how often this is enough to turn the situation around. I'm not saying you should just jump through all of the hoops that they say you should - merely that you should acknowledge their grievances and be as transparent as possible in your process. As successful as this approach is, it's still about as soul destroying as it comes as you have to wade through a lot of bile. There's a reason why the phrase "Never read the comments" is as prominent as it is.
Everyone updated to the new version. We were still featured (except on Google Play where your feature tends to be tied in to the overall rating of the app) so it was time to see if we couldn't get a second wind.
Well the Android version would actually load now, but some people were still having their progress wiped.
The updates didn't stop there though.
The save system was completely overhauled. The previous versions used the PlayerPrefs to store data and, over time, just evolved and became quite unwieldy. To top it all, it turns out that PlayerPrefs aren't the most... secure way of storing data. That is, they don't really do any checks to ensure that what you want to write actually gets written.
Some people were complaining that the game was too hard and didn't like the idea that you had to buy potions to have a chance of completing quests. In our mind, potions are a staple RPG trope - well established and pretty universal. But that didn't stop people thinking that we were trying to gouge them out of their in-game currency. So we made sweeping changes to the balance of the monsters, making everything easier.
We also fixed a simple, but obscure, bug that would hammer the save if you left the game on any of the town screens for longer than 60 seconds. The theory was that this was the save game wipe problem...
For some people, it worked fine. For others... not so much. Now people were experiencing crashes on load or in the middle of quests - none of which had happened before. Well, not since the initial Android issues at least.
 |
| Most popular character at 24.2% |
Stats
To date, here are some of the figures that I've been staring at on the analytics screen.
- iOS Users: 100k. Android: 5k.
- DAU peaked at 17k but is now around 1.2k.
- The Patron Pack (despite being horrendously overpriced, according to some) accounts for nearly 50% of our sales. Note, not Revenue, but Sales. The Starter Pack is second at around 20% with the largest of the currency packs just behind at 18%.
- Android IAP revenue accounts for just over 2% of our total.
- IAPs account for 71% of our total revenue with ads filling in the rest.
- The discrepancy between the Pre-Verified Revenue figure you see on the livestream of the analytics and what actually comes through is truly horrific.
The Pyromancer is the most popular character, being chosen by nearly a quarter of the players.
Next up is the Heka, although some of that might also be attributed to the fact that this is the character that's highlighted by default and so will be the one selected by those that are just tapping through the first screens in a hurry.
The Wiccan just edges out the Nereid into third place by just over 100 players.
Then it's Warlock, Mystic and Summoner.
Bringing up the rear we have the Shaman**.
- On average, it currently takes people about 3.5 minutes to complete a quest. This number is gradually increasing as the playerbase gets further in to the game with earlier quests taking just over 2 minute.
- Over 100,000 quests have been embarked upon and 63% of them have successful outcomes.
- Over half the items used have been Health Potions, rather unsurprisingly. 280,000 of them thus far.
- When players run out of mana and are given the choice to refill it buy paying with Gems, 97% of them refuse.
- When we subsequently ask if they'd prefer to watch an advert to get a free go, 64% say yes.
 |
| Least popular character: 3.2% |
Takeaway
Engage with your community whenever possible. If someone has a grievance, listen to it and, more importantly, have them realise that you're doing so.
It's something you should already know, but the importance of decent QA cannot be understated. Of course there will still be the issues that slip through the cracks, but the trick is to try and have as few of them as possible.
Opt-in Ads are pretty cool. People for the most part are happy to watch if they know they're getting something cool in return. But more than that, it's the idea that they get to make the decision themselves.
Android is a waste of time. Sorry Android users, but even though the install base is massive, the wealth of different machines, piracy and an almost complete unwillingness to pay for anything means that it's just not worth it. Not to us anyway.
Next Steps
Thus far we've spectacularly failed to recoup the costs of making the game and, at the current rate, it's going to take a ridiculous length of time to break even. That means there's no more money coming in for development. This is generally regarded as a bad situation.
We're still trying to track down and fix that crash bug and will continue to update as and when we get the time, but we've completely run out of money. In truth, the income stopped a couple of months ago and we've been coasting by until this point, trying not to spend anything. But the overdraft limit is upon us and it's time to dust of the ol' CV and shelve the Great Indie Experiment once again.
* We roped one of the localisers in to crowd control.
** Perhaps it's her ridiculous hat?
Comments
Post a Comment