Retention Revisited

[Vellos]

So I got bored recently, and I kind of have a thing for spreadsheets. So I made one, charting out registration and activity on each continent based on the game-generated graphs. I made one chart for registration, one for the 3-day activity, and one for the day-to-day activity. I was particularly curious because the game-wide chart shows a strange divergence whereby registrations have been increasing lately, while 3-day activity has been declining.

A few summaries of the past 3 months worth noting:
In terms of registration, Beluaterra and Dwilight increased, FEI and Colonies declined a bit, while EC and Atamara declined the most. Thus, in terms of new characters, it would seem that Beluaterra and Dwilight are the best attractors, while EC and Atamara are the worst. Beluaterra say regstrations increase by almost 5%, Atamara fell by almost 4%.

But registration isn't everything. In terms of number of players who are "active" (3-day login line), Beluaterra and Dwilight still dominated. Colonies and FEI were next, followed, again, by EC and Atamara. The gap was wider, however: Beluaterra had effectively no change in number of "active players," while every other island declined. EC fell by 14%, Atamara by 12%.

In terms of highly active players (the 1-day login line), the story change somewhat. Dwilight and FEI lead the pack in retaining highly active players, with EC and Atamara next, followed by Beluaterra and the Colonies. However, the Colonies is an outlier: it lost 53% of its active players in the last 3 months... mostly because these data reflect a weekday measure beginning with a weekend measure ending, and the Colonies, being a "casual play" island, can be expected to have a bigger weekend effect. Dwilight had effectively no decline in highly active players, Beluaterra lost 27% of its (perhaps a weekend effect again? Though why Beluaterra would be abnormally effected is unclear to me).

What seems evident to me is that we have basically two worlds of BM experience. In Dwilight and Beluaterra, it would seem that player counts are not suffering too much. New players arrive and are broadly able to sustain the population. In EC and Atamara, this is just not the case. Those two continents are experiencing what I would call a precipitous decline (losing 12-14% of the active players seems like a big deal to me, especially on two very large continents).

It is possible that some of the decline in Atamara and EC is due to emigration. However, FEI, the Colonies, and Beluaterra are comparatively small. They would hardly seem able to account for the almost 150 active players Atamara and EC have lost in the past 3 months (probably more, as no doubt some new ones have arrived in the interval). Dwilight is big enough that it could have absorbed many, but, still, it seems like a stretch to me that this entire phenomenon can be explained by older players emigrating from EC and Atamara to FEI, Beluaterra, and Dwilight, especially given my own experience in Dwilight that newer players are remarkably prominent. The end of the invasion probably has a lot to do with Beluaterra's bouyancy and no doubt sucked many active players from other continents, but it seems strange that this "Invasion-ending" effect would drain exclusively from EC and Atamara. Rather, I would expect Beluaterran exiles to gravitate towards other unconventional islands, like Dwilight (the advent of the Manifest Path demonstrates that this migration pattern does have a significant effect). So I would not think the end of the invasion would be to blame for EC and Atamara's decline.

EC and Atamara represent a large chunk of the game: about 1,500 characters between them. Most players have a character on one or the other, I would bet.

However, that may be the source of some of our retention problem. EC and Atamara can't hold players very well, and loss of players on those continents gains a wide audience due to being large continents. Moreover, it isn't like Dwilight were losing nobles is just part of the game, and realms fade and die. On continents of long-term realms that have remarkable aversion to their own death (for BM realms), losing nobles is a remarkable crisis.

I would thus suggest that the retention problem (and it is a problem) should not be phrased as "What is Battlemaster doing wrong?" But, rather, primarily as "What is it that happens regularly in Atamara and EC that we are doing wrong, and that sometimes also happens elsewhere?"

I will venture two guesses:
1. Low turnover in positions (purely hearsay, but I "feel" like the other continents have more new-player opportunity, especially Dwilight, and the Dukes-per-capita rate of FEI is quite impressive)
2. Lack of concept (the islands that are prospering are all "concept islands")
3. Stationary alliances

Large alliances are briefly fun. Arcaea's bloc may be able to maintain the fun in FEI by virtue of being very concept-driven, same for SA in Dwilight. However, even such concept-driven blocs seem likely to have negative effects on retention. A prime example is Thulsoma/Averoth. As much as we were all frustrated about the clanning and abuses, those realms brought in tons of new players who, were they allowed a longer time to prosper and gradually integrate in other realms, might have provided a bigger boost to retention even as they settled down slightly. Maybe that's just wishful thinking, but I think it has a base on a reasonable argument.

This post was quite long. I don't know a way to share my raw data with others, but I can e-mail it to anyone who is interested. I also calculated some composite scores for continents and compared activity rates. For the curious, I was unable to find any meaningful connection between activity and retention: continents with more activity and high activity did not seem much more likely to retain players, and activity throughout BM has declined in the past 3 months, on every continent, but differences in activity have flattened out somewhat.

[Fleugs]

An interesting read, but kind of depressing. I have the feeling, at least in Ibladesh, that we attract enough new players but that we are not capable of making them stick around. I try my best as mentor to offer them all the help they need but I have the feeling that many new players just quit the game after they took a first few looks at it (maybe logged in a few times, then gave up).

If we wish to keep more new players, a friendlier form of "new player policy" should be introduced. By this I do not mean that new players are flamed upon (on the contrary), but that they simply get lost in the complexity of Battlemaster. The single thing I recall, when I began playing battlemaster, was storming into a Falasian army on my own (being a part of Eston). In retrospect that was utterly stupid but it should give an idea to what new players are up.

A more thorough "introduction" to Battlemaster would be nice to make more new players stick around. I know, there are always people that will leave, no matter how much you help them... Nevertheless, I was thinking of a thorough tutorial when you join the game, or even a small realm/mini-island/? in which new players would be guided by a few experienced players to learn the ropes of the game, and where they can get the full attention without being lost in the complexity of a rather large realm.

Another idea just came to me, too: a video tutorial (youtube?) in which someone explains the basics of Battlemaster, and takes the viewer in, say, a 5 minute trip through the game in which he explains certain aspects and gives several tips of do's and don'ts.

Anyway, I'm all up for helping out new players, including working on a (larger) project to ease their welcome into BM. Because after all, this game is magnificent, and I want everyone to enjoy it as much as I do.

[LilWolf]

Another idea just came to me, too: a video tutorial (youtube?) in which someone explains the basics of Battlemaster, and takes the viewer in, say, a 5 minute trip through the game in which he explains certain aspects and gives several tips of do's and don'ts.

I've been thinking of doing a Let's play of Battlemaster once one of my characters dies. After initial introductions to the game maybe do a weekly update of what's going on with the character or something like that. Mainly it's on hold because I don't have headphones with a microphone of them.

[Vellos]

And do either of you have anything substantive suggesting that EC and Atamara players are worse at welcoming new people?

Because there are pretty big differences between some of the continents in that data. Some continents are not suffering, or at least not nearly as much. We have some continents that are succeeding (Dwilight). We should figure out why, IMHO. Maybe Dwilight people are just better at welcoming newcomers? Unlikely, since Beluaterra has no newcomers at all (which is demonstrated in activity statistics, btw: Beluaterra reliably had the most active player populace, except for its "very active" rate plummeted recently it seems).

While no doubt better tutorials would help retention, I personally think most of the people who leave because of the learning curve will leave almost no matter what. My main worry is people who stay for a month, maybe two or three, then leave. Because those are the people we really should have and could have gotten, and I wonder why it seems like some continents are so much better/worse at sustaining themselves.

Helpful data would be:
1. Data on where new players start their first character or two.
2. Data on the average "account lifespan" of each continent.
3. Data on "per capita" amount of players with accounts <6 months old on each continent.
4. Data on deletion rates of accounts between 2 and 12 months old on each continent.
5. Data on average "account age" for position-holders on each continent.

Can process #5 myself for Dwilight, FEI, and Beluaterra, but it'll take a long time.

Good thing it's a gloomy day outside! Time for some more spreadsheets...

[LilWolf]

For EC you should consider that there are 3 or so realms that are dying or already dead. That tends to drive away players from the island as they look for something new.

[Vellos]

True about EC. But Atamara was barely any better by any measure.

By the way, I measured the "account life" for the rulers of FEI, Dwilight, and Beluaterra. I used account number instead of date, because it was easier to enter into a spreadsheet and be done.

FEI: 20522 average, standard deviation of 8582; oldest account ID 3190, youngest 30497
Dwilight: 18716 average, standard deviation of 6685; oldest account ID 6895, youngest 31456
Beluaterra: 14855 average, standard deviation of 10812; oldest account ID 2905, youngest 29500

Beluaterra had several 4-digit account IDs, but three quite young accounts that skewed it and gave it that very large SD.

I don't know a way to access this data for EC, Atamara, or the Colonies. So either somebody else can post the links to the family accounts, or do it yourself and post the results.

Next up: dukes....

[Bedwyr]

A few things...

1. I agree that long-term stable alliances that effect no changes tend to be pretty boring, especially for new players.  I don't think the SA bloc is a problem because they've been steadily spreading and they're not just refighting the same wars all over again (and what Arcaea-bloc are you talking about, anyway?  We've got one ally that we can count on with any certainty at the moment, with the rest of the continent either at war with or neutral in the conflict right now) like, for instance, the Sirion/Fontan conflict.  I think the situation on Atamara is a bit anomalous because the CE bloc clearly was making changes, and somehow no coalition united against their expanding influence until very recently.  Why the current war isn't drawing more people I'm not sure, which leads to the next point...

2. Positions.  After playing the game for several years and watching what happens in various realms, I'm convinced that it's not so much position turnover (though having some is of course required) as position (and influence) availability that draws people.  I'm pretty confident in saying that Arcaea's one Lordship/Council seat per family rule has been the single biggest reason for its success, closely followed by more luck than I can shake a stick at and the general policy of just accepting that spies exist and telling people stuff anyway.  All else being equal, the realms I've been in with position restrictions have had a much more happy and involved playerbase than those that haven't.

Now, as I alluded to, this isn't just game mechanic positions but opportunities to influence.  Fear of spies (or factionalism) has ended up crippling a lot of places by making it so they don't share anything outside of the inner circle, and that leads to confused and bored people who don't think they have any say in the realm.  People who don't think they have an opportunity to have a say tend to leave.

3. Gold.  This one's pretty basic, but places that make an effort to get new characters reasonable oaths, as opposed to doing these godawful "trial oaths" at some ridiculous percentage (10% of a crappy rural, really?) are going to do better.  This can also be mitigated by some sort of central gold distribution system (for instance, Perdan right now has CS requirements per army, and beyond your oath you get however much gold you need from the General to get a proper unit), but you have to make that really clear in the first few days for it to have much of an effect.

4. More effort making sure new players don't get lost.  I'm blackly guilty of this one, as I don't tend to even bother saying anything to a new player until they've actually sent something, and half the time I get distracted and won't say anything even then.

[Nathan]

Could Tom put up a little voluntary questionnaire for when someone deletes their account? After a while, we might have some decent data about what leaving people think of the game, why they left, etc. This questionnaire could also be sent out to people who previous had accounts (if their addresses are still stored somewhere).

One drastic idea might be to make a new small island (smaller than the War Islands) that is specifically for new players. A few chosen experienced players can be there as mentors, but the rest are new players learning the game. After a specified time limit, the players are prompted to move their characters onto a different island to make way for new players. The game mechanics could also be altered on these islands slightly to hide a few bits of things from people - treaties, trade, rogue forces come to mind that can be changed a little to help new players.

EDIT:

2. Positions.  After playing the game for several years and watching what happens in various realms, I'm convinced that it's not so much position turnover (though having some is of course required) as position (and influence) availability that draws people.  I'm pretty confident in saying that Arcaea's one Lordship/Council seat per family rule has been the single biggest reason for its success, closely followed by more luck than I can shake a stick at and the general policy of just accepting that spies exist and telling people stuff anyway.  All else being equal, the realms I've been in with position restrictions have had a much more happy and involved playerbase than those that haven't.

Possibly why Dwilight has a growing population too?

[Vellos]

2. Positions.  After playing the game for several years and watching what happens in various realms, I'm convinced that it's not so much position turnover (though having some is of course required) as position (and influence) availability that draws people.  I'm pretty confident in saying that Arcaea's one Lordship/Council seat per family rule has been the single biggest reason for its success, closely followed by more luck than I can shake a stick at and the general policy of just accepting that spies exist and telling people stuff anyway.  All else being equal, the realms I've been in with position restrictions have had a much more happy and involved playerbase than those that haven't.

Very true. Though I would argue that turnover is a (major) factor in availability. But, position availability would probably be determined by:
1. Frequency and "success rate" of territorial wars; maybe "rate of creative destruction" of realms?
2. Viability of colony attempts
3. Turnover rates
4. Position restriction rules
5. Benevolence by high-ups

Anything else that could be  determinant of position availability?

We should presumably, then, expect high involvement by new players in continents with wars that meaningfully alter power blocs, have viable colonial activity, have restrictions on family position accumulation, and high exposure to wounding and imprisonment.

Sounds like Dwilight, and it basically does exhibit high new player involvement (see earlier post and about dukes below), though not tons better than FEI; but FEI has fairly regular power shakeups.

For the curious, some more data, this time not rulers, but dukes:
FEI: 20016 average, 8925 standard deviation; minimum of 1607, maximum of 30856
Dwilight: 21256 average, 7205 standard deviation; minimum of 7434, maximum of 31456
Beluaterra: 16603 average, 10708 standard deviation; minimum of 383, maximum of 30497

[Bedwyr]

I'd add 6. Ease of getting into decision-making message groups.

[Chenier]

So I got bored recently, and I kind of have a thing for spreadsheets. So I made one, charting out registration and activity on each continent based on the game-generated graphs. I made one chart for registration, one for the 3-day activity, and one for the day-to-day activity. I was particularly curious because the game-wide chart shows a strange divergence whereby registrations have been increasing lately, while 3-day activity has been declining.

Too long, didn't (yet) read it all. However, I did see and have a personal interpretation of the player count stats. I believe, from vague memory, that player count always goes up at the start of summer, which I assume to be linked to students finally having some time off after their exams and looking for new games to occupy their time. However, they don't all stay, and it's also a good time for many older players to go on vacation. This, I believe, would explain why registered accounts grow, while activity trends aren't the least affected.

Overall, this early-summer player count boost has no significant impact on player count trends. Seems like an equal number of people leave as the extra that come for the season.

[Vellos]

Could be completely true, but still doesn't explain the divergence between continents.

[Nathan]

I'd be interested to know how long rulers have been in place on each continent. Perhaps some older rulers are getting a little set in their ways and developing decision making cliques which are hard to get into for new players?

[GoldPanda]

One thing I'd like to note:

Player population is highly seasonal, from my personal, informal observations throughout the year. Many students seem to play BM from their schools' computer labs. We tend to get more players during spring and autumn, when schools start, and bored students start surfing the web instead of doing their homework. Conversely, player population dips during summer and winter, when students (and everyone else) leave on vacations.

Do this survey again in September and you may see a different story.

[egamma]

Could be completely true, but still doesn't explain the divergence between continents.

The Colonies seem to have younger rulers. There's not a lot going on, so that could explain a lot of the population loss--The only war is a 4v1 gangbang on Giblot, led by the largest realm on the continent. So that's probably not too exciting for about 75% of the island--two week march in one direction, fight a battle against peasant militia, then return home.

To make matters worse, nobles in two of the attacking realms have told me that they are fighting primarily because of "too much peace" messages. Hardly an exciting place to be.

Lukon needs a nice secession to make the island really interesting.