Retention Revisited

[Bedwyr]

Thanks again for running the numbers, Vellos, that's useful information.

[Vellos]

Ya'll thought you could just forget about this thread didn't you?

Not so! Boredom strikes again, I have run the numbers!

They sharpen our trends this time. Why? Because of two things:
1. Cleanup scripte deleted tons of inactive accounts
2. The day that BM was down made me take my data on activity from a slightly different point on the lines, which was probably actually more representative

#1 was a very important effect. #2 probably only mattered for Dwilight, but we'll see in a day or two when new stats pages go up and I can cleanup my numbers on activity.

Over my 5 times taking data, I have scored each continent on activity, registrations, high activity, and rates of change of those things. I assign scores. The "rolling average" scores (of a maximum 6) are here for each continent thus far:
Dwilight- 4.83
Beluaterra- 4.08
Far East Island- 4.00
Atamara- 2.92
Colonies- 2.83
East Continent- 2.33

What does this say? It says that, broadly, across my measures, and across the entire period of time from late march/early April to today, we can see basically two "cohorts." Dwilight, Beluaterra, and FEI are reliably better at maintaining registrations, keeping retention, and in some cases propping up activity rates (or, at least, they do better compared to the other continents; they may still not be doing well enough). Atamara, the Colonies, and East Continent are the second cohort, with reliably poor numbers across almost every category. This should not be a huge cause of concern for the Colonies, where the activity numbers will be understandably lower , but it should greatly concern players in Atamara and East Continent... especially Atamara, given its size. Our "flagship continents" are dying, and rarely post any optimistic numbers.

To the most recent data, then.

The cleanup script revealed that, yes, many summer registrations are bogus. Much more than we might have hoped. The dataset I ran today went from May 15 until today, so it's basically our summer.

On EC, we can observe that, when we drop inactive accounts, the summer has seen an actual decline in registrations... to the tune of 9% of the continent's population vanishing. The number doubles when you measure only active players, and remains over 10% for even the most active players. To clarify, EC is losing players at an astonishing rate, and is losing active players at an even faster rate than "registered accounts" in general. Meaning that, not only are registrations not keeping pace with natural turnover, but players involved in BM are fleeing EC.

The story is essentially the same for Atamara, thought slightly less severe. WHile Atamara can be lumped into the same "retention cohort" as EC, it seems to maybe have at least some pocket of strength that is keeping its decline slightly managed.  My intuition based on the charts is that Atamara sees more exogenous registration growth. Why this is, I couldn't say; maybe people invite friends to Atamara more, or experienced players are more likely to run "second characters" there, or its description is more appealing to new players. Atamara's numbers are bad, but not quite as bad as EC.

The Colonies has dismal numbers. It had the largest registration decline over the summer, and a large decline in active players, but actually managed to retain highly active players fairly well. This is extremely peculiar given the Colonies' slower play. However, numbers regarding highly active players are generally circumspect given the volatility in those charts, and especially in this case given the day of BM being down recently.

Beluaterra lost nobles, but only mildly so. Crucially, while in terms of registration Beluaterra looks like it might fit into the "underperformer" cohort, its activity graph is telling. It actually has a positively slopped activity chart. This is a big deal. Beluaterra is, according to the numbers, more densely populated by active players than it was in May. My subjective and unsubstantiated reasoning on this is that Beluaterra's activity is being sustained by the prospect of war. In terms of high activity, Beluaterra actually showed signs of significant growth, but, again, these numbers are dubious.

The Far East is a difficult statistical case. FEI's registration definitely fits with a growing continent; it hasn't grown much over the summer, but hasn't shrunk. Its activity is at first very poor; however, if we discount the multi-lock that messes it up, we will actually see that FEI has a very good activity chart. Outside of the multi-lock, it has actually stayed flat or grown. Regarding high activity, FEI has shown growth, but this growth was within its band of volatility. Overall, FEI is still struggling, but shows signs of strength.

Finally, the darling of anyone who loves retention, the astounding success of a continent, the one, the only... Dwilight!

It's a beautiful graph. Activity is basically steady with some seasonal shifts; little loss or gain. High activity shows gains, but this is, again, unreliable. But what is important here is registrations. Dwilight is one of the only continents that we can say with confidence is not depopulating. Its registration rate is at or above its natural replacement rate. This is a Good Thing. Even after the cleanup script deletions, Dwilight has more nobles today than in may, and the same or more active nobles and very active nobles. It is one of the only continents that can say this without qualifications.

My formal conclusion:
Let all nay-sayers of Dwilight go !@#$ themselves. It's the only continent that's not dying.

[Vellos]

That's right.

I just downgraded EC and Atamara's retention rating from AAA to AA+.

[Shizzle]

Perhaps one of the most useful posts in this entire forum. Thank you, kind sir!

[Chenier]

That's right.

I just downgraded EC and Atamara's retention rating from AAA to AA+.

*gasp*

The world must be collapsing!

[loren]

On EC, we can observe that, when we drop inactive accounts, the summer has seen an actual decline in registrations... to the tune of 9% of the continent's population vanishing.

Does this analysis account for fluctuations in realm size?  We could just be seeing a lot of player churn that are leaving losing realms or realms that lost for other continents.  That's a pretty big confound.

[Vellos]

Does this analysis account for fluctuations in realm size?  We could just be seeing a lot of player churn that are leaving losing realms or realms that lost for other continents.  That's a pretty big confound.

No, it does not, but that doesn't matter for my purposes.

My purpose is only to see where players like to play; what continents are performing the best. I'm treating each continent as a variable– no measure of realms, politics, income, etc.

My goal in starting this thread was simply to start getting some numbers out there on retention. I'm hoping that, by Christmas break from university, I'll have assembled a fairly big pool of data... enough to be able to state confidently whether or not some continents are "winners" and some are "losers."

Once that is accomplished, we can begin to look more closely at why. But I think many players assume certain things are or aren't helping retention without any numbers. For example, you assume realms being destroyed will cause a net decrease in players, and that seems plausible, but maybe that should tell us that, if we want BM to survive, we should destroy fewer realms.

Now, personally, I think that's ludicrous. A destroyed realm only reduces retention if the player has no other fun location in which to play. For clans (Averoth, Thulsoma...) that means a destroyed realm will ruin the game for them. But for most of us, it doesn't. Moreover, if it's a mere "churn" of players from destroyed realms, new realms should pick up new nobles, meaning there should be a balance in the end. If it never balances out, again, maybe that should tell us that we need to find ways of fighting "conclusive" and "decisive" wars that don't involve destroying an enemy realm: maybe allow "victors" to ban X nobles from the defeated realm? Maybe have an easier land-turnover mechanic? Who knows?

But that's all tangential. My point is simply to determine if there are meaningful differences in retention. Thus far, the answer seems to be a resounding "Yes." That could change. Maybe Dwilight players are mostly summer-players (students and new people?) while EC and Atamara players are more year-around (older people, year-round jobs or careers).

[Bedwyr]

Mm, that only works if new realms are set up rather than existing realms expanding.  Destroyed realms tend to cause sharp drops in characters, while expanded realms tend to cause a gradual increase, in my experience.

[loren]

Mm, that only works if new realms are set up rather than existing realms expanding.  Destroyed realms tend to cause sharp drops in characters, while expanded realms tend to cause a gradual increase, in my experience.

Precisely.  Or expanding realms with new regions.  Dwilight is acting as a player sink attracting older players whose realms are losing regions over time.  Sirion I know for a fact is having problems keeping characters even though it is a great realm, with a lot of lordship turnover.  This despite Fontan and SoA losing badly to them.  Westmoor's already started to get a few refugees, but I'm betting more than a few are leaving for other isles.

[Vellos]

No, it doesn't matter.

If destroyed realms lead to declining active player counts, they are bad for retention. End of story.

Across all of BM, numbers of active and very active players are declining, despite strong registration growth. Some continents mimic this pattern. Some do not. We need to figure out WHERE we are losing players. The data indicate it is in Atamara, EC, and the Colonies, mostly.

It is hypothetically possible that EC players are immigrating to, say, Dwilight, when their EC realm dies, and staying a while, then leaving Battlemaster, only to be replaced by even more new players who came directly to Dwilight.

.... it is possible. But is there anyone who thinks this is actually what's happening? EC is actually great at retention, but people emigrate and arrive at other continents, then delete, but those continents are somehow amazing at drawing new people in? That would require an explanation of why Dwilight, Beluaterra, and FEI draw in such a vastly higher number of new, active players.

Even if we get more particular, destroyed realms should not matter for retention even for a specific continent, let alone all continents. Yes, in the short term, they will decrease player counts if they are replaced by expanding old realms. But I'm not measuring short term. My shortest time period is 10-15 days. I intend to keep doing this fairly regularly for a long time. Maybe by Christmas we'll see that this summer, with its clear-cut differences, was a fluke. If so, I stand corrected. But the data, as it stands today, does not indicate that. In the long term a destroyed realm will lose nobles, and an expanded realm should attract nobles; if it fails to attract nobles, it will eventually die, or let regions go rogue. In either case, I would assume that, unless there is a chronic game-wide shortage of nobles on an even more severe scale than we currently have, those rogue regions will eventually be colonized by new realms that draw new nobles, or else be expanded into by overpopulated realms. It should generate an equilibrium. If it fails to do so, it is indicative that something about a continent is fundamentally unappealing.

What about destroyed realms could be unappealing? Hm, here's an idea: imagine a continent where one or two massive realms have never been taken down, never been defeated, despite sometimes vast alliances arranged against them. This/these realms are characterized by strong geographic advantages, very stable internal politics, and ruthless hegemony over their neighbors. They repeatedly destroy or dismember any realm near them that challenges their hegemony. They thus create constant drains on retention on their continents.

I'm not saying this is the case. I don't know why EC and Atamara have worse retention. But they do. And if it's because of realms being destroyed, maybe we should look at why that's happening and, if realms are destroyed on other continents (as they most certainly are) why those continents don't seem to lose as many nobles when it happens.

A final point: even if EC or Atamara are completely failing at retaining their nobles, that is not a justifying cause for closing them. That's not my argument at all. It is entirely possible that, while EC and Atamara are bad at hanging onto their own nobles, they offer "network economies" by providing variety and spice for peoples' "second characters," or for special niche markets of players who, if lost, would delete their "second characters" on other continents. To answer the question of whether EC or Atamara do have these effects would require a systematic network analysis of every account in BM with cluster analysis done on some kind of proxy variable representing the primacy of a character for a player. This would be very subjective and require a level of database access that I don't have, and a level of computational complexity that I'm unwilling to invest at the present time.

[Vellos]

Precisely.  Or expanding realms with new regions.  Dwilight is acting as a player sink attracting older players whose realms are losing regions over time.  Sirion I know for a fact is having problems keeping characters even though it is a great realm, with a lot of lordship turnover.  This despite Fontan and SoA losing badly to them.  Westmoor's already started to get a few refugees, but I'm betting more than a few are leaving for other isles.

Show me the numbers. Show your evidence. In Terran, at least, we haven't seen any immigrants from EC in a while that I know of. Maybe other realms see it differently. But it's exactly this kind of bull!@#$ anecdotal arguing that I'd like for us to avoid.

Why might Dwilight be attracting players? Who knows? But whatever the case, it attracts more of everything: new registrations, active players, and very active players. Is a lot of that "internal migration" within BM? Absolutely. Does that matter? No. The flow of nobles between continents can be viewed as a market. It's a semi-free one; restrictions on movement are fairly short temporal barriers, small capital barriers, but a system of fairly restrictive quotas (nobles-per-continent limits). I expect if those limits were removed on Dwilight, EC and Atamara would suffer even more.

That doesn't mean that Dwilight is a villain or that EC or Atamara are necessarily bad. It just means that EC and Atamara are doing a poor job of competing in providing the kind of fun that players come to BM to get.

[loren]

What about destroyed realms could be unappealing? Hm, here's an idea: imagine a continent where one or two massive realms have never been taken down, never been defeated, despite sometimes vast alliances arranged against them. This/these realms are characterized by strong geographic advantages, very stable internal politics, and ruthless hegemony over their neighbors. They repeatedly destroy or dismember any realm near them that challenges their hegemony. They thus create constant drains on retention on their continents.

I'm not saying this is the case. I don't know why EC and Atamara have worse retention. But they do. And if it's because of realms being destroyed, maybe we should look at why that's happening and, if realms are destroyed on other continents (as they most certainly are) why those continents don't seem to lose as many nobles when it happens.

A final point: even if EC or Atamara are completely failing at retaining their nobles, that is not a justifying cause for closing them. That's not my argument at all. It is entirely possible that, while EC and Atamara are bad at hanging onto their own nobles, they offer "network economies" by providing variety and spice for peoples' "second characters," or for special niche markets of players who, if lost, would delete their "second characters" on other continents. To answer the question of whether EC or Atamara do have these effects would require a systematic network analysis of every account in BM with cluster analysis done on some kind of proxy variable representing the primacy of a character for a player. This would be very subjective and require a level of database access that I don't have, and a level of computational complexity that I'm unwilling to invest at the present time.

I think a better question is why do new players gravitate to one place over the other.  It'd be relatively easy to look at the connections between all the different players and where their characters are, that is to say to actually test where people tend to play.

What might be more interesting to look at is where new players have their most advanced characters.  More titles, more honor, more prestige, more guilds/religions joined.  It could simply be that there is more action to be had and so they go and stay there.  If they choose the older continents there is less churn so they're stuck waiting doing nothing.

Another interesting thing to look at might be where in the first month they receive the most messages from other players.

[Jens Namtrah]

Have you considered very simple "external" reasons - perhaps some islands have players who more actively recruit friends to the game?

Purely anecdotal, but I would say this is far more frequent on Dwilight and FEI than Atamara, for example.

[Gustav Kuriga]

Mm, that only works if new realms are set up rather than existing realms expanding.  Destroyed realms tend to cause sharp drops in characters, while expanded realms tend to cause a gradual increase, in my experience.

Not what we are seeing in Dwilight. Two realms have been destroyed, a third will be soon, and more than likely a fourth (although that doesn't count, it is just a secession from Caerwyn and has very few nobles).

If the numbers we are seeing from Vellos are correct, then Dwilight has been gaining nobles precisely when it should be losing them according to your logic. I think the problem for the other continents is that they aren't big enough. Dwilight is the size of at least two other continents, allowing whole regions to have their own local politics without involving the rest of the continent. This makes for immense variety and a greater chance for small realms to be formed. It also lessens the impact large realms have on the rest of the continent, preventing realms from dominating it. Religion, on the other hand...

[JPierreD]

Have you considered very simple "external" reasons - perhaps some islands have players who more actively recruit friends to the game?

Purely anecdotal, but I would say this is far more frequent on Dwilight and FEI than Atamara, for example.

Why would that be? What has Dwilight that makes it more appealing/demanding for players to actively recruit other friends?