Statistics

[Perth]

People can easily start sending out messages with no real added value just for the purpose of boosting that stat to look more attractive as a Realm.

Then perhaps make it just a stat you can see for your own realm, could just be interesting to see.


Besides, I hardly see that happening.

[Vellos]

There is a problem with trade data.

I've put it all into some excel documents, and found what seems to be a discrepancy.

Tom, the "average" values quoted for prices.... those are average prices/100 bushels with each offer counted as one, equally weighted case, right? That is, an offer to buy/sell 10 bushels is counted the same as an offer to buy/sell 10,000 bushels? I ask because, when I did the fairly simple task of dividing total gold for buy/sell offers by total food for those same offers, I got rather different results than the "average price" quoted by the game data. Maybe you should have the option available to have volume-weighted averages displayed as well?

Also, regarding trade data.... is there a way the game could (maybe on an even bigger timelag) show the amount of food actually bought or sold, and what price those transactions actually occurred at? The problem with tracking offers is that the least profitable offers (or "emergency release valve" offers) will tend to linger on the market for a long time, while profitable offers will vanish, meaning that the data presented will systematically underestimate the profitability of trade.

Just some thoughts on other ways to do the trade data. Not sure how much data you really want to make available, but those are thoughts I've had while trying to work with it.

[Tom]

Tom, the "average" values quoted for prices.... those are average prices/100 bushels with each offer counted as one, equally weighted case, right?

Of course not. The SQL statement generating this value is:

Code: [Select]

select count(*) as Offers, SUM(Amount) as AmountTotal, AVG(Amount) as AmountAvg, SUM(Gold) as GoldTotal, AVG(100*Gold/Amount) as GoldAvg from MarketOffers


Also, regarding trade data.... is there a way the game could (maybe on an even bigger timelag) show the amount of food actually bought or sold, and what price those transactions actually occurred at? The problem with tracking offers is that the least profitable offers (or "emergency release valve" offers) will tend to linger on the market for a long time, while profitable offers will vanish, meaning that the data presented will systematically underestimate the profitability of trade.

That's an excellent point. Yes, I'm for it, but it requires quite a bit of additional new code and a new database table, so please write a full feature request on the bugtracker where it won't get lost.

[egamma]

There is a problem with trade data.

I've put it all into some excel documents, and found what seems to be a discrepancy.

Tom, the "average" values quoted for prices.... those are average prices/100 bushels with each offer counted as one, equally weighted case, right? That is, an offer to buy/sell 10 bushels is counted the same as an offer to buy/sell 10,000 bushels? I ask because, when I did the fairly simple task of dividing total gold for buy/sell offers by total food for those same offers, I got rather different results than the "average price" quoted by the game data. Maybe you should have the option available to have volume-weighted averages displayed as well?

Also, regarding trade data.... is there a way the game could (maybe on an even bigger timelag) show the amount of food actually bought or sold, and what price those transactions actually occurred at? The problem with tracking offers is that the least profitable offers (or "emergency release valve" offers) will tend to linger on the market for a long time, while profitable offers will vanish, meaning that the data presented will systematically underestimate the profitability of trade.

Just some thoughts on other ways to do the trade data. Not sure how much data you really want to make available, but those are thoughts I've had while trying to work with it.

Vellos, I think you missed the second two tabs of the trade data.

[Vellos]

Vellos, I think you missed the second two tabs of the trade data.

No, that's the data I used; the total gold and total food amounts from the second two tabs. It doesn't add up. For example, on February 28:

Gold that is bid for food in buy offers = 2933
Gold that is asked for food in sell offers = 2284
Food that is bid for gold in sell offers = 7850
Food that is asked for gold in buy offers = 10850

Average price, the real, actual, buy off average price for a single bushel will be Gold Bid in Buys / Food asked in Buys. For sell offers, it will be gold asked in sells / food bid in sells.

Quoted for 100 bushels instead of 1, this yields an average sell price of about 29.096 gold, and an average buy price of 27.032 gold.

But the chart for February 28 says that the average sell price is 28.34, and average buy price is 21.81

The sell price is close, but the buy price is way off. And I've noticed this tracking back to February 8; there does not appear to be a single day wherein the game reported the actual average price for which food was offered for purchase or sale.

The line you posted, Tom, seems to be the right code, but it is giving a value which, as best I can tell, is simply wrong. Maybe my math is messed up somewhere; it's easy enough to check. I suggest folks get on the market, take out the data, and check it themselves. Maybe you'll find that my math is wrong, but, as best I can tell, the in-game statistics are misreported by a fairly large margin.

[Vellos]

That's an excellent point. Yes, I'm for it, but it requires quite a bit of additional new code and a new database table, so please write a full feature request on the bugtracker where it won't get lost.

Done.

[Tom]

Well, your calculation is:

TotalGold / TotalAmount

while the statistics calculates:

Sum( Gold / Amount)


So you are right that it does calculate the average per-bushel cost summed over all trades, and not the average cost of a bushel. The difference becomes clear in an example:

Imagine there are 1000 bushels available for 100 gold (10 gold/100) and 100 bushels for 20 gold.

Total food available: 1100 bushels. Total gold asked: 120 gold


The average price of a 100 bushels is 10.91 gold.
But the average price of a sell offer is 15 gold.


Which one is "right"?

Answer: Statistically speaking, both are. They just have different meanings.

The first one is closer if you are looking for the value of a bushel of food. The second one is closer if you are interested in checking if an offer you see is below or above the market average.

However, I tend to agree with you that your number is more difficult to game, i.e. it is independent of whether I offer my 1000 bushels in batches of 100 or 200 or 500. Your price doesn't change if I split my offers. The current value does (putting it into two offers of 500 brings the average down to 13.333).

In addition, your value can be calculated from the totals, so it doesn't need to be seperately stored.

[Tom]

update: Fixed it now.

[Vellos]

Fantastic! That looks great.

Now if we can get transaction data someday, I'll be fully able to metagame my way to Ultimate Supreme Victory as I deploy my economics-fu on Battlemaster! Muahaha!

[Vellos]

Hm, actually... very strange anomaly.

If you go to Trade Prices for Dwilight, and select only the lines to show, not the bars (well, you could leave the bars on, but they make it hard to see), you will notice that the AVERAGE price for food is higher than the MAXIMUM price for food over a fairly long stretch of time (Feb 13-Feb 26). Might want to look into the code there to figure out what's going on. I don't think that the switch to an average of total quantities from an average of offers should have had this effect, no matter how heavy the tail ends are.

Maybe it's a bug in how the game gathers maximum buy offers?

[Tom]

Maybe it's a bug in how the game gathers maximum buy offers?

No, it's an artifact of me now calculating the avg number from a different data set than the max/min numbers were based on. It will sort itself out, new data written into the database will not have this problem, but old data does. I could fix it with a few lines of code, but I don't want to, because it fixes itself if you just wait a while.

Oh, and also: I am thinking about removing those numbers. They don't really make all that much sense. I had something else in mind with that, but I can't (currently) do that with the charts library I use.

[Vellos]

Another trade stats issue:

If you look at the dates for date, it doesn't match the x-axis. Everything appears to be shifted by one day, so that March 1 is listed as February 29. Did leap year cause a bug, lol?

[Vellos]

That's an excellent point. Yes, I'm for it, but it requires quite a bit of additional new code and a new database table, so please write a full feature request on the bugtracker where it won't get lost.

Regarding data on actual food transactions, I see it is "acknowledged" on the Bugtracker: any kind of ETA on this? Persistent weirdness in the Dwilight food markets are making me very hungry for real transaction data.

Hell, even if it's week-by-week instead of day-by-day data, that'd be fine.

[Tom]

I need to store the data first, and that's non-trivial, requires some design.

[Vellos]

I need to store the data first, and that's non-trivial, requires some design.

Sounds good; just thought I'd check.