Also, make sure you have documented evidence of the suspected bug. Statements of "I always see this happen" or "It seems like this is how it works" are useless. The devs are normal people, with normal lives outside the game. They are NOT going to sit down and develop, set up, and run 30 test cases, then analyze them for your suspected improper behavior. Make sure you have all your ducks in a row, with documented examples of the behavior that you see. If you think, for example, that units jump off walls to soon, then provide multiple battle reports showing it happening, with as much concrete evidence as possible. Anecdotal cases are not helpful, unless the particular bug is blatantly obvious.
well, there was one documented issue on bugtracker, where i posted several battle reports and add some analysis i was able to do.
more or less, that was my peak as the one who also does not have large blocks of free time to collect further information.
the main problem is that it is incredibly hard to collect row of battle reports with very similar settings that would prove the case. how to do it in-game? most of battles are far too important that settings would be applied artificially, just for sake of testings, so i posted the question being curious on how it works with new testers.
as one example, for some time i suspect much that box formation is useless, as whenever i tried to rectify defensive bonus against cavalry charge, i found no difference compared to line formation, while attack handicap of box troops was obvious.
i can even call anyone who reads it to contribute, with more evidence we will be clever, with simple analysis.
- look at battle reports where some infantry is in line formation and other is in box and cavalry is seemingly charging against them (so only turns where it happens are relevant)
- compare casualties per hits for both type of designations (they should have approximately the same cs per men)
- compare attack hits per men for both types (again: similar weapon/armor)
so it is needed to have similarly strong infantry troops where some are in line, others are in box, and both of them need to sustain charge at least some time in the battle.
for defense analysis, it is not crucial what is strength of cavalry attackers, but bulk hits received per men.
for attack analysis, it is important that they have about the same cs per man, so as to see how many hits they produce.
battle reports do not show difference between attacking and defensive cs so I put that out of analysis, hoping that larger number of observation could give valid results even if disregarding this.
whole this explanation presents that it is not so easy to reach testing conditions... i had time and ability to reach it few times, and results shown what I suspected.
if someone thinks that methodology has flaws, please post it here.
