(originally posted at https://robocoderepository.com/jive/thread.jsp?forum=31&thread=1231&hilite=true&q=testing&start=0#9202)
Tip1 - Know the average performace of your production bot against 5 or so opponents. To do this give each opponent 300 or so rounds against you to warm up, then run a 5000 round battle and record your percentage of the total score.
Tip2 - Make a habit of recording the percentage score and number of rounds in your notes. Percentage score is (myScore/(myScore+enemyScore))*100%.
Tip3 - If you change your movement ignore the first 300 rounds against an opponent (they will be aiming where the last version of the bot was).
Tip4 - Random chance really messes up the scores. Over 100 rounds the score will be accurate only to about 5%, over 1000 rounds your score will be accurate to about 1.6% (both at 95% confidence).
Tip5 - Make sure your new bot not only beats the old bot - but also improves performance against the other benchmark bots.
Tip6 - For Team and Melee the random variation in score is even more pronounced - don't trust any scores!
The rating difference in the Eternal Rumble can be converted to an expected score percentage thus: Rating Difference Expected Score
The rating differences work for negative ratings - ie a bot with a rating 43 below an opponent would expect to get 46% of the total score.
Tip7 - Check out the Rumble and find out which bots are dragging down your rating (these are the bots that do better against you than their rating would suggest - they can be above and below you in the ratings) - concentrate on improving performace against these bots.
Tip 8 - Make your bot a big pain in the butt to test against. It should take at least half an hour to run 100 rounds against you on Dual 2.8 Ghz processors. Then no one will ever figure out your weakness, because no one's that patient. The more complex your aiming algorithm is and the more turns you skip, the better, just as long as you don't skip any turns in the first 10 rounds (which is all you need in the Rumble). I test against the entire top 10 pretty equally these days, and half of my CPU cycles go to Chameleon 0.83. -- Kawigi
Something tells me that won't be a popular technique... (with your oponents, that it) -- Tango
I have a TaskManager? class that load class implementing Task interfaces. From this I can change some parameters or a Gun or wathever. The I have a stat class that keep various informations about each round. In order to get stable result it load ancient battle data. Then you just take the data in OpenOffice? to see what's the best evolution, rolling average and so on. Usually I run 10 battle of 35 round each to estimate a Task (this may be not enough). The good thing is that you can just let it run for a big time testing various Task. --Synnalagma
![[Home]](/images/RoboWiki.png)