doublep
Forum Guru
Badger author
Gender: 
Posts: 82
|
 |
Regression testing
« on: Oct 1st, 2009, 1:32pm » |
 Quote  Modify
|
I'm curious if any of the developers uses regression testing. It seems not that useful for strategic things, i.e. evaluating whole board positions, but I find it invaluable for tactical searches, like capturing or finding win in one move. Strategical tests could be possible in cases of only one or few "correct" moves (i.e. if you don't make this move, you are inevitably losing a major piece, like that), but I'm not a good enough player to spot this.
Badger uses a slightly modified version of GTP, so I'm able to formulate regression problems like this:
Code:# Pull opponent ahead.
clear_board
setup_squares Ra7 Cb8 ra8 ec7 cb6
50010 find_one_move_win white
#? [true Cb8s-ra8e-Ra7n]
|
|
The last line (that starting with #?) is just a comment from engine point of view, but for regression script it is a marker that provides expected answer to the last command. I.e. the script considers test passed if Badger's answer to 'find_one_move_win white' in that position is 'true Cb8s-ra8e-Ra7n'.
Eventually I even wrote automated tools to find mistakes in various tactical searches, both false positives (i.e. invalid moves or moves that don't yield results the engine claims they would) and missed problem solutions (missed one move wins, missed captures). The tool also interfaces with Badger through GTP. For its use, Badger provides two solvers for all tactical problems it handles: the production solver --- very fast, the one that is tested, --- and "robust" solver --- slow, simple, mostly brute-force and complete (unless there are unspotted bugs, of course ;). Minor custom editing is required, but most work is automatic. Tests generated with such a tool look this way:
Code:setup_board -----r----r--cr--m-h--rr-H-h--r-r---Ed-----DC-eM-----CDdR-R----R
40 capture_anything black
#? [!true hd5s-Dd3w-Dc3x-hd4s]
|
|
Do any other bots use anything similar?
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print
Author
IP Logged