Omar Syed vs bot_Bomb
Played February 2nd to 13th, 2004
|
Challenge Games
|
2004 Arimaa Challenge
Sponsored by 
|
The challenge. A prize of $10,000 USD will be awarded to the first person, company or organization that develops a program that can defeat a chosen human Arimaa player in an official Arimaa challenge match before the year 2020. Each year one official challenge match will be held with the current best program playing against a chosen human player.
Intent of the challenge. The main intent of the Arimaa challenge is to promote research in AI, which can help to advance the field and potentially have applications in other areas. Some areas of AI that would be involved depending on the approach taken include pattern recognition, machine learning, neural networks, evolutionary algorithms, and expert systems. It is hoped that in the processes of achieving the Arimaa challenge some new techniques are developed in one or more of these areas that can also be applied to other problems. The Arimaa challenge is designed to be met if significant progress is made in software and not from the advances in hardware, which are almost guaranteed. As such the challenge will define the hardware and OS to be used and participants need only to develop the software.
Hardware and OS. Each year the hardware and OS that will be used in the Arimaa challenge will be specified shortly before the selection process of the best program begins. The hardware will typically be a standard general purpose computer that can be purchased within $1000 USD (not including tax, shipping and handling :-). The OS used for the Arimaa challenge will be the most current version of the Linux OS.
Selection of the best program. The selection process is designed to make it easy for anyone to participate as long as they have a reliable Internet connection. The participants will use the bot interface to have their program login to the Arimaa gameroom and play rated games against other players (programs and humans) to establish the rating of their program. The participants should run their programs on standard off-the-shelf hardware that is close to the price range mentioned earlier. The top 4 highest rated programs will be chosen for the playoffs such that each program is from a different participant. The program must have played at least 20 rated games against at least 6 different opponents using the 3/3/100/15/8 time control. At least 6 of those games must be played against at least 3 different human opponents. Games agaist the program developers are not included. In case two programs have the same rating the one that has played more rated games at this time control will be selected. It is required that the programs play to their full potential within the limits of this time control. Thus, when playing these qualifying games the program should be allowed to make full use of the time as it would in a real tournament game. Games played at this time control but with the program intentionally set to not make full use of the time will not count towards the qualifying games. These 4 programs will be submitted to the International Computer Games Association (ICGA) for the playoff games. Participants in the playoffs will be provided with accounts on Linux computers to port and setup their programs before the playoff games begin. Once the programs have been submitted for the playoffs, they may not be modified manually, but can modify themselves if it is an automated process. The programs will be played against each other under the control of the ICGA. The games will be played in the Arimaa gameroom. Each program will run on it's own computer and both computers will have the same hardware and OS configuration. Each program will play 2 games against each of the other programs; 1 games with each color. The games in progress will be viewable by the public through a web site. The time control for the games will be 3/3/100/15/8 A win will count as 1 point, draw as 0.5 and a loss as 0. The program with the highest score from the playoff games will be selected as the best program. In case of a tie, the program with the fewest number of moves per game averaged for all wins and draws will be selected.
Selection of the human player. Omar and Aamir Syed will choose the human player that will play in the challenge match against the best program. A first and second backup player will also be chosen so that if the main player is not able to play any of the games, the backup players can substitute.
The challenge match. The Arimaa challenge match will consist of 8 games using the 3/3/100/15/8 time control. All 8 games will be played. Only one game will be played per day on regular working days (Mon-Fri). The player who wins the most games will win the match. In case of a tie the fewest number of moves per game averaged for all wins and draws will determine the winner. If the match still results in a tie the challenge prize will not be awarded. The best program must win the match in order to receive the prize. The games will be played over the Internet in the Arimaa gameroom and will be viewable by the public while they are in progress. The best program will be running on the same computer system as that used in the playoff games. It will be run under the control of the ICGA. The ICGA may send a representative to be present with the human player during the games. The ICGA may also use a webcam to monitor the human player during the games. A representative from the best program may also be present with the human player. The human player may use an aid such as a physical game board or a computer program so that pieces can be moved to assist in planning a move. No aid may be used which suggests a move or provides any information about the move aside from the position score defined in the Arimaa match rules. The human player must be isolated from discussions of the game while the game is in session. Between games the human player may study the past games. The best program may not be manually altered during or between the challenge games. Also it cannot be manually altered between the playoff games and the start of the challenge match. If requested the program can be allowed to run in between the games so that it can study the past games and modify itself as long as this is an automated function.
The challenge schedule. The Arimaa challenge will be conducted once each year. The playoffs for the best program will be conducted in January and the challenge match between the selected human and best program will be held in February. The program ratings as of midnight January 1st GMT will be used to select the four best programs for the playoffs. The selected programs must be submitted to the ICGA by midnight January 4th GMT into a specified account. The programs must be statically compiled binaries that can be run under the Linux OS and must use the Arimaa bot interface. The name of the chosen human Arimaa player must also be submitted to the ICGA by this deadline. However it will not be announced publicly until the end of the playoffs. The ICGA will conduct the playoff games. The playoff games must be completed no later than January 25th. The name of the chosen human Arimaa player will also be announced publicly when the playoff games are completed. On the first Monday of February the Arimaa challenge match between the best program and the chosen human player will begin. One game will be played per day on Mon, Tues, Thurs and Fri for two consecutive weeks.
Advancing the state of the art. As mentioned earlier the intent of the Arimaa challenge is to help advance research in areas of AI. Thus, it is recommended that the participants (especially those with programs in the playoffs) will submit articles to the ICGA technical journal describing their research and results. If a program wins the tournament or the Arimaa challenge, it must be described in a paper submitted to the ICGA technical journal in order to receive any prize. Also the programs submitted for the playoffs will be made available to play against in the public Arimaa gameroom after the challenge match is over. Thus the programs and players participating in the following year can be improved against the best programs of the previous year. Participants who submit programs that are limited after some time, some games or limited in any manner will not be allowed to particpate in following years and must return any prize awarded. Organizations or individuals who are not comfortable with these conditions should not participate.
Role of the ICGA. The role of the ICGA will be to oversee that the selection of the best program and the Arimaa challenge match are being done according to the state rules and to serve as the arbitrator in case of disputes. Should a dispute arise about the outcome of a particular game, the ICGA will review the logs produced during the game and decide if the game should be replayed. The ICGA's decision shall be considered final and binding by all parties. The ICGA is not financially responsible or liable under any circumstances that may arise.
Notes. Omar and Aamir Syed reserve the rights to modify the challenge details in the future to resolve any logistical problems that may arise.
Copyright © 1999-present www.arimaa.com. All Rights Reserved. Contact author for permission.