Author |
Topic: Bot learning (Read 1093 times) |
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender: 
Posts: 1413
|
 |
Bot learning
« on: Oct 7th, 2008, 5:12am » |
 Quote  Modify
|
Omar, I noticed this in the chat from Sept 29:
Quote:| just because the current bots don't learn and behave more like humans doesn't mean future ones will not. |
|
And I am a little perplexed, but it raises an issue I have been meaning to re-raise for a while. The only reason that current server bots don't learn is because you have turned learning off. To my knowledge at least bomb and gnobot are capable of learning in some way or another.
I would still like server learning to be turned on, so that bot technology is frozen at CC levels rather than specific bot play being frozen (which is not true of the non-P bots anyway since hardware changes). I remember that one of your objections to this was that by running on the server it would muddy the ownership of the learning that the bot achieves. I would like to explicitly state that I will never request any learned parameters back, I will be able to train all future bots on the game database anyway, so if the bot games are recorded in the game history, there is no lost learning.
|
|
 IP Logged |
|
|
|
Polyfractal
Forum Senior Member
Arimaa player #3241
Gender:
Posts: 27
|
|
Re: Bot learning
« Reply #1 on: Oct 9th, 2008, 8:36am » |
Quote Modify
|
I'm a little confused why learning would be turned off. If the goal is to develop bots that are capable of defeating humans, shouldn't they be given the opportunity to learn and improve over time? Otherwise, it is the same as expecting a brand new player to Arimaa to be a grand master instantly. Obviously humans require time to become aquainted with the rules and develop strategy, so the same should apply to the bots as well.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: Bot learning
« Reply #2 on: Oct 17th, 2008, 4:59pm » |
Quote Modify
|
Learning is turned off for the bots stored on the Arimaa server. For example I would like GnoBot2005CC to be frozen in terms of leaning and stay the way it was when it played in the tournament. I realize improvements in hardware will make them better, but that's very different than the bot using different parameters for the evaluation. I want the evaluation to be frozen.
I highly encourage the developers to have their bots learn from it's games and also by using the game archive. But you should run the learning bots on your own system.
It is possible to setup a web page so that players can start the bot when they want to play it and it does not have to be logged in all the time. It still means you have to keep a computer running though.
Toby, I can offer you an account on a Linux server where you can setup your bot and have learning turned on.
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: Bot learning
« Reply #3 on: Nov 4th, 2008, 10:00pm » |
Quote Modify
|
on Oct 17th, 2008, 4:59pm, omar wrote:| For example I would like GnoBot2005CC to be frozen in terms of leaning and stay the way it was when it played in the tournament. I realize improvements in hardware will make them better, but that's very different than the bot using different parameters for the evaluation. |
|
To be clear, there are many different definitions of "staying the way it was". Some examples:
1) Always reply the same way to the same board position (or with the same probabilities if the bot is non-deterministic).
2) Freeze the algorithm by which the robot comes up with its move.
3) Freeze the algorithm and limit access to later information (game database).
4) Freeze the algorithm and the hardware.
You have made your preference clear (#3), my preference is different. What I want to know is the reason for your preference. Are you worried about the CPU load on the server? Are you worried that learning software will eventually start beating all the humans? Are you worried that humans will feel they are not playing the original bot?
To be fair, I should give you my reasons for preferring #2:
a) You are trying to encourage developments in AI software. Therefore you should showcase AI software without crippling it. Algorithmic learning is a part of AI.
b) Humans will get more interesting variety in their play, and the ratings will be less distorted. The crippled version of Gnobot_2007 is susceptible to being beaten again and again with exactly the same sequence of moves. The uncrippled version is not susceptible to such lame botbashing because it changes it's opening book based on win statistics.
c) Bots developed later will get a fairer test of how good their software is, given that they have the whole games database available to them.
d) Allowing learning in the bots that play the most frequently (which happen to be those stored on the server) actually enhances everyone's learning, because it adds variety to the games which are then themselves a source of future learning (for either bots or humans). The world doesn't learn much from a bot that eternally plays a refuted opening.
Quote:| I want the evaluation to be frozen. |
|
As it happens, Gnobot satisfies this. Gnobot's evaluation function is not yet able to learn. Only its opening book is. So, would you consider allowing Gnobot's type of learning even if you deny other types that learn the evaluation? What about if a bot kept it's eval constant but learned how to search (I can think of an easy way to do this to a limited degree actually!)?
Obviously I would prefer if all forms of learning were permitted, but if you have reasons for favouring one over others, I'd be interested to hear them.
Quote:| I highly encourage the developers to have their bots learn from it's games and also by using the game archive. But you should run the learning bots on your own system. |
|
Good. Of course we will continue to do so if we feel it is of an advantage to our bots. However this doesn't achieve items (a) or (c) above, and limits the extent to which (b) and (d) are achieved.
Quote:| It is possible to setup a web page so that players can start the bot when they want to play it and it does not have to be logged in all the time. It still means you have to keep a computer running though. |
|
I personally do not care enough to go to these lengths. Even if I did, this wouldn't achieve (a), (b), (c), or (d) as effectively as if the learning bots were integrated into the normal arimaa framework for choosing games against bots.
Quote:| Toby, I can offer you an account on a Linux server where you can setup your bot and have learning turned on. |
|
That's very kind of you, and I may well take this up at some point. Does this mean that your reasons aren't related to concerns about CPU load?
|
| « Last Edit: Nov 5th, 2008, 4:03pm by 99of9 » |
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: Bot learning
« Reply #4 on: Dec 21st, 2008, 6:36pm » |
Quote Modify
|
Well, CPU load is always a concern, but even if it were not, I would still want to keep the evaluation frozen. Keeping the evaluation frozen means the data used by the evaluation function must also not change. For example if a bot kept a parameter file and modified it during/after games I would have to keep an original version of that file and copy that over before starting up the bot next time.
My goal in providing the *CC bots on the Arimaa server is to preserve the original bots that were submitted. When a developer starts up GnoBot2005CC to play against their bot they want to know how their bot would have done against the version of GnoBot that played in the 2005 computer championship (assuming similar hardware for both bots).
But perhaps our difference in opinion is simply due to my strict non-learning definition for *CC bots. I would be OK with having another version of GnoBot with learning turned on but with a different name like GnoBot2005CCL (in addition to GnoBot2005CC with no learning). This would be with the understanding that the bot developer will not be able to request the learned knowledge files (as you have already acknowledged). Also the availability of such learning bots would be dependent on how much compute and storage resources they take.
|
| « Last Edit: Dec 21st, 2008, 6:40pm by omar » |
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print