Welcome, Guest. Please Login or Register.
Nov 22nd, 2024, 9:27pm

Home Home Help Help Search Search Members Members Login Login Register Register
Arimaa Forum « 2010 Bot Ladder »


   Arimaa Forum
   Arimaa
   General Discussion
(Moderator: supersamu)
   2010 Bot Ladder
« Previous topic | Next topic »
Pages: 1 2 3  ...  5 Reply Reply Notify of replies Notify of replies Send Topic Send Topic Print Print
   Author  Topic: 2010 Bot Ladder  (Read 5758 times)
omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
2010 Bot Ladder
« on: Sep 18th, 2010, 9:49am »
Quote Quote Modify Modify

I've been meaning to change the bots on the ladder and reset the ladder so we can all have fun clearing it again.
 
To help me decide what bots to put on the ladder, please post suggestions on what bots you would like to see; how many bots you think should be on the ladder, etc.
IP Logged
clojure
Forum Guru
*****



Arimaa player #5004

   
WWW

Gender: male
Posts: 207
Re: 2010 Bot Ladder
« Reply #1 on: Sep 18th, 2010, 10:30am »
Quote Quote Modify Modify

I just had this discussion in the chat room. Maybe it helps?
 
Code:

06:00:15 clojure what i dislike strongly about the bot ladder is that in the middle you hit your head against fast/blitz bots. why not make a ladder of CC only? my progress halted totally since i don't want to play fast
06:01:06 novacat I was not aware anyone actually thought that was true, although I guess it is not surprising
06:01:33 aaaa Actually, people don't like the CC bots because the games take so long.
06:03:55 hanzack what does CC stand for?
06:04:12 novacat computer championship
06:04:28 clojure aaaa, it's pretty normal to have long games in go for example. so maybe it's different for chess players?
06:04:44 hanzack cclemon?
06:05:03 clojure i feel that the additional time has really big impact on the quality of games
06:05:19 clojure i got immersed totally different way than in blitz
06:05:48 novacat I agree, but the bot ladder does not need to reflect only that side
06:05:52 clojure and it's even more unfortunate for to watch live games when they are not slow... it's hard to know anything but very superficially
06:06:03 clojure maybe not, that's true
06:06:07 aaaa Maybe introduce 1-minute versions of the bots.
06:06:45 clojure i don't really mind what time bot takes unless i can have as much as i like
06:07:06 aaaa It should not be too easy to reach the top of the ladder. It should still represent an accomplishment. I lean against dropping the blitz bots.
06:07:10 clojure if bot plays better than me, i can learn, whether bot played at 5 sec or 2 min
06:10:25 novacat I am not good at blitz, but I did learn a lot from playing many blitz games (49 games to beat Bomb2005Blitz)
06:10:27 clojure maybe the problem is that there's a group of fast bots in sequence, so you cannot progress at all, and play them later. a better alternative could have cc/fast/blitz in every third place and thus if you always won the cc version, you got to the top but haven't finished the ladder yet
06:10:51 clojure one problem of a blitz is that of interaction with mouse is cumbersome
06:11:20 clojure the physical act of making move is too big a part of the time
06:11:46 novacat that is why there must be a plan of action
06:12:09 clojure i don't get how do you do that in a complex local situation
06:12:13 novacat knowing what moves you will start with will give you a few more seconds later
06:12:31 aaaa Well, that's how you'll develop an instinct for the tactical side of the game.
06:13:32 clojure ok, i'll have more tries then
06:13:35 novacat you also quickly learn what moves don't work, and you can get in 5 + games in an hour
06:14:12 clojure i'm not at this point so sure how important the quantity of games is for arimaa
06:14:32 clojure i rather think that the time spent alone is important
06:14:43 novacat quantity helps learn the basics, quality helps improve them
06:15:39 clojure i personally find that i don't learn much with quick play
06:16:14 novacat what I was really saying, is that if you make a bad move on move 14, you didn't just waste 30 min, but more like 6
06:16:25 clojure but sure experts in go suggest to play both blitz and slow
06:17:06 clojure if you make a bad move in slow game, it's not blunder probably
06:17:12 clojure so you'll learn something really new
06:17:43 clojure besides making bad move forces you to learn to cope as an underdog
06:18:01 clojure you are forced to make the situation complex for the opponent
06:18:32 clojure as in chess you wouldn't want to trade pieces to simplify, and in go you would make fierce tactical battles
06:20:41 novacat If you only play postal, there is plenty of time to consider situations and blitz will not help you much
06:20:43 clojure this was just some feedback on the current system. it's not a big deal
06:21:44 clojure it's funny that i actually play in postals faster than in CC  
06:21:48 novacat if you play at 1 minute, it is good to trim the number of options you consider so you don't have time crunches

Is there anyone feeling the same way?
IP Logged
Sconibulus
Forum Guru
*****



Arimaa player #4633

   


Gender: male
Posts: 116
Re: 2010 Bot Ladder
« Reply #2 on: Sep 18th, 2010, 12:55pm »
Quote Quote Modify Modify

I think that adding the CC bots would cause most people to never finish the ladder, not because they can't, but because they don't wish to expend the two hour time investment.
 
Bots playing with time controls use the full time allotment, unlike the p1 and p2 bots which move in only a few seconds, but still give you the full two minutes, to use or ignore as you see fit (I prefer ignore).
 
IP Logged

Eltripas
Forum Guru
*****




Meh-he-kah-naw

   


Gender: male
Posts: 225
Re: 2010 Bot Ladder
« Reply #3 on: Sep 18th, 2010, 2:20pm »
Quote Quote Modify Modify

No CC bots on the ladder please, those games take an eternity not all the people have the time to play those, I think we should keep the bots that are on the ladder plus PT, Marwin and Badger also updating  Clueless, Gnobot, Sharp and Opfor would be nice but the addition of the new bots is more important for me, I like the current time controls for p1 and p2 bots, also I think that the existence of fast and blitz bots on the ladder is ok, is not like I'm asking to put lightning bots on the ladder.
IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: 2010 Bot Ladder
« Reply #4 on: Sep 18th, 2010, 4:07pm »
Quote Quote Modify Modify

on Sep 18th, 2010, 10:30am, clojure wrote:
Is there anyone feeling the same way?

Yes, there are people who feel the same way that you do, i.e. that playing fast and blitz bots is less fun and not the best way to improve.  However, you are in the minority.  We have experience to prove it.  Originally there were CC bots on the ladder, and many people gave up on the ladder because of it.  They only wanted games that they could finish quickly.  Now that the CC bots are off the ladder, replaced by fast bots, there is a different class of people who give up on the ladder, but not as many as before.
 
This isn't a theoretical argument about what kind of ladder is better, just a practical observation about what the masses want.  The "careful thinking" crowd is smaller than the "fast action" crowd.  It is what it is.
IP Logged

clojure
Forum Guru
*****



Arimaa player #5004

   
WWW

Gender: male
Posts: 207
Re: 2010 Bot Ladder
« Reply #5 on: Sep 18th, 2010, 5:06pm »
Quote Quote Modify Modify

One must notice that careful adjustation of the bot ladder gives different class of players a longer run in the ladder. This is due to the rule that three (3) of the next bots are playable. So the problem is mainly incidental: there is a sequence of fast/blitz bots in the middle of the ladder. If every 3rd bot was at least a bit slower than fast, some people might get further, and it wouldn't hinder those that don't like to waste their time on bots.
 
But I can fare without the ladder. I'm just saying how I feel about it.
IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: 2010 Bot Ladder
« Reply #6 on: Sep 18th, 2010, 6:15pm »
Quote Quote Modify Modify

My first concern for the ladder is for it not to include broken and/or resource-hogging bots.  We have plenty of variety available to choose from; there is no need to create unpleasant experiences from timeouts and/or server overload when plenty of well-behaved bots are available.  If there are 2010 bots with no switch to restrict them to one core and no switch to limit their memory use, simply leave them off the ladder, and ask that developers put in such switches for 2011.
 
Within that constraint, however, there should be as great a variety as possible in strategies.  It is both more fun and more instructive to have different engines than multiple strengths of the same engine.
 
My main objection to having four strengths (P1, P2, Fast, Blitz) of every engine is that the ladder gets too big.  About forty bots on the ladder seems like a maximum to me, and thirty would not be too few.  The ladder does not have to be exhaustive.  The world will not stop turning if there is AamiraFast but no AamriaBlitz while there is a MarwinBlitz but no MarwinFast.
 
I have a small request that is unrelated to how the ladder is constructed in general: For measuring historical rating stability it would be nice if Arimaazilla, Gnobot2005P1, Gnobot2005P1, Bomb2005P1, and Bomb2005P2 were all left in the ladder.  I don't care much about the Fast and Blitz versions, since they have already varied with changing server hardware, but the fixed-performance versions allow a direct comparison to the past.
« Last Edit: Sep 18th, 2010, 6:19pm by Fritzlein » IP Logged

aaaa
Forum Guru
*****



Arimaa player #958

   


Posts: 768
Re: 2010 Bot Ladder
« Reply #7 on: Sep 18th, 2010, 7:04pm »
Quote Quote Modify Modify

Ignoring time control settings or any issues with particular bots, I simply can't fathom how it is possible for there ever being too many of them on the ladder. Would people really be discouraged so much from playing any bot on the ladder if they didn't see themselves completing it in any foreseeable time, especially compared with how many bots there are on it already? Anyone starting out will have to invest a considerable time regardless.
 
I think the ladder should have every reasonably fast bot that doesn't cause any problems to the server and doesn't have a chronic timeout problem.
I count 80 such bots: bot_Arimaalon (which is oddly currently absent from the ladder, probably for being taken for the slower bot_Arimaazon), bot_ShallowBlue, bot_Arimaazilla and all other server bots with suffixes P1, P2, Fast & Blitz except for bot_Clueless2006Blitz, bot_GnoBot2009Fast and bot_GnoBot2009Blitz which time out way too often.
That would be equivalent to leaving out bot_Bomb2005Lightning, bot_Arimaazon (for taking a minute per move), the three postal bots (bot_ArimaaScoreP3, bot_Bomb2005P3 and bot_Bomb2005P4), aforementioned three tardy bots and all bots with suffix CC.
This would make completing the ladder a truly momentous accomplishment and lead to accurate ratings for as many bots as is reasonable.
 
My biggest peeve with the current ladder is that Bomb is the most dominant bot on it by far and I speak from personal experience that attuning one self to just one bot invariably leads to bad habits. You can't have too much variety and there isn't much of a downside to having to play potentially similar versions of bots; if they are indeed similar, then it should likewise be no problem for players to replicate their mastery over one bot with a (near-)clone.
 
I don't know whether one can currently advance by winning by timeout, but given the "loss by illegal move" mechanism in place and the absence of any bad apples, this should not be the case.
IP Logged
Eltripas
Forum Guru
*****




Meh-he-kah-naw

   


Gender: male
Posts: 225
Re: 2010 Bot Ladder
« Reply #8 on: Sep 18th, 2010, 7:24pm »
Quote Quote Modify Modify

I don't see any good reason to have limit the amount of bots in the ladder, in fact, for me, the more the merrier because I (and I'm pretty sure I'm not the only one)  use the ladder as a fast way to start a game, I find a little annoying the need to go to the top rated bots to start a game.
IP Logged
Hippo
Forum Guru
*****




Arimaa player #4450

   


Gender: male
Posts: 883
Re: 2010 Bot Ladder
« Reply #9 on: Sep 19th, 2010, 4:38am »
Quote Quote Modify Modify

1) Bot ordering depends on their actual rankings this could explain blitz bots near one another. I would expect clueless/marwin bots on all speeds together as well.
 
2) It would be very time consuming to beat MarwinCC and CluelessCC especially when not done on the first try.
 
3) There could be possibility of assymetric time controlls that would allow to play good bots in comfortable human speed. It seems to me fast marwin/clueless with 2 minutes for human could be good training
 
4) I dont think lightning would be good on ladder as it is higly dependent on hardware to be able to play it.
« Last Edit: Sep 19th, 2010, 4:39am by Hippo » IP Logged

aaaa
Forum Guru
*****



Arimaa player #958

   


Posts: 768
Re: 2010 Bot Ladder
« Reply #10 on: Sep 19th, 2010, 10:59am »
Quote Quote Modify Modify

The following table shows the proportion of lost rated games that were due to a timeout (plus sample size) for each server bot for which this is a positive number:
 
bot_Clueless2006Blitz0.896

125

bot_Clueless2010Blitz0.778

9

bot_Rat2009Blitz0.667

3

bot_Clueless2010Fast0.625

8

bot_GnoBot2009Fast0.548

31

bot_PragmaticTheory2010Fast0.500

4

bot_Rat2009CC0.500

2

bot_GnoBot2009Blitz0.452

31

bot_OpFor2009CC0.375

8

bot_Occam2004CC0.354

48

bot_Zombie2008CC0.326

43

bot_OpFor2008Blitz0.270

259

bot_OpFor2008Fast0.244

246

bot_Loc2005P10.232

1455

bot_GnoBot2004CC0.227

44

bot_Clueless2009CC0.217

23

bot_Clueless2007Blitz0.187

235

bot_OpFor2009Blitz0.179

28

bot_Zombie2008Blitz0.176

301

bot_Zombie2008Fast0.174

282

bot_Loc2005Blitz0.167

269

bot_Clueless2008Blitz0.167

12

bot_Loc2005Fast0.159

226

bot_GnoBot2006Blitz0.146

82

bot_OpFor2009Fast0.145

55

bot_Loc2005CC0.140

121

bot_Marwin2010Blitz0.132

38

bot_Loc2005P20.126

538

bot_Loc2006Fast0.121

174

bot_Bomb2005Lightning0.104

547

bot_Sharp2010Fast0.100

10

bot_Loc2006Blitz0.094

160

bot_Clueless2007Fast0.092

272

bot_GnoBot2006CC0.091

44

bot_Clueless2006Fast0.090

155

bot_Loc2006CC0.088

91

bot_GnoBot2006P20.082

134

bot_GnoBot2006Fast0.082

98

bot_Clueless2009Fast0.079

152

bot_Loc2006P20.077

337

bot_Clueless2009Blitz0.074

149

bot_Clueless2006CC0.071

56

bot_Clueless2006P20.061

231

bot_OpFor2010Blitz0.059

17

bot_Bomb2004CC0.053

76

bot_OpFor2008CC0.053

38

bot_Marwin2010CC0.045

22

bot_Loc2006P10.044

1507

bot_Badger2010Blitz0.042

24

bot_Bomb2005CC0.041

412

bot_GnoBot2005CC0.039

103

bot_Marwin2010Fast0.037

27

bot_Aamira2006CC0.036

83

bot_OpFor2008P20.034

589

bot_GnoBot2005Fast0.034

414

bot_GnoBot2005Blitz0.027

548

bot_Loc2007Fast0.027

374

bot_GnoBot2005P20.024

1270

bot_Sharp2008CC0.024

41

bot_Sharp2008P20.024

1078

bot_Loc2007P20.024

927

bot_Loc2007P10.020

1664

bot_Bomb2005Fast0.019

1343

bot_Clueless2005CC0.018

219

bot_Bomb2005Blitz0.018

2289

bot_Loc2007Blitz0.017

350

bot_Clueless2005Blitz0.017

411

bot_Clueless2006P10.014

694

bot_Clueless2005Fast0.014

289

bot_ArimaaScoreP20.013

3549

bot_Arimaazilla0.013

4903

bot_Sharp2010Blitz0.010

98

bot_OpFor2008P10.010

1485

bot_Clueless2007P10.010

918

bot_Arimaazon0.009

552

bot_Bomb2005P20.009

1899

bot_GnoBot2006P10.008

666

bot_Clueless2005P10.005

1728

bot_Clueless2005P20.004

463

bot_Aamira2006Fast0.004

517

bot_ArimaaScoreP10.004

5725

bot_Bomb2005P10.004

4646

bot_Aamira2006Blitz0.004

549

bot_ShallowBlue0.003

3009

bot_GnoBot2005P10.003

2730

bot_Clueless2007P20.003

367

bot_Sharp2008P10.003

1529

bot_Arimaalon0.001

2268

bot_Aamira2006P20.001

1582

bot_Aamira2006P10.001

2831

IP Logged
Nombril
Forum Guru
*****



Arimaa player #4509

   
Email

Gender: male
Posts: 292
Re: 2010 Bot Ladder
« Reply #11 on: Sep 19th, 2010, 3:54pm »
Quote Quote Modify Modify

If the Bot Ladder is meant as a training tool, especially for new players, than I would emphasize variety of bots and time controls.
 
on Sep 18th, 2010, 7:04pm, aaaa wrote:
I simply can't fathom how it is possible for there ever being too many of them on the ladder.
Speaking as someone who just started Arimaa last fall, and used the bot ladder as a learning tool, I would not have liked a longer ladder.  There were times when I would make a leap of understanding, and then suddenly the next 5 or so bots were very easy to beat.  If doubling the number of bots on the ladder meant there were now 10-15 bots I have to beat in order to get to something stronger, I would have been very discouraged - out of boredom.  (5 easy games was bad enough...)  I'm OK with making the ladder longer if it is because we have now developed some stronger bots.  But I would not want the number of bots in rough ELO bins to get too high.
 
I think the Blitz bots were a good training tool, forcing me to learn to think/plan ahead.
 
I also think that having one CC bot at (or near) the top of the ladder could be a good way to make the ladder taller and add variety.  I certainly understand the concern of long games being a burden due to time constraints, but after having played thirty "quick" bots, would it would be possible for people to have the patience to play one CC bot?
IP Logged

clojure
Forum Guru
*****



Arimaa player #5004

   
WWW

Gender: male
Posts: 207
Re: 2010 Bot Ladder
« Reply #12 on: Sep 19th, 2010, 5:20pm »
Quote Quote Modify Modify

So, as encouraged to play the ladder with blitz bots, I gave a shot: http://arimaa.com/arimaa/gameroom/opengamewin.cgi?client=1&gameid=15 4975&role=v&side=b
 
See the move 28g by bot. Huh
 
Is it normal that the game ends as half of the board nearly empty and all the pieces in congestion in the other?
IP Logged
omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: 2010 Bot Ladder
« Reply #13 on: Sep 23rd, 2010, 6:27am »
Quote Quote Modify Modify

on Sep 18th, 2010, 7:24pm, Eltripas wrote:
I don't see any good reason to have limit the amount of bots in the ladder, in fact, for me, the more the merrier because I (and I'm pretty sure I'm not the only one)  use the ladder as a fast way to start a game, I find a little annoying the need to go to the top rated bots to start a game.

 
You are right I also often use the bot ladder as a quick way to get to the available bots. I added a new 'Bots Available' option under the 'Play Now' menu. You might need to click the 'Refresh' link to see it.
IP Logged
omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: 2010 Bot Ladder
« Reply #14 on: Sep 23rd, 2010, 6:31am »
Quote Quote Modify Modify

Thanks for posting this aaaa, it will be very helpful in removing from the ladder bots that could lead to a bad experience for new players.
 
on Sep 19th, 2010, 10:59am, aaaa wrote:
The following table shows the proportion of lost rated games that were due to a timeout (plus sample size) for each server bot for which this is a positive number:
 
bot_Clueless2006Blitz0.896

125

bot_Clueless2010Blitz0.778

9

bot_Rat2009Blitz0.667

3

bot_Clueless2010Fast0.625

8

bot_GnoBot2009Fast0.548

31

bot_PragmaticTheory2010Fast0.500

4

bot_Rat2009CC0.500

2

bot_GnoBot2009Blitz0.452

31

bot_OpFor2009CC0.375

8

bot_Occam2004CC0.354

48

bot_Zombie2008CC0.326

43

bot_OpFor2008Blitz0.270

259

bot_OpFor2008Fast0.244

246

bot_Loc2005P10.232

1455

bot_GnoBot2004CC0.227

44

bot_Clueless2009CC0.217

23

bot_Clueless2007Blitz0.187

235

bot_OpFor2009Blitz0.179

28

bot_Zombie2008Blitz0.176

301

bot_Zombie2008Fast0.174

282

bot_Loc2005Blitz0.167

269

bot_Clueless2008Blitz0.167

12

bot_Loc2005Fast0.159

226

bot_GnoBot2006Blitz0.146

82

bot_OpFor2009Fast0.145

55

bot_Loc2005CC0.140

121

bot_Marwin2010Blitz0.132

38

bot_Loc2005P20.126

538

bot_Loc2006Fast0.121

174

bot_Bomb2005Lightning0.104

547

bot_Sharp2010Fast0.100

10

bot_Loc2006Blitz0.094

160

bot_Clueless2007Fast0.092

272

bot_GnoBot2006CC0.091

44

bot_Clueless2006Fast0.090

155

bot_Loc2006CC0.088

91

bot_GnoBot2006P20.082

134

bot_GnoBot2006Fast0.082

98

bot_Clueless2009Fast0.079

152

bot_Loc2006P20.077

337

bot_Clueless2009Blitz0.074

149

bot_Clueless2006CC0.071

56

bot_Clueless2006P20.061

231

bot_OpFor2010Blitz0.059

17

bot_Bomb2004CC0.053

76

bot_OpFor2008CC0.053

38

bot_Marwin2010CC0.045

22

bot_Loc2006P10.044

1507

bot_Badger2010Blitz0.042

24

bot_Bomb2005CC0.041

412

bot_GnoBot2005CC0.039

103

bot_Marwin2010Fast0.037

27

bot_Aamira2006CC0.036

83

bot_OpFor2008P20.034

589

bot_GnoBot2005Fast0.034

414

bot_GnoBot2005Blitz0.027

548

bot_Loc2007Fast0.027

374

bot_GnoBot2005P20.024

1270

bot_Sharp2008CC0.024

41

bot_Sharp2008P20.024

1078

bot_Loc2007P20.024

927

bot_Loc2007P10.020

1664

bot_Bomb2005Fast0.019

1343

bot_Clueless2005CC0.018

219

bot_Bomb2005Blitz0.018

2289

bot_Loc2007Blitz0.017

350

bot_Clueless2005Blitz0.017

411

bot_Clueless2006P10.014

694

bot_Clueless2005Fast0.014

289

bot_ArimaaScoreP20.013

3549

bot_Arimaazilla0.013

4903

bot_Sharp2010Blitz0.010

98

bot_OpFor2008P10.010

1485

bot_Clueless2007P10.010

918

bot_Arimaazon0.009

552

bot_Bomb2005P20.009

1899

bot_GnoBot2006P10.008

666

bot_Clueless2005P10.005

1728

bot_Clueless2005P20.004

463

bot_Aamira2006Fast0.004

517

bot_ArimaaScoreP10.004

5725

bot_Bomb2005P10.004

4646

bot_Aamira2006Blitz0.004

549

bot_ShallowBlue0.003

3009

bot_GnoBot2005P10.003

2730

bot_Clueless2007P20.003

367

bot_Sharp2008P10.003

1529

bot_Arimaalon0.001

2268

bot_Aamira2006P20.001

1582

bot_Aamira2006P10.001

2831


IP Logged
Pages: 1 2 3  ...  5 Reply Reply Notify of replies Notify of replies Send Topic Send Topic Print Print

« Previous topic | Next topic »

Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1!
YaBB © 2000-2003. All Rights Reserved.