|
||
|
Title: Use of computers to assist analysis Post by Fritzlein on Nov 17th, 2007, 7:21pm We started discussing use of computers in the thread for Move 17, but I think it should have its own thread, because I would like it to come to a vote, and the discussion leading up to a vote should be easy to find. So far the arguments against using computers to analyze include: 1. We will get less mental exercise if we use computers to help analyze. 2. We could get an advantage in an endgame as computers discover forced goals that elude chessandgo. The arguments for include: 1. Chessandgo doesn't mind. 2. It's interesting to see what computers think even if they are way off base. 3. We could learn from the computers. We also have more than two options to put on the ballot in term of our attitude towards use of computers: 1. Total ban on using computers until after the game. 2. Unrestricted use of computers. 3. We can't use computers for any analysis, but we can see computer suggestions after we vote. 4. We can use computers to analyze, but only positions at least 3 moves old. 5. We can make unrestricted use of computers for the present, but we agree to re-vote after move 40 in case the end game is approaching and we feel differently. Please let me know of any options and arguments I have omitted. :-) |
||
|
Title: Re: Use of computers to assist analysis Post by Janzert on Nov 17th, 2007, 8:32pm As a 'mobster' I don't really care one way or the other. As a bot developer I'm interested in showing some of the moves the bot suggests in order to see the excellent analysis from the mob as to why the move is bad. Yes in other words, very selfish, hijacky reason. ;) Janzert |
||
|
Title: Re: Use of computers to assist analysis Post by UruramTururam on Nov 18th, 2007, 5:01am We should also take into account that in fact as long as human beings play Arimaa better than bots it's hard to check whether one uses a computer to analyze a position or not. Is there a reason to establish laws that could not be enforced? |
||
|
Title: Re: Use of computers to assist analysis Post by Fritzlein on Nov 28th, 2007, 9:50pm Ron, can we have this vote while we are waiting for chessandgo to move, and we therefore have nothing else to discuss? |
||
|
Title: Re: Use of computers to assist analysis Post by 99of9 on Nov 29th, 2007, 12:29am on 11/18/07 at 05:01:41, UruramTururam wrote:
We already establish such rules in nearly every tournament that gets run. Thankfully arimaa is still a small enough community with small enough prizes to rely on integrity. |
||
|
Title: Re: Use of computers to assist analysis Post by RonWeasley on Nov 29th, 2007, 8:33am I've been trying to reconcile my opnion on this. I can see both sides. When you consider that bots are currently intermediate players at best, their use should not contaminate the process. Are they really any different than an intermediate human player? Shame on TheMob if we rely too heavily on them. As it is, we rely heavily on the analysis of a small number of humans. Is it any different to have my candidate move analyzed by Fritzlein than by bomb? The main risk is that bots might show us goal threats that TheMob might otherwise miss. On the other hand, assuming you have exactly two, there is an asthetic appeal to a collection of humans matching wits with the world champion human. If we collectively miss a good move that a bot might have considered, we should accept our punishment and chessandgo can be congratulated for seeing what we didn't. I think there are good players genuinely curious about whether chessandgo can be defeated by the rest of us in a highly analyzed game. There's also the issue of what humans think a highly analyzed game looks like. I was thinking a big advantage to including bots, especially Janzert's, is that the developer can gain valuable insight comparing TheMob's analysis to the bot's. You can use any other previously played game for this, of course, but then the developer would have to get us to provide the detailed analysis. In this game the analysis is interactive. Here are the options so far in the order I favor them: 1) Allow only bots being actively developed to participate. Developers are encouraged to discuss differences between the bot's analysis and TheMob's as part of the bot's development. Bots not actively being developed are prohibited. 2) Bot analysis is prohibited. 3) Bot analysis is allowed. |
||
|
Title: Re: Use of computers to assist analysis Post by UruramTururam on Nov 29th, 2007, 9:01am on 11/29/07 at 00:29:30, 99of9 wrote:
I envy you your trust in people... (Really; without any sarcasm or irony.) I don't have one. :( |
||
|
Title: Re: Use of computers to assist analysis Post by Fritzlein on Nov 29th, 2007, 12:11pm I share UruramTuraram's general suspicion of human nature, particularly in the specific case of using computers even when they are against the rules. It is common practice to use computers to cheat on Internet chess servers, despite the server administrators taking steps to catch and punish cheaters. Also we have Adanac’s testimony as a postal chess player that he often wondered if his opponents were cheating, and found it a relief to play Arimaa because he didn’t care if his opponents used computer assistance or not. I also share 99of9’s trust in the Mob as presently constituted. If we agree to ban computer assistance, there is a high probability that we will all respect the ban. However, this probability declines for every new member that joins. Every growing community struggles to keep its norms; the larger a community gets, the less people can be expected to refrain from any prohibited behavior that is not enforced. If enforcement is difficult to impossible, then the banned behavior will become normal among some percentage of players. I expect that my interest in Arimaa will wane once computers surpass the best humans. World championships over the Internet will become meaningless, and even face-to-face tournaments will be plagued by accusations of cheating. Look what happened in the Kramnik-Topalov chess world championship for a demonstration of human nature in action. Topalov used Kramnik’s frequent trips to the bathroom as a basis for accusing him of getting computer assistance while in the bathroom. Even if Kramnik didn’t have a computer in his bathroom, he would have needed only a communication device to cheat, because now and forevermore there will be computers that play better chess than either of the two best humans in the world. Right now Arimaa is exciting partly because beginning humans know they will be able to surpass the best computers with patient study. It’s also exciting because computers are pretty good, and may reign within the foreseeable future. The “close but not quite” state of Arimaa is more interesting today than either chess, which is already decided, or Go, at which computers are not even close. The Arimaa Challenge must, by its nature, try to enforce that the human defenders play unassisted, no matter how difficult such enforcement is. If the Arimma Challenge isn’t pure man versus machine, it is nothing. For all other events, however, there is a balance between the value of prohibiting computers compared to the difficulty/cost of such a rule. This applies to the world championship, the postal championship, and the Mob game. For the human world championship, the value of prohibiting computers is high in my mind, but the difficulty is also enormous. Fortunately the enforcement doesn’t matter much this year, because the eight finalists will all be better than the best computer, but it does matter somewhat because Bomb can provide tactical assistance at live time controls, even to top players. There is pride and money on the line. If computers get closer to the top humans in skill in the coming years, it is inevitable that someone will eventually cheat, after which we will have to ask ourselves if we are indeed going to require Web cameras and/or human referees present for players to enter the human world championship. For postal games, meanwhile, enforcing the non-use of computers is simply impossible. You can’t monitor somebody all day. This negates most of the positive effect that a ban is trying to achieve. What if we ban computer assistance and then beat chessandgo? Will we have shown that a collection of humans can beat a single human? How will we know we didn’t benefit from computer analysis after all, introduced into the discussion without our knowledge? Yes, it would be nice to have a pure experiment, but no matter what we say, there will be some doubt. Meanwhile, like all unenforceable rules, a computer ban is a rule that punishes honest people only, because only honest people are constrained by it. Cheaters can use computers anyway, to their own fame and glory; the rule doesn’t deter them but it does reward them. Who among us thinks it is good to punish honesty and reward cheating? As an aside, there are those who would argue that using a computer punishes only the user, and that honest people get their own reward by learning how to think for themselves. If that's the reason for a rule, then the rule reflects an arrogant desire to protect undisciplined idiots from harming themselves. Note that chessandgo has no intention of using a computer even if allowed, but he has no objection to use of computers by the Mob. I respect and admire his liberality as much as his resolve. He doesn’t think it will be useful to consult computers, but allows those with a different opinion to behave differently. But let me leave the “you’re only hurting yourself” argument aside, and assume that computer analysis will be beneficial, on the whole, in helping the Mob choose a better move. The help should be minimal except in endgame positions, but it will be a small net positive in any position, because there are some ways computers already surpass us. This is, in my mind, an argument in favor of allowing computer use. Because of the “close but not quite” abilities of computers relative to humans, I am eager for every opportunity for comparison. I want to know more about the hidden strengths and hidden weaknesses of the bots. Could a supercomputer running Bomb beat top humans? We don’t know, but a postal game is an excellent time to gain insight on that question, especially since it is a time when we can’t prevent supercomputer usage. Are endgames the only strength of current computers relative to humans, or are they also able to outplay us in wild tactical midgame positions? I think they can’t, but again, postal games are an excellent opportunity to investigate. I understand that banning computers would be an attempt to achieve a certain type of game, and if that is what the Mob wants, I'll go along with the majority. However, to summarize my reasons against a ban, since banning computers is unenforceable, it removes most of the positive effect of a ban. A ban also has the negative effect of punishing honesty relative to dishonesty. Meanwhile there are positive effects of allowing computers, in terms of improving our understanding of Arimaa, as well as our understanding of our abilities relative to machine abilities. |
||
|
Title: Re: Use of computers to assist analysis Post by 99of9 on Nov 29th, 2007, 3:28pm Are you proposing the same for the Postal Tournament? |
||
|
Title: Re: Use of computers to assist analysis Post by Fritzlein on Nov 29th, 2007, 8:43pm Yes, I would like folks to be able to use computer assistance in the Postal Tournament if they wanted to, and would only have the rules request that each entrant say whether they were getting computer assistance or not. There should be little incentive for denying the use of computers if using computers is allowed by the rules. Maybe folks will like to appear smarter than they are by using computers while denying it, but of course that would be an incentive whether or not computers are banned. Of course, many people, probably the majority, will prefer to play without computer assistance, which is fine. The tournament would likely remain more or less a human championship unless bots improve a lot from their current state. It's just that more experimentation would be allowed and encouraged in the mean time. Also I would like to substantially ease restrictions on computer players. Specifically there should be no limit on hardware strength, and frequent code changes and time control changes should be allowed. Thus there would be wider latitude to still be considered a "pure computer" acting without human assistance, even if the code was often changing. And again, if a developer didn't want to abide by some minimal restrictions on code changes, they could enter their computer as a cyborg player. If we allow humans assisted by computers, naturally we would allow computers assisted by humans, and would request only that the entrants declare which category they are in. So I guess I am suggesting a "freestyle" Postal Tournament, where you are allowed to say whether you are human, computer, or cyborg, with only guidelines as to the boundary between computer and cyborg, but no real incentive for anyone to be dishonest about which class of player they are. That's a separate discussion, however. We could decide to allow computer use for the Mob and ban it for the Postal Tournament, or vice versa. Although I guess for the Postal Tournament, there is no vote, just a decision by Omar after he listens to our input. |
||
|
Title: Re: Use of computers to assist analysis Post by arimaa_master on Nov 30th, 2007, 6:19am on 11/18/07 at 05:01:41, UruramTururam wrote:
I am for unrestricted computer use since the rule of not using computer couldn´t be enforced (like UruramTururam wrote). For any time frame and game type (postal, mob game, anything). |
||
|
Title: Re: Use of computers to assist analysis Post by RonWeasley on Nov 30th, 2007, 8:34pm I propose we vote on this beginning Monday morning. The voting period can be several days long. I'll notify in this thread when it will stop. Not that it should take long to vote, but it will give interested parties time to check their email if they don't do that every day. |
||
|
Title: Re: Use of computers to assist analysis Post by 99of9 on Dec 3rd, 2007, 6:03pm on 11/17/07 at 19:21:15, Fritzlein wrote:
What happened to these other options? I think some of them are quite good! |
||
|
Title: Re: Use of computers to assist analysis Post by RonWeasley on Dec 4th, 2007, 7:58am Oops, forgot about these. That's not so bad since there's no time limit on this decision. We can always decide to change things later. Let's see what this vote gives us and decide what to do from there. If the unrestricted use option wins, the others don't matter, unless there are mobsters who would have chosen 3 or 4 over their vote for unrestricted use. Such mobsters should make themselves heard and we can reconcider if there are enough pitchforks. |
||
|
Title: Re: Use of computers to assist analysis Post by Fritzlein on Dec 4th, 2007, 12:22pm I think anyone on either extreme has a beef if their extreme loses while the middle ground wasn't an option, because not everyone will order the choices from most restrictive to least or vice versa. Some people, like you Ron, have a middle option first. True, the result may be overwhelming anyway, but to err on the side of caution, if even one person mentions they are voting "unrestricted use of bots" first but would rather vote partial restrictions first, then I recommend restarting the vote with their preferred option included. There's no hurry to get this vote done, so we might as well get it right. By the way, Ron, you are doing a great job as Mob Coordinator. |
||
|
Title: Re: Use of computers to assist analysis Post by RonWeasley on Dec 5th, 2007, 8:42am Gosh, Fritzlein, :-[. Put that picture of McGonagle away before you get caught. Anyway, I'm satisfied that the mobsters are participating in the original spirit of the contest. The vote on three options was won by the unrestricted use of bots choice by a 10-4 margin over the others. This looks like a mandate. Let me ask, would anybody who voted for unrestricted use choose one of the qualified use options I omitted from this vote? If there's three of you, we MUST vote again with the expanded choices. If less than three, we can vote again if you ask (nicely). Next question: Anybody think we should vote with the expanded choices anyway? It's OK with me. It might be fun, but I don't want to be spamming people with too many votes. We'll do it if mobsters say they want it. If we don't get a call for a revote by Friday, I will declare unrestricted use of bots to commence. Remember, TheMob can decide to change this policy any time in the future. I think chessandgo would benefit from seeing this discussion. I should have thought of this before, but I think he should be encouraged to read and comment in this topic. Any objections? |
||
|
Title: Re: Use of computers to assist analysis Post by Fritzlein on Dec 5th, 2007, 9:52am OK, I admit that was a decisive preference, and the other options were in all likelihood irrelevant. Also 14 votes is more than we get in a typical election on a move, which makes the result all the more solid. Please pardon my pushing for an unnecessary restart. For the record: . . . . . . . . 1. 2. 3. 1. Allow unrestricted use of bots. . . - 10 10 2. Allow bot analysis only by its developer. 4 - 8 3. Prohibit any use of bots. . . . 4 3 - Now I am interested to see whether this vote makes any difference at all. Mostly I expect we will point out what bots think only so that we can laugh at them. Perhaps we won't even think it is worth the trouble to consult bots. However, I expect that we will use computers somewhat, and that there will be some moves we waste less time on because computers refute them instantly, and other moves we pay more attention to because some bot likes them. Just last night I spent about twenty minutes on my postal move against camelback before seeing a three-move forced horse win. This morning, because this vote made me wonder if computers will make any difference in postal games, I fed the position to Bomb. Bomb's quiescence search rocks; it found the material win at 8-steps search depth, i.e. in three seconds. So Bomb would have saved me a lot of time if I had used it, although it wouldn't have changed my move. More importantly, if camelback had used Bomb it would have saved him a horse blunder. I don't think either side in the Mob game has blundered tactically so far, but maybe I just think that because we haven't been using computers! Perhaps this Mob decision will be more momentous than I expect... |
||
|
Title: Re: Use of computers to assist analysis Post by Janzert on Dec 5th, 2007, 2:13pm on 12/05/07 at 08:42:47, RonWeasley wrote:
I had been sort of wishing that chessandgo had been able to see the topic as well. Can you as a moderator or perhaps Omar simply move it to the general discussion forum? Janzert |
||
|
Title: Re: Use of computers to assist analysis Post by The_Jeh on Dec 5th, 2007, 3:38pm Well, we did agree a long time ago that he could look at all threads except "Move X" ones. |
||
|
Title: Re: Use of computers to assist analysis Post by Fritzlein on Dec 5th, 2007, 5:25pm on 12/05/07 at 15:38:43, The_Jeh wrote:
I agree, John. I think we should invite him to read the thread in place. After all, he is only on his honor not to read the "Move X" threads anyway; if he hasn't been overwhelmed by temptation to read those threads yet, I doubt he will give in just because he gets one step closer. |
||
|
Title: Re: Use of computers to assist analysis Post by RonWeasley on Dec 5th, 2007, 7:20pm Wow, I thought I posted essentially what the last two posts said, but I don't see it. I guessed I messed up or used disappearing letters or something. I will invite chessandgo to read this topic through the game window. Others are free to inform him through other means. |
||
|
Title: Re: Use of computers to assist analysis Post by chessandgo on Dec 9th, 2007, 11:31am Thanks Ron, I just read this thread. I like the fact that Janzert (and other developpers if they want to) will be able to develop their bot with the help of the Mob analysis. This said, you guys are going to crush me without computer's help it seems ; my last hope is that Bomb's analysis will trouble you ::) Joke apart, have fun for the rest of game ! Jean |
||
|
Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1! YaBB © 2000-2003. All Rights Reserved. |