Welcome, Guest. Please Login or Register.
Mar 26th, 2019, 9:39am

Home Home Help Help Search Search Members Members Login Login Register Register
Arimaa Forum « Arimaa games shared database »


   Arimaa Forum
   Arimaa
   Bot Development
(Moderator: supersamu)
   Arimaa games shared database
« Previous topic | Next topic »
Pages: 1 2 3  4 Reply Reply Notify of replies Notify of replies Send Topic Send Topic Print Print
   Author  Topic: Arimaa games shared database  (Read 16659 times)
foggy
Forum Full Member
***



Arimaa player #6010

   


Gender: male
Posts: 13
Arimaa games shared database
« on: Mar 7th, 2011, 11:15am »
Quote Quote Modify Modify

Hi all!
 
I was (and still am) curious to see various statistics and data  of the whole Arimaa games database. It is possible to find some bits of data in the forum, like game count, move count, gold vs silver stats, openings stats and so on. However, these bits are often outdated and sparsed between various threads.  
 
My first impulse was to download everything, write a parsing program, save all data (maybe minus irrelevant pieces, like pings and  times) in more convenient format, then use another program to go thru the data to find information I want. Or to feed data into some relational database  and use SQL queries to get info.
 
I guess that somebody in Arimaa community could have already done something like that. Would it be possible to share processed game data? In that case Arimaa  enthusiasts could drive their effort into making/producing/learning something new instead of reinventing the wheel.
 
If this is not yet done I could volunteer to do the job and try to produce some data to share with everybody, as my time and energy (much more limited that I would prefer, unfortunately) permits.
 
Another open question is what format would be the best to keep games in reachable and convenient format. Criteria that came to my mind are: ease to query by human, human readability (for checking reasons),  occupied space on disk, ease to query by program (machine learning and so on).
IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Arimaa games shared database
« Reply #1 on: Mar 7th, 2011, 1:39pm »
Quote Quote Modify Modify

on Mar 7th, 2011, 11:15am, foggy wrote:
Or to feed data into some relational database  and use SQL queries to get info.[...]  Would it be possible to share processed game data?

I have done this.  What do you want to know?  My inspiration for queries petered out a while ago; I would be interested if you have new ideas for what we should query.
 
Quote:
Another open question is what format would be the best to keep games in reachable and convenient format. Criteria that came to my mind are: ease to query by human, human readability (for checking reasons),  occupied space on disk, ease to query by program (machine learning and so on).

Oh, is what you are asking for a game database in a different format than the flat text file Omar offers?  I'm afraid the demand for that is both low and fragmented.  That is to say, given any two people who want an alternate-format game database, they will consider different bits of the data relevant and/or want those bits of data presented in a different format.  There aren't enough people who want the data to create two people who want it in the same form.  The flat text file Omar offers, clunky though it is, is at least universal.
IP Logged

ocmiente
Forum Guru
*****




Arimaa player #3996

   
WWW

Gender: male
Posts: 194
Re: Arimaa games shared database
« Reply #2 on: Mar 8th, 2011, 7:46pm »
Quote Quote Modify Modify

I have a query for you.  
 
What game first used any of these setups for gold or silver:
Code:

 r r r r c r r r  
 h h d e c r m d
 . . X . . X . .  
 . . . . . . . .  
 . . . . . . . .  
 . . X . . X . .
 H H D E C R M D
 R R R R C R R R
 
 r r r c r r r r  
 d m r c e d h h
 . . X . . X . .  
 . . . . . . . .  
 . . . . . . . .  
 . . X . . X . .
 D M R C E D H H
 R R R C R R R R

 
That is, the first game in which either the gold or the silver pieces were setup like either of the positions on these two example boards.  Hope that makes sense.  
 
The reason I ask is that I was curious about this a while ago, and I think I know the answer.  I'm curious to know whether you can confirm this with your database.  
 
IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Arimaa games shared database
« Reply #3 on: Mar 8th, 2011, 7:59pm »
Quote Quote Modify Modify

Ah, I can try, but this is hard to do with an SQL query because the pieces of the setup can be in any order.  That is to say "Ra1 Rb1 ..."  is equivalent to "Rb1 Ra1 ..."  I suppose this speaks to the general issue of everyone wanting a database in a different format depending on what they want to query from it.  I imagine that woh could answer this question easily from his database that he used to analyze openings.  Still, I will give it a shot.  Smiley
« Last Edit: Mar 8th, 2011, 8:02pm by Fritzlein » IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Arimaa games shared database
« Reply #4 on: Mar 8th, 2011, 10:11pm »
Quote Quote Modify Modify

OK, that was a fun exercise in using SQL in ways it was not intended.  Now that I have made and run the four queries, I can guess your motivation for asking the question.  Only three people have used this setup: mojyaschik (twice); ocmiente (over a dozen times); and rabbits (over a hundred times).  Actually, I didn't update my database past January, so the setup may have been used a lot since then, but your question was about the first use of the setup, so it doesn't matter.
 
I infer that there is a priority dispute between you and rabbits as to who used the setup first, and who copied whom.  Even as I say it, I realize the argument is a straw man; you might each have come up with the setup independently.  But be that argument silly or no, you lose.  The first use of this setup was on September 4, 2010, in game 153540, by rabbits as Silver.  The second use was on October 9, 2010, in game 156670, by ocmiente as Gold.  That is to say, a month after rabbits first used the setup, you used it before he used it a second time.  Thereafter both of you started using it much more frequently, i.e. there was a much shorter gap between uses.
 
Creating lots of queries of this exact type would be tedious, but doing one was actually rather fun as it challenged me to think how it could be efficiently done with just SQL and no "programming".  I'm actually very impressed the that the MS Access query engine didn't barf on my spaghetti query over the whole Arimaa game database, but instead returned an answer in less than a minute.  Does anyone else want to know anything?  Smiley
« Last Edit: Mar 8th, 2011, 10:35pm by Fritzlein » IP Logged

ocmiente
Forum Guru
*****




Arimaa player #3996

   
WWW

Gender: male
Posts: 194
Re: Arimaa games shared database
« Reply #5 on: Mar 8th, 2011, 10:22pm »
Quote Quote Modify Modify

Very good!  I think you're correct, though I left the answer on a different machine - I'll have to come back and edit this tomorrow if I'm mistaken.  I figured out the answer using C# code rather than SQL.
 
on Mar 8th, 2011, 10:11pm, Fritzlein wrote:
OK, that was a fun exercise in using SQL in ways it was not intended.  Now that I have made and run the four queries, I can guess your motivation for asking the question....
 
I infer that there is a priority dispute between you and rabbits as to who used the setup first, and who copied whom. ...

 
Well, not exactly.  rabbits knows very well that I stole the setup from him Smiley  And I know it too.  (though when I initially borrowed it, the camel and dog were in different places)
 
I do think that this is one of the best setups out there, and that it deserves to be named officially, "rabbits' setup".
 
 
« Last Edit: Mar 8th, 2011, 10:57pm by ocmiente » IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Arimaa games shared database
« Reply #6 on: Mar 8th, 2011, 10:28pm »
Quote Quote Modify Modify

on Mar 8th, 2011, 10:22pm, ocmiente wrote:
Well, not exactly.  Rabbits knows very well that I stole the setup from him Smiley  And I know it too.  
 
I just think that this is one of the best setups out there, and that it deserved to be named officially, "Rabbit's setup".

Ahhh, so you just wanted to make sure no one else had used it prior to the two of you, before crediting it to rabbits.  Very generous of you.  Your hunch is confirmed.
 
In general, though, I don't think you need to be the first to use an opening to have it named after you.  It might be that the first user was just messing around, didn't see the power of the idea, and never came back to it.  To get your name attached to something, you need to believe in it enough to explore the consequences somewhat deeply.  This could be rabbits' setup even if he wasn't the first.
 
On a more general note, I think it is marvelous that opening theory of Arimaa remains so fractured.  A priori, one might have predicted that by now all Arimaa players would have agreed on a single best setup for Gold and a single best response for Silver.  The fact that this is further from happening today than it was five years ago is a very good sign.
« Last Edit: Mar 8th, 2011, 10:38pm by Fritzlein » IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Arimaa games shared database
« Reply #7 on: Mar 8th, 2011, 11:08pm »
Quote Quote Modify Modify

Footnote: for the month of February, two more players were persuaded to try this setup: Better (once) and OffbeatJM (three times).  Rabbits used it four times and ocmiente fourteen times.  I guess when rabbits beat both chessandgo and Adanac at the end of January with this setup, at least a couple of people took notice.
IP Logged

foggy
Forum Full Member
***



Arimaa player #6010

   


Gender: male
Posts: 13
Re: Arimaa games shared database
« Reply #8 on: Mar 9th, 2011, 2:33am »
Quote Quote Modify Modify

on Mar 7th, 2011, 1:39pm, Fritzlein wrote:

I have done this.  What do you want to know?  My inspiration for queries petered out a while ago; I would be interested if you have new ideas for what we should query.

 
Well, I would be interested in many queries, but I'm afraid these would present nothing new for "veterans" yet. I play Arimaa just 2 months, so everything is new, exciting and interesting for me.  
 
The initial idea was to get the data in some way (myself or from somebody else if I'm lucky) and have possibility to "swim in it" as needed, without bothering somebody to run/present the results. And to extend such a possibility to other community members also. Fritzlein, would you care to share the data you gathered in some common format(txt,csv,xls or other)?
 
Another idea was to gather and present all the statistics that could be interesting to community in one place, with occasional renewal as new data(games) arrive.
« Last Edit: Mar 9th, 2011, 2:34am by foggy » IP Logged

foggy
Forum Full Member
***



Arimaa player #6010

   


Gender: male
Posts: 13
Re: Arimaa games shared database
« Reply #9 on: Mar 9th, 2011, 2:51am »
Quote Quote Modify Modify

on Mar 8th, 2011, 7:59pm, Fritzlein wrote:
Ah, I can try, but this is hard to do with an SQL query because the pieces of the setup can be in any order.  That is to say "Ra1 Rb1 ..."  is equivalent to "Rb1 Ra1 ..."  I suppose this speaks to the general issue of everyone wanting a database in a different format depending on what they want to query from it.  I imagine that woh could answer this question easily from his database that he used to analyze openings.  Still, I will give it a shot.  Smiley

 
Speaking about different demands - I suppose we can find some common format that suits best for most of the people and share the effort of creating/maintaining the data.
 
Another idea of coming up with this thread was to launch discussion on  possible game information formats that would allow faster/easier data retrieval. For example, how initial setup data could be kept (maybe a field for rabbit formation and 8 fields for other pieces or have a flag/field for some common rabbit formations). Also rabbits can be presented by 16 bitfield, which can be treated as unsigned number for brief reference.
 
For initial setup analysis (and maybe board positions set analysis) I think "normalization" is needed, so that gold elephant is always on west or east, transforming position/setup with vertical mirror if needed. This effort could also be done only once and shared.
IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Arimaa games shared database
« Reply #10 on: Mar 9th, 2011, 9:37am »
Quote Quote Modify Modify

on Mar 9th, 2011, 2:33am, foggy wrote:
Fritzlein, would you care to share the data you gathered in some common format(txt,csv,xls or other)?

Sharing my database as txt would just convert it back to Omar's format with a couple of fields missing.  My only value-add was putting everything into MS Access.  You don't want MS Access to become the standard database format for the purpose of your project, do you?  And are you sure nobody wants the fields I left out, just because I didn't want them?  Wink  (If you are just thinking how to best get the data yourself, and MS Access with a few missing fields suits your needs better than Omar's text files, I'm sure we could arrange a file transfer somehow.)
 
Quote:
Another idea was to gather and present all the statistics that could be interesting to community in one place, with occasional renewal as new data(games) arrive.

One very cool instance of your suggestion that already happened was the ratings graph for each player.  I use this feature constantly, and I think I am not the only one.  A ratings history is something that I could query for myself any time I really wanted to know, and in the old days I posted a bunch of graphs for selected players, but as cool as it was I got tired of doing it.  Now everyone has it at their fingertips a couple of clicks away, which rocks.
 
What are some of the other things you want to know?  Maybe if you started listing some of those concretely, other people (including Omar) might realize that they want to know too.  (Not to put too fine a point on it, but nobody jumped up and said they would write their own queries against the common data store you are proposing).  If there were a particularly cool specific stat that garnered interest, maybe someone would automate it against Omar's database, as happened for the ratings graph.  Concrete suggestions for "stats that it would be cool to know" might draw supporters out of the woodwork, even if the stats are ones that veterans investigated long ago but stopped producing because it was too much hassle to do manually all the time.
« Last Edit: Mar 9th, 2011, 9:40am by Fritzlein » IP Logged

mistre
Forum Guru
*****





   


Gender: male
Posts: 553
Re: Arimaa games shared database
« Reply #11 on: Mar 9th, 2011, 11:37am »
Quote Quote Modify Modify

The bot bashers page -http://arimaa.com/arimaa/twiki/bin/view/Arimaa/BotBashers - has a lot of neat statistics for human-bot games.  However, it has not been updated in a while and I am not sure there is interest anymore.
« Last Edit: Mar 9th, 2011, 11:37am by mistre » IP Logged

ocmiente
Forum Guru
*****




Arimaa player #3996

   
WWW

Gender: male
Posts: 194
Re: Arimaa games shared database
« Reply #12 on: Mar 10th, 2011, 4:25pm »
Quote Quote Modify Modify

I would be interested to see what kind of database you could come up with to make queries easier.  I am definitely interested in some of the statistics.  
 
I've been writing queries in a rather crude way - just reading in the raw data and using C# to analyze it.  My concern with an SQL database is that it might not be flexible enough to do everything everyone would like, and one that is flexible enough will be too complicated for anyone to understand Smiley
 
For instance, I was interested recently to know what the most common openings for gold and silver were, accounting for symmetry (i.e don't count a setup twice if it's just a mirror or another setup).  
 
Code:

Gold......................Silver.
Count...Setup.............Count...Setup.
25945...R H D E M D H R...20129...r h d e m d h r
........R R R C C R R R...........r r r c c r r r
18727...R H C E M C H R...16769...r h c e m c h r
........R R R D D R R R...........r r r d d r r r
13805...D H C E M C H D...13630...d h c e m c h d
........R R R R R R R R...........r r r r r r r r
11239...H D C E M C D H...10337...h d c e m c d h
........R R R R R R R R...........r r r r r r r r
 4464...R D H E M H D R... 5955...r m d e h d h r
........R R R C C R R R...........r r r c c r r r
 4287...R H D E H D M R... 3906...r d m e h h d r
........R R R C C R R R...........r r r c c r r r
 4166...R D H E M D H R... 3809...r d m e h d h r
........R R R C C R R R...........r r r c c r r r
 4060...R H D E M H D R... 3795...r m d e h h d r
........R R R C C R R R...........r r r c c r r r
 2266...C H D E M D H C... 2997...r h d e h d m r
........R R R R R R R R...........r r r c c r r r
 1900...R H C E H C M R... 2275...c h d e m d h c
........R R R D D R R R...........r r r r r r r r
 1773...R H C E M D H R... 2167...r m c e h c h r
........R R R D C R R R...........r r r d d r r r
 1686...R M C E H C H R... 1808...r h c e h c m r
........R R R D D R R R...........r r r d d r r r
 1354...R H C E M D H R... 1353...r h c e m d h r
........R R R C D R R R...........r r r d c r r r
 1222...R M D E H D H R...  874...h r c e m c r h
........R R R C C R R R...........r r r d d r r r
 1215...R H D E M C H R...  850...r h d e m d h r
........R R R C D R R R...........r r c r r c r r
 1038...R H R E M R H R...  813...r h r e m r h r
........R D R C C R D R...........r d r c c r d r
 1011...R H D E M C H R...  775...r h d e m c h r
........R R R D C R R R...........r r r c d r r r
  929...R H R E M R H R...  679...c d h e m h d c
........R C R D D R C R...........r r r r r r r r
  906...R H D E M D H R...  644...r h c e d c h r
........R R C R R C R R...........r r r d m r r r
  810...C D H E M H D C...  637...d h c e c d m h
........R R R R R R R R...........r r r r r r r r

 
 
Sorry about all of the '.'s in there.  It's my brain-dead way of trying to make sure the columns align properly.  Spaces weren't working for me.
 
This was based on the setups from 173,001 games. Some games weren't accounted for due to setup data that wasn't easily read (and I just mean that it was easier for me not to write the extra code to handle it than to handle it), like positions with a 'takeback' used, or incomplete data.  
 
What I thought was interesting about this was the 4th most popular setup for gold that has the horses on either end, because I don't think I've tried that one.  I suppose now I should run another query to see if I have played this before.  Is there a bot that uses that setup all of the time?  Do people use it very often?  More queries needed..
« Last Edit: Mar 10th, 2011, 4:36pm by ocmiente » IP Logged

ocmiente
Forum Guru
*****




Arimaa player #3996

   
WWW

Gender: male
Posts: 194
Re: Arimaa games shared database
« Reply #13 on: Mar 10th, 2011, 4:53pm »
Quote Quote Modify Modify

Here's another question I'd like an answer for:  
How many Human vs. Human games have been played on the site?  My first shot at figuring this out came up with 8416, but that seems low.  
 
The database should also be able to distinguish between Human vs. Human games, bot vs. human and bot vs. bot.
 
IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Arimaa games shared database
« Reply #14 on: Mar 10th, 2011, 5:16pm »
Quote Quote Modify Modify

on Mar 10th, 2011, 4:25pm, ocmiente wrote:
What I thought was interesting about this was the 4th most popular setup for gold that has the horses on either end, because I don't think I've tried that one.  I suppose now I should run another query to see if I have played this before.  Is there a bot that uses that setup all of the time?  Do people use it very often?  More queries needed.

That setup was the default in the client from very early on when it seemed like a good idea, i.e. after figuring out that rabbits forward was bad but before figuring out that horses on the corners is good.
 
Also 8,000 HvH games seems (sadly) about right to me.  So far HvB games are dominating the game room.  For queries on opening theory, I am interested only in HvH games.  In fact, for most areas of investigation, I am interested only in HvH games.
« Last Edit: Mar 10th, 2011, 5:17pm by Fritzlein » IP Logged

Pages: 1 2 3  4 Reply Reply Notify of replies Notify of replies Send Topic Send Topic Print Print

« Previous topic | Next topic »

Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1!
YaBB © 2000-2003. All Rights Reserved.