Arimaa Forum - Print Page


    
      
        Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Site Discussion >> New website & new client
        
(Message started by: Brendan_M on Nov 9^th, 2014, 7:48am)

Title: New website & new client
Post by Brendan_M on Nov 9^th, 2014, 7:48am

I've finally gotten around to starting to write a new website for arimaa - yay!

It's going to be a single page web app. I will put all the code on github and hope for many pull requests

Before I do that I want at least some basic functionality:
- login
- chat
- list of games
- ability to open the current game client for a game

Unfortunately the public protocol (http://arimaa.com/arimaa/protocol/) is incomplete as it has no chat. Another problem is that the server doesn't return CORS headers so I've had to implement a basic server that just forwards requests to arimaa.com

In order to get around the incomplete protocol problem I reviewed the network traffic on the website. It is actually making requests to http://arimaa.com/arimaa/gameroom/client2gr.cgi and http://arimaa.com/arimaa/chat/srv.php neither of which are documented. When I replicate the chat request I get no data response at all - I suspect there is some domain security but am not sure

Does anyone know anything about this or any alternative protocols?

btw. My second signon is arimaaweb. You may see me messing around on the site

Title: Re: New website & new client
Post by Fritzlein on Nov 9^th, 2014, 1:39pm

How does this compare to what rabbits is (or has been) working on?

Title: Re: New website & new client
Post by Brendan_M on Nov 9^th, 2014, 2:28pm

I hadn't seen that rabbits had been working on something before. I've found that thread now but the link that he had up doesn't seem to be working - http://arimaaland.com/

It would be great to work from some static html if I could get my hands on it - I don't have much experience of web design, am much better as a programmer

Do you think I should contact Omar directly about any technical issues I am having?

Title: Re: New website & new client
Post by lightvector on Nov 9^th, 2014, 11:02pm

You should probably contact Omar. I've contacted Omar about this sort of stuff in the past when I was considering trying to add online gameroom functionality to the android Arimaa app that I released. At least for me, he was quite helpful and open to the idea.

Unfortunately, I set that project aside due to some combination of getting bogged down and confused in the not-so-well-documented protocols and having too any other (non-arimaa) things that I wanted or needed to do.

Title: Re: New website & new client
Post by Janzert on Nov 10^th, 2014, 9:05am

At least regarding the gameroom chat I don't believe there is any domain check. You do have to include a valid user and auth token with the request. My simple script to check that weekend tournament participants are present is at https://github.com/Janzert/tournament_tools/blob/master/chat_check.py.

There should also be a forum thread around here discussing the initial implementation of the chat, although that didn't include the user authentication.

Janzert

Title: Re: New website & new client
Post by Brendan_M on Nov 10^th, 2014, 4:34pm

I worked out why it wasn't working for me. I was using the sid from protocol version 1 as the value for auth. Instead the value that should be used is glob.me.auth from the gameroom. This is retrieved using the http://arimaa.com/arimaa/gameroom/client2gr.cgi url using the following form-data as a param!
{"sid":"**********","action":"state"}

Unfortunately, however much I try I cannot avoid getting a Internal Server Error when I try to replicate this request

Time to go direct to Omar

Title: Re: New website & new client
Post by aaaa on Nov 10^th, 2014, 7:54pm

In the chatroom, rabbits has been making occasional noise about his new website http://arimaaden.com/ this last few months and has even offered access to his source repository to others. You'll have to delve into the chat archive for details, but I think you should seriously consider teaming up with him.

Title: Re: New website & new client
Post by rabbits on Nov 18^th, 2014, 5:08pm

Hi Brendan! I'm glad to see that someone else is serious about working on the site! I'm curious about what technologies you're using. In the chatroom, someone mentioned Clojure. And your comment about a single-page site hints at Angular.js or something similar.

My site is built on the Node.js platform with the Express.js web framework. That is, it uses server-side Javascript. It stores sessions in Redis, and its main database is MongoDB (through Mongoose). It uses Socket.io for WebSockets, and it uses Angular.js somewhat. (I am not very comfortable with Angular.js yet.)

I haven't worked on my site in a few weeks now. If someone else starts working on it actively, I will be more motivated to keep working on it myself. I'm currently on my third go at it. The Arimaaland link you posted in this thread is from my attempt using Google's Go language. I started with Node.js last December, and ran out of time to work on it in February. And in September, I started again from scratch with test-driven development. The second version had more features, but the current version is built better. I have a continuous integration site set up at http://testing.arimaaden.com

I see that you intend to match the protocols that Arimaa.com currently uses. I am planning on more or less developing new interfaces (inspired by AEI).

If you post your github link, I would happily pull your code and look at it. I felt the same way about not wanting to share my code before I had made sufficient progress. I can give you access to my (Bitbucket) repository if you want it.

Rabbits

Title: Re: New website & new client
Post by Brendan_M on Nov 18^th, 2014, 6:25pm

Hi Rabbits!

Wow, based on your tests you've got a lot further than me.

So far I've just been messing around with the existing APIs and putting the data on a basic website.

My plan was just to reuse the existing arimaa server and put a nicer front-end on it. I didn't want to write a server side at all but resorted to it in order to handle web security. I had a minor brainwave over the weekend and realised I can create a domain record that will route all requests to arimaa.com and can revert to client side only again

Are you writing an entirely new backend or wrapping the current APIs in a nicer protocol?

Whoever was in the chatroom read my mind. I'm using clojurescript because I want to learn it and as it has 2 libraries I massively rate:
- core.async - allows writing asynchronous code much more clearly than using callbacks
- reagent - my feeling is that facebook's react is a better framework than angular. Reagent wraps that up nicely for clojurescript and seems incredibly simple to use

I've created a github repo just now as a placeholder https://github.com/brendanator/arimaa
There's nothing in it right now but hopefully will be soon.
I would certainly like to check out your repo too

Hopefully we can work out a way of joining forces and doing this thing!

Brendan

Title: Re: New website & new client
Post by rabbits on Nov 18^th, 2014, 6:50pm

I hadn't even heard of React. It looks cool!

I like your idea of just rewriting the front-end and using the same back-end.... My strategy is to scrape the entire site and import all of that public data into the new site.

If you send me your email address in a private message, I'll give you access to my Bitbucket repository.

Title: Re: New website & new client
Post by Brendan_M on Nov 19^th, 2014, 1:12pm

I've sent you my email address in a private message. I hope it gets through - I sent you a private message last week but I'm not sure it got through

I think there definitely does need to be a new backend but that was too much for me to try and bite off to start with so that's why I just wanted to work on a new front-end.

I love your github profile pic!!

Title: Re: New website & new client
Post by rabbits on Nov 19^th, 2014, 10:05pm

I sent an invite to the Bitbucket repository!

My Github icon is my Gravatar, which is what I use on this forum too. Oh, and Gravatars are already somewhat supported / used in my Arimaa Site!

Title: Re: New website & new client
Post by Brendan_M on Dec 1^st, 2014, 2:51pm

I've tried contacting Omar but I haven't had any response. I don't know if the email address associated with the private message is still in use.

Does anyone have an email address that they know works for Omar or alternatively does anyone have access to the server to make a small change for me and to look at some server logs?

Or even better would I be able to get access to do these things?

Title: Re: New website & new client
Post by omar on Dec 2^nd, 2014, 7:59am

Hey Brendan, check your PMs.

Title: Re: New website & new client
Post by leo on Jan 9^th, 2015, 8:00pm

Hi Brendan, send me your email address in a PM, I can help with the chatroom protocol if needed.

Title: Re: New website & new client
Post by Brendan_M on Jan 17^th, 2015, 5:15am

Hey, thanks Leo, I've sent you an instant message
If anyone is interested the site can be seen at http://arimaa.club
And the code is at https://github.com/brendanator/arimaa

Any suggestions or pull requests are welcome
My rate of progress will definitely be slower during the world championships but it should pick up afterward

Title: Re: New website & new client
Post by leo on Jan 26^th, 2015, 6:20pm

Thanks, Brendan! As I told you by email, your project looks very promising, let's try to gather strengths to complete it.

As you know, I am currently doing some fixes to the chatroom (which are starting to look like a complete reworking), and I would like to discuss the technicalities to make the chatroom compatible with your framework - and development frameworks.

First off, I am considering a change in the chat log format and chat protocol that would make it possible to keep the log unified while allowing versioning and handling any future updates.

The current log format is such:

time<TAB>user<TAB>cmd[<TAB>color<TAB>text]<NL>

I am considering using the following new format for all future versions:

time<SPC>version<TAB>user<TAB>msg<NL>

The space after time instead of the tab is there to mark the new format unambiguously and in a faster way (no parsing of the second token required to check for a number instead of a user name).

The tabs around the user name remain necessary as some older "Guest NNNNNN" names contained spaces. Alternatively, we could also escape those spaces by processing the log and in all future writes.

Merging the command and data makes storing future commands with any number of arguments more consistent. So, what is currently stored as

msg<TAB>000000<TAB>hello

would become (with the default color):

msg hello

and with text color and background color (yes, some kind of colored speech balloons are in the work):

msgc 000040/F0F0FF hello

So, msg is a command with one argument (the text), whereas msgc is a command with two arguments (colors, then text), and the arguments are separated by just spaces, not tabs.

I am working on a kick command that woud use spaces the same way:

Input: /kick browni3141 Forgot to log out of the chatroom before his WC game.

stored as:

123467890<SPC>2<TAB>Janzert<TAB>kick browni3141 Forgot to log out of the chatroom before his WC game.

Next, I have converted the chatroom to html5 but it is still pure html/css/js, no framework used, so I am wondering how to best integrate it into your project.

I am working on separating the code but have not gone full length into encapsulating the whole chatroom. Do you have any recommendations for that?

And one last thing (although I have a hundred more to list): I am working on a command to display an Arimaa board position in place of a chat message, using HTML5 canvas. How to best integrate it within your project?

I believe the discussions about the functionalities would benefit from the input from the whole community, so anybody reading this please spread the word and bring up feedback! :)

Thanks!

Title: Re: New website & new client
Post by Janzert on Jan 26^th, 2015, 8:44pm

one chat feature I've wished for more than once is for the archive to allow get requests to act as permalinks to a specific day, (e.g. chat/arch.php?date=[whatever date format you care to support]) maybe even ?date=blah#time to anchor to the specific time?

Janzert

Title: Re: New website & new client
Post by leo on Jan 27^th, 2015, 6:33pm

Hi Janzert,

Sure, such a permalink would be useful, and it is easy to implement, I have added it into my endless list of action items :) Time zones would need to be taken into account, as not all viewers will agree on when midnight sits.

I am also considering adding a command to call the archive from the chatroom input line:

/arch yesterday
/arch Jan 15
etc.

I also pondered about displaying the archive logs in the chat window itself, but that would make the archive accessible only to logged in users, something that we discussed years ago. Permalinks would still work but would require logging in before viewing the content. I am not sure if this is a good idea.

Another improvement to the archive viewer would be the ability to select a time range rather than a 24 hour span YLT.

Since you are there, I have a question about improving things around the progressive polling frequency, as I am not entirely clear about the logic of it. It seems to me that currently, when the frequency is low, sending a message will have to wait a long time for the update timer to allow the next request.

Thanks!

Title: Re: New website & new client
Post by Janzert on Jan 28^th, 2015, 10:42am

I have a small preference that the archive remains openly available. But I also wouldn't bother adjusting the archive display to local time. :P

I've lost my copy of the chat source years ago and haven't looked at it in even longer. From what I recall though, any sent message should go out immediately and also reset the polling frequency to its quickest interval.

I did just recall another thing I would like to see change, even more than the archive link. ;) The auth code should be moved out of the url. So that copy and pasting the url somewhere public doesn't reveal sensitive information.

Janzert

Title: Re: New website & new client
Post by leo on Jan 28^th, 2015, 7:04pm

Thanks, Janzert!

I am not sure I get you, but the archive viewer is already in the local time of the browser just like the chatroom. The problem I was seeing is that a permalink should contain either a gmt time or a time zone to ensure that the span and anchor work around the world for any viewer.

The message sending function sends it right away indeed, then turns off stand by if on (stand by is more or less independent from the progressive slowdown), then calls the polling function, but that latter function first checks the status of the polling activity and aborts if any request is scheduled, even in a long time. I would add a frequency reset in the message sending function but I am wary of touching the many variables that you used for the progressive polling, it would also require cancelling the timer for the next scheduled request I guess.

I am totally with you to remove the authentication from the url, and Harvestsnow will even more. That will require changes to the gameroom code though, as that url is generated by the gameroom to open the chatroom. Maybe the chatroom could simply share the session with the gameroom - I don't quite remember why we didn't implemented it so in the beginning, I know it was considered. If the archive remains public, and even more if the chatroom gets integrated into the gameroom as in Brendan's project, then the need for a separate authentication and a list of logged in users would belong to the past.

A note about user options in the chatroom: The development version uses web storage instead of cookies, to save bandwidth (as cookies are sent along with all the polling requests). I am devising the local storage keys with "chat." prepended in order to prevent conflicts with any future local storage used by the rest of the site, gameroom and other sections. I am not sure if this is enough, advices are welcome.

One drawback of local storage is that it is browser specific, which means that if you use several browsers, you will have to set your options in each. A solution could be to store the options on the server, in some kind of user profile table, and a user's options would come up on any browser and any computer. This might turn out too much development though.

Title: Re: New website & new client
Post by Janzert on Jan 30^th, 2015, 3:49pm

on 01/28/15 at 19:04:56, leo wrote:

Sorry, I was just trying to say that the time localization is not that important to me. In some ways I think it actually hurts since people have to refer to past discussion via relative time stamps.

Quote:

The message sending function sends it right away indeed, then turns off stand by if on (stand by is more or less independent from the progressive slowdown), then calls the polling function, but that latter function first checks the status of the polling activity and aborts if any request is scheduled, even in a long time. I would add a frequency reset in the message sending function but I am wary of touching the many variables that you used for the progressive polling, it would also require cancelling the timer for the next scheduled request I guess.

If you like I don't mind looking at it again, but point me to a location to find the current code.

I probably wouldn't worry a whole lot about an extra request getting sent, i.e. cancelling an already scheduled one doesn't gain much.

Quote:

I am totally with you to remove the authentication from the url, and Harvestsnow will even more. That will require changes to the gameroom code though, as that url is generated by the gameroom to open the chatroom. Maybe the chatroom could simply share the session with the gameroom - I don't quite remember why we didn't implemented it so in the beginning, I know it was considered. If the archive remains public, and even more if the chatroom gets integrated into the gameroom as in Brendan's project, then the need for a separate authentication and a list of logged in users would belong to the past.

I wasn't really involved with this part, so not sure why the decisions went the way they did.

One caution on chat integrated in the gameroom itself is the effect of game commentary when any logged in player also receives chat. I think this is probably fine though if every game also gains its own kibitzing chat. Except that other potential kibitzers won't even know there is a discussion in progress unless they have the game open. What I think would be ideal is if the kibitz chat stayed with a game permanently and replaced the current game comment system.

Quote:

A note about user options in the chatroom: The development version uses web storage instead of cookies, to save bandwidth (as cookies are sent along with all the polling requests). I am devising the local storage keys with "chat." prepended in order to prevent conflicts with any future local storage used by the rest of the site, gameroom and other sections. I am not sure if this is enough, advices are welcome.

One drawback of local storage is that it is browser specific, which means that if you use several browsers, you will have to set your options in each. A solution could be to store the options on the server, in some kind of user profile table, and a user's options would come up on any browser and any computer. This might turn out too much development though.

Given that the chat options shouldn't be that terrible for the user to reproduce I wouldn't worry too much about their permanency. Also having different settings on different devices may very well be a desired thing fairly frequently.

Janzert

Title: Re: New website & new client
Post by leo on Jan 30^th, 2015, 6:59pm

on 01/30/15 at 15:49:13, Janzert wrote:

Sorry, I was just trying to say that the time localization is not that important to me. In some ways I think it actually hurts since people have to refer to past discussion via relative time stamps.

I will add an option to display the time stamps in UTC so that people can more easily refer to a point in the archive.

Quote:

If you like I don't mind looking at it again, but point me to a location to find the current code.

The send function is at line 82 of chat.js (another function was added at 110), fetch is at 299 -- that's where we get requests cancelled depending on the timers. The function that toggles stand by is at 148.

Quote:

What I think would be ideal is if the kibitz chat stayed with a game permanently and replaced the current game comment system.

Sounds like a good idea. In the dev version I am encapsulating the chat code for it to allow several instances so we can have several chat elements in the DOM associated with different server handlers.

Quote:

I am currently feature creeping, adding options to customize the interface, display, command interpretation... Maybe having the locally saved options override the account saved options would make it perfect.

I would do server stored options only on a database though, and am considering converting the chat log to a database as well, to improve speed - especially that of the archive viewer. Also, when testing a conversion of the current log I bumped into one corrupted line and it is worrying, a database would be cleaner.

Title: Re: New website & new client
Post by harvestsnow on Feb 2^nd, 2015, 2:57pm

Quote:

The development version uses web storage instead of cookies, to save bandwidth (as cookies are sent along with all the polling requests).

I think the option cookies are only sent to the authentication page. But cutting all the HTTP overhead would be nice anyway. Have you looked into websockets?

Title: Re: New website & new client
Post by leo on Feb 2^nd, 2015, 5:16pm

Hi Harvestsnow, I just checked that and you are correct, only the root arimaa.com cookies are sent along with the polling requests. I can't recall if the chat cookies were restricted to just chat.php since the beginning or if that was tweaked afterwards.

Yes, websockets were considered at different moments in the past and the time should be mature now to use them across browsers. I am unsure of how to make the system compatible with all the gameroom flavors existing and upcoming. Any ideas?

Title: Re: New website & new client
Post by harvestsnow on Feb 2^nd, 2015, 7:30pm

on 02/02/15 at 17:16:17, leo wrote:

I am unsure of how to make the system compatible with all the gameroom flavors existing and upcoming. Any ideas?

*scratches head*
Nope.

Title: Re: New website & new client
Post by leo on Feb 5^th, 2015, 5:51am

*scratches head too* :P

I have back pedaled about the changes to the syntax of the commands and events, giving up adding a version number to the log lines and keeping the command word as a tab-separated token to make analysis and search easier if we ever need that. There are some changes though:

OLD EVENT: time<TAB>user<TAB>cmd[<TAB>color<TAB>data]<NL>
NEW EVENT: time<TAB>user<TAB>cmd[<TAB>data]<NL>

Here is the detail of the events:

OLD

in
out
timeout
msg<TAB><text color><TAB><message>

NEW

in
out[<TAB><goodbye message>]
tmo
md<TAB><message>
mc<TAB><text and background colors><SPC><message>
me<TAB><emote message>
sp<TAB><spoiler message>
pre<TAB><table or code with preserved format>
bp<TAB><board position>
kick<TAB><kickee>[<SPC><reason>]

There would be no format conversion of the existing chat log on the server, an archive viewer would have to process the old "msg" event differently, checking for emote messages with initial "/me" or ":" that is not part of a smiley.

The server commands now include a "version" command that a client should send prior to any processing in order to be able to notify the user if they are using an outdated client.

The current chatroom client has no mechanism to warn the user of an update, so I expect some problems with the users who keep their chat window open during the upcoming update.

Title: Re: New website & new client
Post by Janzert on Feb 5^th, 2015, 11:38am

on 01/28/15 at 19:04:56, leo wrote:

So I finally stopped being lazy and figured out that http://arimaa.com/arimaa/chat/chat.js was the url I needed to hit to check this.

So here's my reading of the code. Notice I say reading not actual testing. ;)

As you said, after sending a message the send function calls fetch to get the latest updates (including the message we just sent). The first thing fetch does is check if another update request is in progress and aborts if there is (lastrequest exists, is not complete and hasn't been going for > 30 seconds). It doesn't pay any attention to the update timer frequency before sending the update request. After sending the update request, it then immediately schedules the next one.

So there is a race condition (all network requests are asynchronous) but in most cases it will send an update request right away after sending a message and will get the new message. The problem arises that currently for any update received the current code is already waiting one pre-update interval to send the next request, i.e. it always waits one extra long interval after sending a request that contains an update before speeding back up.

The best way to fix this is probably to move the

Code:

if (lastupdatetime && currentupdateinterval < 10000)

out into its own function, say 'schedulenextupdate()' . Then call that function from the fetch function where it currently runs and the gotupdate function, calling it right after lastupdatetime is changed.

To mostly fix the race problem between the message send and update from the server. The send functions should also set lastupdatetime just before calling fetch. But this will less often have an effect.

One side effect of rescheduling each time an update is received though is that update requests will slow down by the server round trip time when active chatting is in progress. The shortest interval could be decreased some to compensate but it's probably not necessary.

Janzert

Title: Re: New website & new client
Post by leo on Feb 8^th, 2015, 8:36am

Thank you so much, Janzert! And sorry for misunderstanding your question about the location of the code as inquiring about the line numbers instead of the file url.

I still have some doubts about the current code that turns off the stand by mode (line 148 ) after sending a message. I wonder if it does the right thing in the middle of the polling system.

Brendan, I had a closer look at your cljs, it looks like a fantastic language. Unfortunately I have very little time these weeks to study yet another language, so instead I am turning the chatroom into a javascript object that supports several instances in one DOM. The user, authentication, server, and other parameters, can be set on the prototype or each instance depending on which are shared. Will this be ok?

As a replacement to the current chatroom, it will be a single instance in one wrapper window.

Title: Re: New website & new client
Post by Janzert on Feb 8^th, 2015, 4:15pm

on 02/08/15 at 08:36:15, leo wrote:

I still have some doubts about the current code that turns off the stand by mode (line 148 ) after sending a message. I wonder if it does the right thing in the middle of the polling system.

On a quick review I don't see any problems. Do you have something specific you think could go wrong?

Janzert

Title: Re: New website & new client
Post by leo on Feb 10^th, 2015, 5:29pm

I was under the impression that the code to toggle standby did not update all the necessary variables that will be tested for a fetch request to actually be sent. I must be wrong, since it works just fine. Sorry, I am just being confused. In a few weeks I will have some time again to devote to the chat room, I can't wait to show screenshots of the new interface and hopefully trigger feedback and suggestions for improvement :)

Title: Re: New website & new client
Post by leo on Mar 2^nd, 2015, 5:06am

I am resuming work on the new chatroom DOM object and I have a few questions about the server side: Since multiple chat instances (on a single web page or separate pages) are now supported client side, what do you think would be the best way to make it work with the current server and the new server projects? In addition to managing separate logs (gameroom, game instances, other?), do you think online users management should be kept on each instance -- or dropped entirely?