March 2016

Monthly Archive

We’re Baaaa-aack!

Posted by on 31 Mar 2016 | Tagged as: Uncategorized

Kind of.

We had 2 disk failures early Tuesday morning (2016-03-29 05:14:49 PST). Jim and I spent today at the data center where the server lives, fighting with the BIOS and getting 3 replacement drives installed (1 disk had died a few months back).

The drives are back and the servers are serving, except for the main database which wasn’t redundant on the server. We have backups of it off site, but it will take some hours to get it copied back.

The database is where we store the scores for the high score tables and leader board, as well as user login information. This means those features are not going to work until the database is restored. In the meantime, you should be at least be able to play games anonymously.

Jim and I were impressed by how many people tried to reach us when they couldn’t get the site to work, and at the lengths some of you went to: one of you figured out my cell phone number and called me (I was asleep and didn’t hear my phone ring), another one of you figured out who our hosting provider was and emailed their support team (leading to a funny conversation when we went into the data center today). Don’t take that wrong, we aren’t annoyed—we’re heartened that so many of you like the site so much, and we’re sorry for the downtime.


Update 2016-03-31 09:25 PM PST: The site doesn’t work well without the database so we’re temporarily pointing to our offsite backup database. This is just hosted on my cable modem at my house so I might not have enough bandwidth to support the full load of green felt. This is just temporary until the backup is copied back to the main server. Hopefully it’ll hold up well enough until then ;-).


Update 2016-04-05 01:25 AM PST: The backup has been restored and we switched over tonight at midnight. Everything seems to be OK.

Downtime and database upgrades

Posted by on 20 Mar 2016 | Tagged as: Uncategorized

Last night our hosting provider was having work done in the facility where our server is hosted and the power got turned off. We run a very minimal setup here at Green Felt, which means we don’t have backup servers for situations like these. Hence the downtime last night and this morning.

While things were down we decided to take the opportunity to upgrade one of our offsite database backups. Now that the machine is back up we’ll be copying that upgraded data back to the server. During this time the site will be up and games will be playable, but high scores and the leader board will not work. Also logging in and creating users will not work, though if you are currently logged in you should stay logged in. Games will still be saved to our offline queue, and when the DB is back up and running we will write all the games in the offline queue back to the database.

We’re not sure how long the DB copy will take, but lets assume it will be a couple hours.


Edit [1:52 PM]: We’re having trouble with the upgrade so we’re going to bring the DB back up and try again sometime in the future. Everything should be running now.