July 25, 2012

AQWorlds Mega Bug Fix

Yorumi good, NaN-bug bad!

Thanks to all of the amazing testers on the AQWorlds' PTR server, Yorumi was able to find "THE CRAZY NaN BUG." I would like to share the nitty gritty details of this bug... but first, from your Twitter and Facebook messages, I realized that many players do not know what is actually happening behind the scenes on the PTR (aka Public Test Realm).  And why it restarts so much.

Really tricky bugs like the Crazy NaN Bug do not appear on our normal staff-only test servers. In order to cause them we need to do two things: 1) Have tons of players on.  2) Create a special version of the game server which watches for specific things and then writes the errors somewhere we can investigate.

So, everytime you have seen the server restart, you are playing on a new version of the game server which is doing massive bug-hunting and testing. Because of this... by process of elimination.... we found the Great GrandDaddy of all AQWorlds problems: THE CRAZY NaN Bug.

If you played our older games you may have hit a monster for "NaN" damage. What is NaN? (Not to be confused with the delicious bread at Indian Restaurants) NaN stands for "Not a Number." It is what happens when you ask Flash what 1 + "a" equals. That is not a variable, that is actually the letter "a."

It  breaks the computer's brain. In human terms it is sort of like asking someone what the color blue smells like. But how could this be a problem for us? We have gotten really good at protecting our games from NaN problems... REALLY GOOD.

I mean we use a function EVERYWHERE that makes sure that nothing isNaN. That was not a typo, the function is actually called isNaN(). Say "isNaN" in a silly cat voice in your head... it is sorta cute. Remember the NaN Cat from the AQWorlds Meme release? Anyway... despite all of our isNaN checks, every once in a while some NaNs were still getting through and generating hordes of mysterious and unexpainable errors on the server. 

After two days of non-stop banging heads against desk, Yorumi finally figured it out. isNaN, as cute as it is, does not work in ONE very specific situation. To demonstrate, let's feed isNaN some things that are Not a Number and see what happens:

Is Not a Number?

isNaN("a") = True

isNaN("chicken") = True
isNaN("I like dinosaurs that shoot lasers out of their eyes") = True

isNaN() = True

isNaN("") = False!?!?!?!?!?!?!?!?!?!?!

WHAT!? How is nothing a valid number? This was flat out illogical & baffling! So there it is... because the isNaN function fails under that one specific condition, there are mabye thousands of tiny rare bugs that occassionally happen in AQWorlds, generating thousands of errors. Once we found the problem it was super easy to fix.

Yorumi changed every instance of isNaN in the game to also look for empty "" values. This fix might actually speed up literally every part of the game server.... the PTR server is currently running the fix. Help us continue the tests!