Tips and Secrets for Dealing With App Engine

Google’s App Engine is a wonderful, amazing, and revolutionary service. A free, managed hosting experience in which you can build scalable applications. That is huge. I can honestly say I would not know half the things I know or have half the relationships I have with the tech world, were it not for this program. It makes writing server-side software simple; it allows you to only know about writing the software, and not having to worry about system level vulnerabilities, system load, or scaling headaches. I don’t have to think about things like iptables, SSH permission levels, or other ways nasty people who are better sysadmins than I am can get into my application. I don’t have to worry about hard drive failures at 3 in the morning. Rather, it just asks me to be a programmer.

That divorce between programming and systems administration comes with some costs, though. The most commonly touted one is the lack of control over the system: if I want to run node.js or Redis, I’m out of luck. But there are other, deeper costs as well, things that aren’t intuitive to the average user. Things that don’t come to mind when you think about the shortcomings of App Engine. And really, they all come down to one thing: if what you want to know isn’t documented, you probably are out of luck.

Date and Time

The handling of date and time on App Engine is inconsistent, and it caused me quite the headache. See, the datastore is written in UTC time, but a “day”, in App Engine terms, is determined by Pacific time. Your quota, for example, resets at midnight Pacific time. Meanwhile, the datastore holds values that are from UTC time, which means seven hours in the future (according to the quota). Furthermore, the “system time” (what you get if you call datetime.now()) on App Engine is in UTC. The only way to get the values you’re writing to the datastore, the values datetime.now() are returning, and the datetime that the quota is based on all in sync is to translate the values to Pacific time, strip the timezone information from them (lest the datastore see it, take it into account, and attempt to correct for it), and then write to the datastore. Then everything will be in sync.

That is all documented, or at least relatively easy to figure out. You can even see it in practice at http://timezones.appspot.com. But I had to implement a soft quota over Google’s quota, so I needed to know exactly what quota day I was on, and had to keep the two in sync. And that meant one big question that was not in any documentation and that would take me quite some time to figure out: does the App Engine quota take DST into effect?

Fortunately, Moishe Lettvin, an App Engine employee, has been kind enough to keep in touch with me after I took part in the Channel API trusted testers session. He checked the source code and got back to me. I don’t know how I would have handled the situation had he not responded; it would have taken me months to find out on my own. For those who are curious, yes, App Engine’s quota takes DST into account.

OAuth

I have a love/hate relationship with App Engine’s built-in OAuth provider. I love it because it’s a built-in, out-of-the-box, free way to authenticate against someone’s Google account. I hate it because if something goes wrong (it’s listed as an experimental API, and things do go wrong) or if something isn’t working as I expect (it differs from the main OAuth API in a couple crucial areas; mainly it seems to lag. It still uses 1.0a and will not accept xoauth_displayname), I have no real recourse. Beyond not being able to tell why getting a request token threw an error for 38 requests in the last 24 hours, besides undocumented differences between it and Google’s regular OAuth offering (not even their 2.0 offering, which is now available in experimental status), my biggest gripe has to be one I discovered just recently: it will not allow a non-http, non-https callback URL. If the URL you are redirecting to uses a different protocol, it will error out. While this seems like a silly gripe, it’s actually pretty important for Android developers who rely on callbacks to app-specific protocols to allow the OAuth flow to remain secure and user-friendly.

Those are the biggest gripes I have with App Engine for now. I’m sure more undocumented things I need to know will show up as I continue to learn more and more about the environment. I’m sure I’ll be stymied in other places. But I reiterate the point I opened with: App Engine remains a wonderful, powerful tool that makes my development possible. These gripes don’t come close to overshadowing the cause of them: the removal of the system from my list of “things I need to concern myself with” in most cases. Really, these gripes are created by the edge cases in which I do need to concern myself with the system.