Midnight Oil

I’ve had my first brush with EC2 today, and I’m thoroughly confused and exhausted. It definitely has a much steeper learning curve than the other Web Services. However, since you’re dealing with booting and configuring fully functional virtual machines, I guess that’s to be expected.

Still, I managed to spawn an instance, set all the necessary permissions, and then connect to it via ssh. Not exactly something to brag about, but it is a milestone. I consider it $0.20 well spent. (I had to terminate and re-run the instance because I didn’t store the key-pair RSA key the first time.)

The goal

For Obsidian Portal, the map processing and tiling is very CPU and RAM intensive. If run on our Slicehost VPS, it totally bogs down the whole system, making the main website completely unresponsive.

So, we’ve offloaded the processing to a machine in Ryan’s attic. Although it’s worked so far, it’s not exactly a professional solution. Plus, Ryan’s getting ready to move and his machine will be off for at least a week. That means we need a new place to run our stuff. Since I’ve been meaning to dive into EC2 for a while now, this seems like the perfect opportunity to put in place a permanent and professional solution.

The plan

Whenever we need to tile a map, I plan to spin up an EC2 instance and run the tiler. I’ll keep the instance up for the rest of the hour, in case more maps come in. When the hour runs out, if there are no more maps to process, then we’ll spin down the instance. Computing power on demand…nice!

grempe-amazon-ec2

If you’re a Ruby hacker, skip the EC2 toolset altogether. Just go get the grempe-amazon-ec2 gem and run all your EC2 requests through an irb shell. This allows for opening up the EC2 developer reference and learning the API commands straight from the docs, rather than having to learn the EC2 toolset. You’re probably going to have to spawn/terminate instances dynamically at some point, so you may as well just learn how right from the start.

Besides, the EC2 toolset route involves setting up a JVM, setting JAVA_HOME, and all that Java mumbo-jumbo that always takes way longer than it should. Save yourself a headache and go straight to the source.

ec2onrails

I’m not looking to run a full Rails server (yet), but we do want to run some rake tasks. Rather than hand build an image, I’ve decided to start with the ec2onrails image and see if that gets me most of the way there.

The main downside is that this image will need to install RMagick, ImageMagick, and possibly other gems and packages each time it spins up. ec2onrails has built-in support for this, but it means that the map processing can’t start for several minutes after the image boots up, meaning our users have to wait to see their maps. For now, it should be acceptable, but it’s something to improve on.

Next Steps

I’ve had enough for today, but there’s still a ways to go. I need to complete the following:

1. Setup the cron job on our VPS to spawn the EC2 instance whenever a map needs processing (and there’s not already an instance running)
2. Setup the script (rake task?) for the EC2 image that will do the processing. It may as well just run forever. Maybe it can be responsible for terminating the image? Hmm…that sounds dangerous. If it should crash out, then the instance will keep running and billing us!
3. Surely there’s more…

Oh, and one other next step: I’d better go terminate the instance I left running!