Midnight Oil

I’ve had my first brush with EC2 today, and I’m thoroughly confused and exhausted. It definitely has a much steeper learning curve than the other Web Services. However, since you’re dealing with booting and configuring fully functional virtual machines, I guess that’s to be expected.

Still, I managed to spawn an instance, set all the necessary permissions, and then connect to it via ssh. Not exactly something to brag about, but it is a milestone. I consider it $0.20 well spent. (I had to terminate and re-run the instance because I didn’t store the key-pair RSA key the first time.)

The goal

For Obsidian Portal, the map processing and tiling is very CPU and RAM intensive. If run on our Slicehost VPS, it totally bogs down the whole system, making the main website completely unresponsive.

So, we’ve offloaded the processing to a machine in Ryan’s attic. Although it’s worked so far, it’s not exactly a professional solution. Plus, Ryan’s getting ready to move and his machine will be off for at least a week. That means we need a new place to run our stuff. Since I’ve been meaning to dive into EC2 for a while now, this seems like the perfect opportunity to put in place a permanent and professional solution.

The plan

Whenever we need to tile a map, I plan to spin up an EC2 instance and run the tiler. I’ll keep the instance up for the rest of the hour, in case more maps come in. When the hour runs out, if there are no more maps to process, then we’ll spin down the instance. Computing power on demand…nice!

grempe-amazon-ec2

If you’re a Ruby hacker, skip the EC2 toolset altogether. Just go get the grempe-amazon-ec2 gem and run all your EC2 requests through an irb shell. This allows for opening up the EC2 developer reference and learning the API commands straight from the docs, rather than having to learn the EC2 toolset. You’re probably going to have to spawn/terminate instances dynamically at some point, so you may as well just learn how right from the start.

Besides, the EC2 toolset route involves setting up a JVM, setting JAVA_HOME, and all that Java mumbo-jumbo that always takes way longer than it should. Save yourself a headache and go straight to the source.

ec2onrails

I’m not looking to run a full Rails server (yet), but we do want to run some rake tasks. Rather than hand build an image, I’ve decided to start with the ec2onrails image and see if that gets me most of the way there.

The main downside is that this image will need to install RMagick, ImageMagick, and possibly other gems and packages each time it spins up. ec2onrails has built-in support for this, but it means that the map processing can’t start for several minutes after the image boots up, meaning our users have to wait to see their maps. For now, it should be acceptable, but it’s something to improve on.

Next Steps

I’ve had enough for today, but there’s still a ways to go. I need to complete the following:

1. Setup the cron job on our VPS to spawn the EC2 instance whenever a map needs processing (and there’s not already an instance running)
2. Setup the script (rake task?) for the EC2 image that will do the processing. It may as well just run forever. Maybe it can be responsible for terminating the image? Hmm…that sounds dangerous. If it should crash out, then the instance will keep running and billing us!
3. Surely there’s more…

Oh, and one other next step: I’d better go terminate the instance I left running!

We’re 95% done on RioFlexPay, a front-end graphical user interface (gui) to the Amazon Flexible Payment System (FPS). We’ve got a splash page up right now, and we’re getting ready to put the production server together in anticipation of a pre-June deployment.

It’s been a tough ride, but we’re almost there! In a few short days or weeks, managing your FPS account will get much, much easier.

Stay tuned!

We have finally gotten to the point where we are ready to start offering subscriptions to Obsidian Portal. We don’t expect there will be a lot of interest, but it’s always a sort of chicken v. egg problem. If you don’t have paying subscribers, then it’s not worth the effort to make the features. Conversely, without the features, no one is going to pay. On second thought, I guess it’s not chicken and egg, it’s pretty clear: you need features or no one will pay.

Asking for payment means you will need to be able to accept it. Currency on the web is passed almost exclusively via credit cards (except for PayPal…), so that’s the direction we need to go in. That requires us to select a credit card processor. For today, we will be looking at TrustCommerce.

I won’t go into the details of how credit card processing works, mainly because I don’t really understand it myself. Suffice it to say, there are a lot of middle-men, and they are all trying to take a cut. Each cut is either a percentage of the total charge or a flat fee or both. So, a typical fee structure might be $0.30 flat fee plus 2.5% of the total transaction.

Note: If you don’t care about the analysis and just want to see a rundown of their prices, then jump to the pricing.

Go easy on me; it’s my first time

When selecting a processing agent, our first priority right now is ease of use. We don’t expect there will be a lot of people signing up for our premium service, so we don’t want to expend a lot of effort on a payment system only to never see it used. Also, we’re willing to pay a higher rate to the processor since 3% of $30/month is a lot different than 3% of $30,000/month. I’ll pay 3% vs 2.5% if the 3% service takes 2 hours to implement and the 2.5% service takes 10. So, for us, ease of use trumps competitive pricing.

Since we’re talking about subscriptions as opposed to purchases, there is a recurring element to the payments. Since we want easy-to-implement solutions, we are scoping our search to only include the payment processors that offer a recurring service. This is a very important thing to note, especially if you’re in the same boat. A 1-time payment processor model (like Google Checkout) just will not work if you want to do subscriptions. The main reason is that you will have to store the users’ credit card info on your server in order to pass it to the payment processor each billing cycle. Do not do this! There are actual laws and regulations detailing what sort of security procedures you have to maintain in order to hold that sort of sensitive data. It’s much easier to simply pay someone else to deal with that crap. If you do choose to store their info in your database, you should be looking for a lawyer right now, not a payment processor.

Just plug in your credit card info

In Rails, ease of use means finding a plugin. I write a lot about plugins on this blog, so why should credit card processing be any different? Doing a quick Google search led me to the TrustCommerce subscription payment plugin.

Finding this bit of code brought a smile to my face, as I thought I had just finished 90% of the work. Sign up for an account, drop in the plugin, and wait for the money to roll in. Too bad there were a few red flags that derailed the money train.

Sitting by the phone

TrustCommerce does not list any pricing on their website. Instead, they say you have to sign up for a test account, and then you’ll be contacted. Not a big deal, I guess. So, I signed up for a test account.

The first red flag went up when I did not get an immediate callback. Sure, I signed up at 11:00pm Eastern Time, but that’s normal business hours in Internet time. In other words, if you’re an Internet company that requires phone contact, you had better have someone manning the phone at all hours. A lot of Web jockeys like me have a regular 9-5 job that precludes us from doing our business dealings during normal business hours. I want to deal with companies that understand this and have staff available during my normal business hours.

Red flags: 1

The ball sits in my court

The second red flag went up at their lackluster eventual response. My cell is in a dead zone at work, so whenever I leave for lunch, I get all my messages. On the day after requesting contact, I had a voicemail message from TrustCommerce. Still no pricing info, just a short message to call them back. Seeing as how I was busy, I couldn’t do it right away. Then I forgot. Dead silence on their end. No e-mails, no more calls, nothing.

Now a lot of people may disagree with me on this, but I think they should have been hitting my inbox and voicemail pretty hard. “Mr. Wedemeyer, we’re still interested in talking to you about blah blah.” or “Send us an e-mail with the best time to call you.” That’s how the mortgage people behaved when I used LendingTree. Sure, it was annoying, but you knew they wanted your business. To me, an anemic response indicates that someone isn’t really serious about recruiting me as a customer.

Red flags: 2

Little fish: prepare to get fried

When I finally did get in touch with someone from TrustCommerce, he was quite happy to answer my pricing questions. I don’t know if I’m allowed to post that info, but since they didn’t expressly forbid it, here you go:

Basic pricing

• $95 1-time fee
• $20 / month
• $0.20 / transaction

Citadel (recurring payments)

• $145 1-time fee
• $10 / month
• $0.10 / month / billing id (ie. subscription)

Holy crap! $240 just to get started, plus an additional $30 per month, just to be allowed to use their service? Seeing as how I expect Obsidian Portal to be making around $10 / month, at least until we can recruit more people, this is insane! I politely said thank you to the salesman, hung up the phone, and started writing this post.

I guess I see these huge front-loaded fees like this: If you’re making enough money that the fees don’t matter, then you already have a lot of subscribers, which means you’re already handling credit cards. Maybe their service is so great compared to the competition that it’s worth it for the big boys. But, if you’re a small time operator like me, forget about it.

Red flags: 240 + 30 / month

The search continues

Although I said pricing was not our top priority, the front loaded fees with TrustCommerce completely invalidate them as a viable option. It would be a very long time before we paid off the initial investment, and with our none-to-clear business prospects with Obsidian Portal, that’s a gamble I’m not willing to take.

In the next exciting chapter we will be looking at Amazon Flexible Payment System (FPS). This new web service from Amazon is meant to rival Google Checkout and PayPal. I’ve been extremely pleased with S3, and maybe they can do one better with FPS. Stay tuned to find out.