Yesterday I did a bunch of experiments on three cloud-computing services. Partly this was just to get some actual hands-on experience with each one. Partly it was to calibrate expectations with respect to I/O performance, for New Job which is approaching quickly. On each one, I set up a small instance using Fedora/CentOS (would have used RHEL except that it incurs extra licensing charges). Then I installed gcc, which wasn’t in any of the base images, compiled iozone, and ran some very basic I/O tests. In each case I tested with both one and four I/O threads. Here are some observations.

  • Amazon EC2′s instance-storage performance was pretty bad, at 20-25MB/s for write and 40MB/s for read.
  • EBS was even worse for write, but quite good for read – up to 90MB/s with one thread and twice that with four. I’m not really sure how the latter number is even possible (do they have dual GigE interfaces on these things?) but I retested to make sure I was transferring more than the instance memory size etc. Thought: maybe I’m exceeding the instance memory size, but not the system memory size. That’s a bit of a gotcha; I think I’ll do some more testing to explore this idea.
  • Rackspace (formerly Mosso) was much better at 100-130MB/s for write and 140-190MB/s for read. This is roughly equivalent to EC2′s instance storage; unfortunately there seems to be no direct equivalent of EBS so storage remains completely instance-private. Running a parallel filesystem across instances might be an interesting experiment.
  • Flexiscale performance was very similar to Amazon’s, across the board.

There were also noticeable differences other than performance. Cost wasn’t one of them; prices for similarly configured instances seemed very close for all three. Most of the differences had to do with the administration interfaces.

  • Amazon had by far the nicest control panel, with by far the largest selection of images and instance types and configuration options (e.g. “elastic IP addresses”). It was also the only one that seemed decently set up for kicking off a whole bunch of instances all at once.
  • Rackspace’s control panel was usable, but with far fewer options. Console access is nice, but I’m not wild about getting email every time a server’s state changes. That would get pretty annoying for serious numbers of servers that are used sporadically. I’m also a little disturbed that my “total disk space” still shows 90GB even after I’ve terminated all of my instances. If there’s an additional step to clean that up so I don’t get billed for the whole month, they’ve sure hidden it well; if there’s not, it shouldn’t be reported the way it is.
  • Flexiscale has some more serious usability issues to deal with, starting with registration. They ask for a phone number, but reject any familiar form for a US number (they’re in the UK). Hint 1: you have to use “+1 xxx xxx xxxx” but there’s no help or example to tell you that. Next, you have to “add credit” up front – instead of being billed after-the-fact as with the other two – before you can really do anything. Then, having done that, you’re on the wrong screen to start an instance and it’s not at all clear how to get to the right one. Hint 2: click on the Flexiscale logo at the top left. Attaching an extra disk involved another Adventure-like hunt through the interface, and so on. It was simply unpleasant.

I know some people will say I shouldn’t be using control panels, and should use the API instead. This is clearly an option with Amazon, though it seemed unnecessary for what I was doing given how well the control panel worked. Ditto for Rackspace. Flexiscale’s control panel almost seems designed to push you toward using their API instead, but their documentation for it is lousy – missing information on general session setup, clearly necessary calls not mentioned, etc.

Overall, it looks like Amazon and Rackspace get my approval based on this (admittedly brief and lame) test. With its ability to spawn many instances at once and manage them as a group, and its richness of cloud-oriented images, Amazon seems the most truly cloudy . . . but performance is a bit disappointing. For a smaller number of longer-lived instances that might correspond to a more traditional web-service kind of usage, Rackspace seems to offer significantly better performance at approximately the same price. If Flexiscale’s performance were better – as it should be with their directly SAN-attached storage – or if they had superior functionality then the setup unpleasantness could be overlooked, but as it is I can’t think of any reason I would choose them over Amazon.