I am writing the first draft of this posting less than 24 hours after the announcement of the public beta release of Amazon EC2 (Elastic Computing Cloud) and it is making me relive the emotions that I and many other developers felt when Amazon S3 was announced. A shared culture is clearly at work. In each case, Amazon create and offer a service that has very few features: but those that it has, it implements consistently, cleanly and robustly. The documentation has the forceful simplicity of a 1970s IBM manual, saying exactly what needs to be said, straightforwardly and unambiguously: it has been written by someone with a mind with the intention that it should be read by someone with a mind. The developer forums are monitored by intelligent people who get to the bottom of the questions that are asked – sometimes deeper than the questioner himself – and give answers uncontaminated by considerations of marketing or public relations. Nothing like that has been seen since the Microsoft Windows 3.1 beta test programme.
Outsiders do not realise how deeply technological development, like scientific research, is shaped and driven by emotion. If you have ever looked at an iPod and wished that you needed one, you will have had an inkling of it. The reason that half of us are in computing at all is that we see computers as things that we can make beautiful things out of: and S3 and EC2 arouse the same emotion.
Some people have criticized S3 and EC2 for being bony, but that is the point of them. You cannot go wrong if you have good, strong bones to build on. If the foundation is right, things just go on getting better. A woman with good bones is six times as beautiful at 60 as she is at 20.
To take one example: when an EC2 computer is shut down, it evaporates. Completely. It is as if it had never been. In particular, any data that it stored are lost. This sounds ridiculous, but it keeps the EC2 architecture simple and beautiful. Anyone building a system on EC2 already has access to data storage of great robustness and infinite capacity: it’s called S3. So each system developer can put together the bones – instantly expandable computing capacity and bulletproof data storage – to make the body he needs. There is no need to contaminate EC2 by building into it assumptions about whether persistent storage is needed, and how much of it is needed, and how it should be administered. Each developer can make the decisions that are right for him and not be constrained by someone else’s idea of what he ought to want.
This series has been about applying the excitement generated by S3 (and now, by EC2) to doing real work in the real world. It has been about reconciling the romantic thrill of “darling, we’re going to have a baby” with the reality of coping with something damp that yells.
For server-based services such as tunesafe.com, S3 is ideal. The user interacts with the web server but all big data transfers are shunted directly into and out of S3. S3 costs money, of course; but it costs so little money that in many cases (such as probably tunesafe.com) it is possible to charge a flat fee because the wholesale data storage and data transfer costs can be bundled up into a single retail offering that still has an attractive price.
For startups that want a million users but don’t want to issue a million invoices a month, the availability of premium-rate storage (“golden buckets”) would be a major enabling technology. The profit from the golden buckets would pay for software development and the maintenance of the necessary servers; Amazon would take their cut; the user would have just one monthly bill to pay; and again, because S3 is so cheap, even a premium-rated version of it would be competitive and affordable. Whole new business models will be possible as a result. The demand is certainly there from developers: we all hope that Amazon will get round to providing this option.
No business plan is complete without some assessment of risk. For a start, Amazon need to find a way of convincingly reassuring us that we are not betting our companies on Amazon’s continuing interest and benevolence. Possible reassurances include the provision of limited guarantees of continuity or the availability of rival services that behave identically to S3. It might even be in Amazon’s interest to help set up such services.
S3 is claimed to be reliable but there are no figures to support this and users of S3 are thus taking on an unquantified risk. Quantifying the risk by publishing data is one way forward. A more interesting approach is to offer insured data storage for a small extra premium. Apart from being potentially extremely profitable, this could catalyse the transformation of the computing industry into one that takes responsibility for its products.
The contract under which S3 is currently offered is entirely unsuited to its purpose. It is not only unacceptable to anyone who plans to depend on S3, it also contradicts everything we know about Amazon’s intentions and corporate philosophy. It needs to be torn up and rewritten from scratch.
Amazon intend to spread their data centres worldwide so that computing capacity (EC2) and storage capacity (S3) are available at equally high speeds all round the world. They must pay serious attention to the legal consequences of storing data in various countries because there is no way for a user to control where, physically, data are stored in the S3 network. There is no point in virtualising away the technical differences between storage technologies if you cannot similarly virtualise away the legal differences between various jurisdictions. It is impossible to expect every Amazon S3 user to ensure that his data comply with the laws of the United States, Britain, Spain, China, and so on. In a world where every country tries to extend its jurisdiction over the whole of the Internet, we need Amazon to work on our behalf to obtain some sort of extraterritorial status for our data.
Let me finish by referring back to the start of this posting. With S3 and EC2, Amazon are revitalising computing by bringing it back to its roots. They have started a revolution. If some of my criticisms have sounded harsh, it is because I am applying some of Amazon’s own perfectionism to the services they have created and because I think that they deserve a large share of the profits from the revolution that they have started.