Amazing Datacentre Architectures

I read the book "The Google Story" by David Vise and Mark Malseed on a business trip to cambodia a few months ago. It's enjoyable for it's anecdotes and little known facts about Brin and Page, but comes across as something written by fanboys or a TV documentary script aimed at the masses, rather than serious authors. The one section that made the whole book worthwhile for me talked about how the Google founders put together their own servers, made from off the shelf parts from Frys, and built an operating system that allowed all these industry standard computers to work well together, tolerate failures of the nodes and self heal. The system basically kept running and searching and indexing, while maintaining data integrity and accepting new nodes as these were put together and added to the pile.

I was reminded of the Google datacentre architecture when reading a blog post by Robin Harris of ZDNet.

Inside Amazon by ZDNet's Robin Harris -- You’re running one of the world’s busiest e-commerce sites, handling up to 4 million checkouts per day. Response time is critical. Every page is customized on the fly using over 150 network services. And the system must manage failures of any component, including entire data centers. You are taking real money and shipping real goods. [...]

I've always thought of Amazon as a retailer who was really smart about how it got me to buy more books, movies and music from them than I intend to when I visit their website. I now have new found respect for them as computer system designers as well. This is a side of Amazon I never realised existed.

Suppose the Google and Amazon datacentre architectures are the way of the future.......

What would things be like on the desktop? I'm willing to bet we'll all have 1cmx1cm storage cubes we carry around that we'd connect wirelessly to terminals scattered all over for public access, that would contain our entire desktop environments in convenient virtual machines. Much of our data would already be living on the net - in google, microsoft and yahoo servers. We'd have good tools for synchronizing this data living in the cloud with the data in our virtual machines housed in the little cubes.