Posts Tagged AWS
Update: This post was written in 2011 when Windows Azure was PaaS only. The Spring 2012 IaaS features now allow more direct comparisons, but the PaaS principles discussed in this post are still relevant.
AWS is the market leader of cloud computing, by virtue of its early entry, rapid innovation and monopolisation of the definition of cloud computing. Microsoft is a viable contender with their established customers, channels and developers as well as a sound offering backed by a company that is “All in”.
Little surprise then that customers, particularly those with a Microsoft bias, that are looking at public cloud platforms want to compare the de-facto standard of AWS with the apparently familiar world of Windows Azure. Business then turns to technical people for an opinion and a simple explanation of the differences between them.
On the surface, a technical person can draw parallels. Windows Azure (AppFabric) Queues map to Amazon Simple Queue Service. SQL Azure maps to Amazon Relational Database Service and, more recently, Windows Azure AppFabric Caching maps to Amazon ElastiCache, and so on. But those are services offered on the platform, so while you can compare and map individual services, it becomes difficult to provide a technical comparison of the overall platforms.
The fundamental building block of AWS applications is the EC2 instance and on Windows Azure it is the compute role (mostly web and worker roles). EC2 is a fully fledged virtual machine and Azure roles are containers for compute resources. (Drawing parallels between EC2 and Windows Azure VM Roles does a disservice to EC2, as Azure VM roles are not the building block of Azure applications.)
The importance of EC2 as a basic building block breaks down most of the technical comparisons between the two platforms. One may, for example map Windows Azure Table Storage, as a NoSQL database, to Amazon SimpleDB. But, with EC2 as a building block, SimpleDB is not the popular NoSQL choice on AWS, where MongoDB and Redis run well. Also, EC2 allows anything to be run (although some may not be recommended), giving EC2 an unfair edge over Windows Azure. You can, for example, run SOLR on Linux for search on EC2 and it manifests itself as a first class service, whereas no search service exists on the Windows Azure platform.
In turn, Windows Azure roles, as the container for applications and the basic building block, have advantages over EC2. Windows Azure roles require significantly less hand holding, configuration and management than EC2. There is no Chef for Windows Azure. The Windows Azure AppFabric takes care of keeping the roles running smoothly. (AWS is catching up with CloudFormation and the Java oriented Elastic Beanstalk, but it still cannot be compared to the Windows Azure fabric controller).
It would be easy to say that Windows Azure is PaaS and Amazon Web Services is IaaS, so that is where the comparison should be made. But, firstly AWS is not IaaS, and secondly all cloud vendors are moving further up the stack, providing more services that can be consumed instead of the infrastructure services. Amazon’s ElastiCache is a good example of this, where a component that previously had to be hosted in EC2 instances is now made available as a service. AWS will continue, I am sure, to add more services and the infrastructure appearance of their model will be unimportant in a few years’ time. At the same time, Microsoft has the opportunity and engineering resources to add really good services and even entire application suites to their platform — further confusing the comparisons.
So when business asks for a comparison between the two, work with them to do a higher level comparison that makes sense in their particular environment. Don’t cheat by trying to map low level services, features and prices — they look eerily similar at that level and you won’t find the droids you are looking for. Rather try something like this,
- Map out a general architectural picture for each of the platforms
- Decide which components are less architecturally significant, so that compromises can be made (and there are always compromises)
- Factor in the business and internal IT attitude towards Microsoft, anti-Microsoft and Amazon
- Look at existing IT support processes and how they would fit in with each platform.
- Work out, more or less, what the development, hosting and support costs will be for each platform. Comparing EC2 pricing to Azure Role pricing is not enough.
- Look at the application roadmap to determine if it looks like it will add high degrees of specialisation, forcing it down the AWS route, or integration with existing services and applications, tending it more towards Windows Azure.
- Get internal IT on board, they probably won’t like either of them, so you need to get working on them early.
Lori MacVittie has an excellent post ‘When Black Boxes Fail: Amazon, Cloud and the Need to Know‘ on the sparseness of documentation from Amazon on how their availability zone failure triggers work.
“…most customers didn’t – and still don’t – know how Availability Zones really work and, more importantly, what triggers a fail over. What’s worse, what triggers a fail back? Amazon’s documentation is light. Very light. Like cloud light.”
As someone who is trying to work with cloud platforms at the architectural level, I echo the lack of detailed documentation on how things work. Without clear documentation it is difficult to make the correct architectural decisions.
Amazon is not alone in this. Yesterday I had a conversation with a well respected database professional who was shocked to hear that an Azure VM role could be ‘whacked’ (my term) for any ‘ol reason and without warning. This had a direct impact on what he was proposing in his architecture and is something that is not immediately obvious in the documentation.
The lack of documentation is probably a combination of a number of factors:
- The dynamic nature of the environment, which makes documentation outdated. Nobody wants outdated documentation hanging around on the Interwebs – it has a tendency not to forget.
- A degree of trade secrets being protected. Detailed documentation gives an insight into how things work.
- Not having enough focus (or people) on producing usable documentation.
- Most documentation has to pimp up the services and make them look good. As much as AWS tells you to design for failure they don’t want to drum into that their service is prone to failure. I’m sure a lot of documentation gets the once-over by marketing – which is a bad thing for architects.
We are not yet at the nirvana of simply choosing a service and having the implementation abstracted away from us. The underlying infrastructure still seems to leak through the abstraction – to a lesser degree for some services (S3) than others (EBS). So because we have leaky abstractions we need to understand them in more detail in order to make the correct architectural decisions.
Even if we could get services ‘a la carte’, we would need to know more about them, such as expected latency, throughput, availability etc – it is, after all, an attribute of the service. You won’t find documentation on that either.
Documentation that has been assembled by other vendors (as 10gen create for mongoDB on AWS) and can be found in forums is indicative of the lack of, and desire for, more documentation. Documentation that is less light like clouds.
I’ve found myself stuck in meetings trying to come up with an answer to a question like this:
“If we have no first line support with AWS, how do we know to get a second line support out of bed at 3am to fix things if they go wrong”
This opens up an interesting debate about the value of automation in a resilient and available system versus the ability to have somebody on hand who can own a problem and marshal the troops to get it fixed.
In this post on my personal blog I speculate on what would be needed to automate first line support and came up with a system that contains
- On Duty Schedule
- Voice Dialler and IVR System
- “I’m Fixing” Mode
- Host Platform Integration
I really wish someone was building one of these.
A little furore erupted over Gartner’s famous and influential Magic Quadrant dissing the Amazon cloud in response to the report that Gartner published placing Amazon high on vision and low on execution. This was followed by the Gartner Analyst, Lydia Leong, trying to ‘splain herself.
The simple fact is that Gartner decided, when analysing the cloud industry, to lump Cloud IaaS AND Web Hosting (emphasis theirs) together. This is obscure and reflects quite badly on Gartner’s understanding about cloud computing. Very few cloud computing specialists will throw IaaS and web hosting together – in fact we spend most of our time trying to differentiate the two. It is shocking that Gartner has defined cloud computing so poorly and they definitely lose some credibility with that kind of definition.
Gartner’s target market does need to be considered when figuring out what this all means. Their market is enterprise CIO’s – you know the kind of people that have the money to spend on their misguided and overpriced opinion. This does not even include business owners in enterprises who are kept in the dark about Gartner reports, lest the CIO looses some power through sharing of said information. (Never mind the rest of us that have access to the Internet and blogs and tweets of well respected thinkers in order to come to our own opinion on the market leaders). Gartner is not for the rest of us and in the context of the Enterprise CIO, it makes perfect sense, after all, enterprise CIO’s are the ones confusing cloud computing and traditional web hosting.
Gartner magic quadrants are maybe useful for determining the market leaders of 90’s technology such as, say, whether SAP HR is better than Peoplesoft. Gartner has illustrated that their grasp of cloud computing is tenuous and irrelevant – it is just a shame that people pay attention to them.
The recent takedown of Wikileaks data from Amazon Web Services and the delayed and wishy-washy response from AWS has caused quite a stir in the tech community. Ultimately, Amazon had little choice; the last thing the world’s Internet retailer wants in the run-up to Christmas is a threat from a US Senator calling for a boycott of everything ‘Amazon’ and a backlash of American Patriotism.
The outcome of Wikileaks putting any services of AWS, particularly on US based servers and the announcement of DOS attacks, was inevitable. So why did Wikileaks do it? Surely there was no real need? Wikileaks could have planned for a DOS attack and used any number of mechanisms to manage it, there is not real need to buy AWS services – they could have planned for and used any number of servers/services/hosts (even Uruguay apparently). Surely the nature of their business means that they have reasonably good anti-DOS skills? Besides, is the unavailability of the Wikileaks servers a big deal? Wikileaks has been working with news organizations and most people have been getting the news from mainstream media and not trawling through thousands of inane diplomatic cables by themselves. I know that Wikileaks was under DOS attack, but I didn’t notice as I didn’t go to their site. Wikileaks simply doesn’t rely on their own site being available in order to do ‘business’. In fact, their site is currently down, but their message is top of the BBC News.
WikiLeaks states that
its “primary interest is in exposing oppressive regimes in Asia, the former Soviet bloc, Sub-Saharan Africa and the Middle East, but we also expect to be of assistance to people of all regions who wish to reveal unethical behavior in their governments and corporations.”
I wouldn’t be surprised in Wikileaks sees Amazon fitting into that overall picture. Amazon is, after all, a large corporation that is bound to have secrets and some ‘evil’ practices. At the very least their waning support of net neutrality should be enough to fire up proponents of Internet and information freedom. Indeed, Amazon intends to dominate the ebook and surely would be partial to preferred service for Kindles. Even the removal today of the Wikileaks DNS entry is enough to raise a few net neutrality eyebrows.
There is no doubt that Wikileaks knows a thing or two about marketing and using the media; and they broadcast themselves to the world instead of hiding in some dark corner of the Internet. I’m sure that Amazon would rather Wikileaks never landed up on their servers in the first place and would rather not have had to make the decision that they have. Perhaps Wikileaks played Amazon as a way to show them up and create some bad publicity, difficult questions and lingering concerns about their intentions for what Wikileaks may believe is unethical behaviour by a corporation.
According to TechCrunch, Amazon has taken the WikiLeaks website off its EC2 servers –
“Wikileaks’ illegal, outrageous, and reckless acts have compromised our national security and put lives at risk around the world. No responsible company – whether American or foreign – should assist Wikileaks in its efforts to disseminate these stolen materials. I will be asking Amazon about the extent of its relationship with Wikileaks and what it and other web service providers will do in the future to ensure that their services are not used to distribute stolen, classified information.”
Regardless of your personal, political or patriotic position on the WikiLeaks saga; as a consumer of cloud computing services you have to feel comfortable that your business is your business and should only be taken down by someone that has legal authority – not your hosting provider under pressure from their government.
This action plays to the fears of geographic location of your data and the influence of the authorities and laws in the country that happens to host your servers. Or, as in this case, the location of the business that hosts your servers – regardless of where the data is hosted.
I have always thought that, in terms of application hosting, your data is more safe in Zimbabwe than the US – at least Zimbabwe would give politicians the finger. What we politically consider to be rogue nations are, in terms of ownership of data assets, the friendliest and the rogue nations are those that exert their influence over something belonging to someone else. Also, lets bear in mind, we are not talking about Amazon making the disputed data unavailable, but the whole damned system – they have turned off the running processes.
Did WikiLeaks violate the licence agreement with Amazon? Apparently it did because Amazon considered the data to be ‘stolen’. Did Amazon satisfy their SLA, or did they just turn it off and leave WikiLeaks hanging? Since when does American patriotism override contractual (implied or otherwise) arrangements?
I’m not trying to debate the legal issues of the hosting or the WikiLeaks data. I am observing that if the data is on-premise only a court order, issued in the country that the data resides, can take it down. On the public cloud, it seems that data can be taken down without any legal process. Your applications and data are simply not safe on the public cloud. Period.
Good luck to anyone who has a meeting lined up with executives to tell them that their applications are safe running on any public cloud.
Amazon have fucked us all over with this one.
Update 1: Carl Brooks has more detailed coverage of the story and suggests that “AWS was supplying only incidental, not critical, services to WikiLeaks”, so the takedown is technically not a big deal. But the perception about public cloud vulnerability remains – which what Senator Lieberman wants.
Update 2: Amazon’s response is now here
Follow up: Did Wikileaks Outwit Amazon?
Amazon’s own flavour of Linux optimised for AWs has just been updated to Amazon Linux AMI 2010.11.1 .
At last the fedora team are fully committed to providing timely AMI’s for use on AWS with the release of Fedora 14
Fedora has that all important ability to run anywhere not just EC2 while the Amazon Linux version is fully supported by Amazon. The fact they derive from similar code bases makes deciding between them an interesting dilemma.
The register has been keeping track of RedHat and it’s acquisition of Makara which is interesting as it means RedHat now have a viable cloud application framework “that will be integrated with the RHEL, RHEV, and JBoss tools to make a “cohesive Red Hat solution” According to Scott Crenshaw, general manager of Red Hat’s cloud business unit. The press release can be seen here