Archive for category Windows Azure
Too often we see cloud project fail, not because of the platforms or lack of enthusiasm, but from a general lack of skills on cloud computing principles and architectures. At the beginning of last year I looked at how to address this problem and realised that some guidance was needed on what is different with cloud applications and how to address those differences.
The result was a book that I wrote and published “CALM -Cloud ALM with Microsoft Windows Azure”, which takes the approach that implementation teams know how to do development, they just don’t know how to do it for the cloud, and they need to adopt cloud thinking into their existing development practices.
The “with Windows Azure” means that the book has been written with specific examples of how problems are solved with Windows Azure, but is not necessarily a book about Windows Azure — it applies as much to AWS (except you would have to figure out the technologies that apply yourself).
CALM takes an approach to look at certain models and encourages filling in the detail of the models in order to come up with the design. The models include the lifecycle model, which looks at load and traffic over time, the availability model, data model, test model and others. In looking at the full breadth of ALM (not just development), some models apply to earlier stages (qualify and prove), as well as post-delivery models, such as the deployment, health and operational models.
CALM is licensed as open source, which also means that it is free to download, read and use. It is available on github at github.com/projectcalm/Azure-EN, with pdf, mobi (Kindle), and raw html available for download on this share. A print version of the book is also available for purchase on Lulu.
I encourage you to have a look at CALM, let others know about it, ask any questions, and give me some feedback on how it can be made better.
Microsoft’s biggest strength has always its partner network and it seemed, at least for a couple of decades, that a strong channel was needed to get your product into the market. Few remember the days where buyers only saw products in computer magazines, computer trade shows and the salespeople walking through the door — the first two no longer exist and the last one may be on its last legs.
The overriding reaction from Microsoft’s Surface announcement was the snubbing of their traditional OEM partners and building a device on their own. As a consumer I think it’s a great idea — the Dell that I use for work is the sorriest excuse for a premium laptop and the more Microsoft can own the hardware and drivers that support their OS, the better. Surface, and Microsofts ownership of the design, manufacturing, distribution, sales and support of the device is a response to the pressure that they are under from the iPad — and having OEM partners messing around with their own hardware, with Android shoehorned in, obviously doesn’t work for them.
There is more to this story than consumer tablet devices. What about the rest of their business? What about the channel for enterprise software? In terms of traditional enterprise accounts I don’t expect much movement yet — there is a well established channel and an organisational culture within Microsoft to push product through that channel. I don’t expect that enterprises are going to be buying Windows Server for the datacentre from a local Microsoft store any time soon. This is probably because the channel supplier is already camped out in the data centre and rolls everything up in services and support contracts that are attractive to the enterprise (or at least appear to be).
What is going to change the Microsoft to partner to customer channel is the cloud — and it is already happening. Focussing on SaaS for a moment, consider the ‘buy direct’ model of Office365. Customers can go direct to Microsoft and get thousands of seats without going through a partner. The most engagement that they will have with partners is for migration or configuration of AD. Increasingly we are seeing Microsoft cash cow products (Exchange and Office) being sold direct and this trend will only continue. Should anything else be expected of Microsoft? With Google going direct and promising customers all sorts of magic and unicorns, why would Microsoft do anything less? Should they rely on the partner channel that either cowers in the corner with cloud phobia or is still out having an expensive lunch funded by multi year MOLP agreements and software assurance plans? The answer, of course, is no. Microsoft should not hang around while partners are milling about and Google is eating their lunch. Just like Surface, where Microsoft has to take the fight to Apple, Microsoft has to go direct because relying on (OEM) partners hasn’t been working too well.
Not only can customers buy Office365 direct, but they can also buy Windows Azure direct. With the new IaaS oriented VMs, it makes it easier to buy direct than go through the channel for on-prem Windows Servers. Microsoft has tried for years to get partners on board with Windows Azure but it has largely failed. That may be because partners don’t understand Windows Azure, but also because Microsoft is making the money (somehow) on the sales. Without the sale of the hardware, OS, networking, installation, configuration and support of a traditional server, there seems to be very little meat left on the bone for the channel — even if a few pennies are thrown their way by skimming a percentage of the monthly Windows Azure bill. To blame the channel and say that it has been lethargic is not entirely fair — Windows Azure effectively blows their business models out of the water, so it is unsurprising that they failed to embrace it with open arms. As with the SaaS offerings, what is Microsoft to do? Amazon Web Services goes direct and customers go direct to Amazon, without waiting for their incumbent IT suppliers to recommend AWS. In the face of that competition, Microsoft has little choice than to go direct and act like a huge multinational with huge investments in infrastructure, as Amazon does, rather than leaving it up to local partner minnows.
If the Microsoft channel continues to collapse (and I believe it will), when does Microsoft stop? Do they offer more and more professional services on top of their SaaS, PaaS and IaaS? I believe that they have little choice. If a customer buys Office365 direct from Microsoft, who would they choose to do the installation and support? The supplier of the service, or a partner? I believe most customers would choose to get the professional services from Microsoft — after all, they would reason, the Microsoft professional services have more direct access to the people looking after the physical infrastructure than partners would (and they would be right). Should Microsoft snub the channel and offer Windows Azure based software development services? Yes, if their existing channel is fixated on building apps on-prem (whether through ignorance or protecting their market).
Microsoft partners that continue to ignore cloud based offerings are going to fall by the wayside — both because Microsoft will ignore/undercut them by offering direct cloud services, or because their own customers will choose to go direct themselves (even to Google or Amazon). Partners need to work differently and re-invent themselves — find a way to add value and make money off the cloud. Microsoft in turn needs to protect those partners that are cloud-oriented (by, for example, allowing partners direct access to internal teams), after all, they need lots of partners in order to scale. There is no way that Microsoft can offer all of the professional services themselves — yet.
While many criticise the languishing 90s era Microsoft it appears that the ship is beginning to turn. Building a closed ecosystem consumer computing platform seems to be the only way to satisfy the needs of the consumer market (and compete with the iPad). Building out cloud services and massive computing infrastructure seems to be the way for satisfying the needs of the emerging business software market (and competing with Google and Amazon). Nobody cares as much about Microsofts survival than Microsoft does themselves, and they seem to be realizing that. Perhaps Microsoft is shedding the 90s culture and moving with the times. It is a big bet for Microsoft, but Microsoft seems poised to assert their place in the new markets and I wouldn’t bet against their ability to deliver.
One of the most significant, highly anticipated, and worst kept secrets of the Windows Azure spring release is the inclusion of persistent VMs, with the notable addition of support for Linux on those VMs.
The significance of the feature is not that high architecturally — after all, Windows Azure applications that were specifically architected for Windows Azure run well already. The aspects that I find more significant are,
- Closing the gap to AWS — It is has always been difficult to compare Windows Azure and AWS because of the IaaS bias of AWS versus Windows Azure. With the addition of persistent VMs, the two platforms can be better compared and better choices made.
- Base understanding — Windows Azure is widely misunderstood, largely due to its PaaS nature. In the face of this misunderstanding, AWS as the de-facto choice, and the more common understanding of IaaS, has been easy. The addition of persistent VMs allows decision makers to go with something that is more familiar before branching out into some of the specific Windows Azure features (as customers moving to AWS tend to do).
- Not just Windows — The inclusion of Linux is a big deal for Microsoft. Regardless of Microsoft’s own reasons, having first-class support of Linux breaks the perception that Windows Azure is Windows and .NET only. Support of Java, Node.js, Ruby and now Python under Windows Azure now has more credibility with the addition of Linux to the stable.
- Architectural choices — I’ve never been a fan of running everything under the Windows Azure ‘role’ model. Running something like MongoDB or Solr in this way just seems wrong. The addition of persistent VMs now gives architects the chance to deploy technologies that work well under Linux, where there is better support and understanding of how they run. Building a solution with MongoDB running on Linux on Windows Azure is architecturally significant and very useful.
- Enterprise comfort — Enterprises with legacy applications have struggled to make the move to Windows Azure and they are probably the largest drivers of the inclusion of persistent VMs (the ‘listening to our customers’ part of Microsoft). Regardless if it is a good idea or not to run SSIS or old-school SharePoint on a cloud platform, it is something that lots of people want to do. Enterprise customers can now run whatever they like, including Linux-based parts of their solutions.
- Bring your stack — When the announcement of the spring release was made yesterday I was most interested to see the flurry of accompanying press releases. I saw news from RightScale, Cloudant, Opscode and 10Gen. These, and similar, organisations are the backbone of the cloud community and their support of Windows Azure (however extensive it may be) greatly increases the reach of Windows Azure into areas of the cloud playground where the cool kids are hanging out.
It will be interesting to see, over the coming weeks, how the markets and the clouderati respond to these announcements. It was a move that Microsoft had to make and they need to get the right messages about the changes out to the market in order to gain better traction of Windows Azure.
When Amazon announced RDS for SQL Server and .NET support for Elastic Beanstalk, the response over the next few hours and days was a gushy ‘AWS cosies up to .NET developers’ or something similar. My first thought upon reading the news was “Man, some people on the Azure team must be really, really pissed at the SQL Server team for letting SQL Server on to AWS”. It’s not that AWS is not a good place for .NET people to cosy up to, and some AWS people are very cosy indeed (except for one, who’s been avoiding me for a year), but .NET people getting friendly with AWS people is bad for Azure. While it is great for .NET developers, the problem for Microsoft is that SQL RDS erodes the primary competitive advantage of Windows Azure.
AWS has been a long time supporter of the Windows and .NET ecosystem but the missing part was the lack of a good story around SQL Server. Sure, you have always been able to roll your own SQL instance, but keeping it available and running is a pain. What was lacking, until this week, was a SQL Server database service that negated the need to muck around by yourself. What was needed was a service provided by AWS that you could just click on to enable. Not only does AWS now support SQL (although not 2012 yet) it seems to superficially offer a better SQL than Microsoft does on SQL Azure. I personally think that SQL Azure is a better product and has been developed, from the ground up, specifically for a cloud environment, but that process has left it somewhat incompatible with on-premise SQL Server. AWSs RDS SQL is plain ‘ol SQL Server that everyone is familiar with, with databases bigger than 150GB, backups, performance counters and other things that are lacking in SQL Azure. While the discerning engineer may understand the subtle edge that SQL Azure has over RDS SQL, it will be completely lost on the decision makers.
AWS has recently been making feints into the enterprise market, a stalwart of more established vendors, including Microsoft. And, if AWS want to present a serious proposition to enterprise customers, they have to present a good Windows/.NET story without gaps — and it seems that they are beginning to fill in those gaps. It is particularly interesting and compelling for larger enterprises where there is a mish-mash of varied platforms, as there inevitably are in large organisations, where one cloud provider is able to take care of them all.
Windows Azure has Windows/.NET customer support at the core of its value proposition and SQL Azure is a big part of that. If you have a need for SQL Server functionality, why go to anyone other than a big brand that offers it as part of their core services (and I mean ‘service’, not just ‘ability to host or run’)? Windows Azure was that big brand offering that service, where the customer would choose it by default because of SQL support. Well, now there is another big brand with a compelling offering.
Microsoft obviously can’t go around refusing licenses for their software, and for a business that for decades has had ‘sell as many licenses as possible’ as their most basic cheerleader chant, it is virtually impossible to not sell licenses. The models for the new world of cloud computing clash right here with the old business models that Microsoft is struggling to adapt. For an organisation that is ‘all in’ on the cloud, the only ‘all in’ part of the messages that I am getting is that Microsoft wants to sell as many licenses of their products to cloud providers as possible — putting Windows Azure in a very awkward position. If it was me in the big Microsoft chair, I would have fought SQL RDS as long as possible — but hey, I’m not a highly influential sweaty billionaire, so my opinion doesn’t count and won’t make me a sweaty billionaire either.
The competitor to Windows Azure is not AWS, or AppEngine or any other cloud provider — the competitor is Windows Server, SQL Server and all the on-premise technologies that their customers are familiar with. I’m sure that Microsoft desperately wanted to get SQL onto RDS and helped as much as they could because that is what their customers were asking for (Microsoft is apparently quite big on listening to customers). I can’t help thinking that every time Microsofties went over for a meeting at the Amazon office to hammer out the details, the Azure team was left clueless in Redmond and the Amazon staff were chuckling behind their backs.
How does Microsoft reconcile their support for Windows Azure and their support for their existing customers and business models? How do they work with AWS as one of their biggest partners and competitors? While Microsoft struggles with these sorts of questions and tries to decide where to point the ship, Amazon will take whatever money it can off the table, thank you very much.
A tweet appeared in my stream mentioning an Azure and Office 365 data centre coming next year in south Africa.
I assume that this is from people attending the TechEd Africa keynote today and does refer to local servers, not just billing. No official press release yet, but I’ll keep an eye on it.
For those who don’t know where South Africa is, it is right at the bottom end of Africa and is fairly insignificant as far as computing and market strength is concerned. While it may have relevance as a developing nation and investors see it as the ‘springboard to sub-Saharan Africa’, as a technology and bandwidth hub, it is a backwater or stagnant pond.
So it is interesting that Microsoft has a business case for an Azure data centre in such a marginal market. Perhaps it is going to be powered by the Fujitsu Windows Azure Appliance (as with Japan), done in partnership with a local co-location and not a fully fledged Microsoft run data centre.
It is an indication that the market is heating up and that those with significant cash on hand are racing ahead. At the end of the day, the winner may be decided, not by the technological aspects, but by whether or not servers are local in whichever part of the world that customers may operate.
This looks like it is happening quite soon and it will be interesting to compare the list of localised data centres amongst all the players a year from now.
I was asked via email to confirm my thoughts on running MongoDB on Windows Azure, specifically the implication that it is not good practice. Things have moved along and my thoughts have evolved, so I thought it may be necessary to update and publish my thoughts.
Firstly, I am a big fan of SQL Azure, and think that the big decision to remove backwards compatibility with SQL Server was a good one that enabled SQL Azure to rid itself of some of the problems with RDBMSs in the cloud. But, as I discussed in Windows Azure has little to offer NoSQL, Microsoft is so big on SQL Azure (for many good reasons) that NoSQL is a second class citizen on Windows Azure. Even Azure Table Storage is lacking in features that have been asked for for years and if it moves forward, it will do so grudgingly and slowly. That means that an Azure architecture that needs the goodness offered by NoSQL products needs to roll in an alternative product into some Azure role of sorts (worker or VM role). (VM Roles don’t fit in well with the Azure PaaS model, but for purposes of this discussion the differences between a worker role and VM role are irrelevant.)
Azure roles are not instances. They are application containers (that happen to have some sort of VM basis) that are suited to stateless application processing – Microsoft refers to them as Windows Azure Compute, which gives a clue that they are primarily to be used for computing, not persistence. In the context of an application container Azure roles are far more unstable than an AWS EC2 instance. This is both by design and a good thing (if what you want is compute resources). All of the good features of Windows Azure, such as automatic patching, failover etc are only possible if the fabric controller can terminate roles whenever it feels like it. (I’m not sure how this termination works, but I imagine that, at least with web roles, there is a process to gracefully terminate the application by stopping the handling of incoming requests and letting the running ones come to an end.) There is no SLA for a Windows Azure compute single instance as there is with an EC2 instance. The SLA clearly states that you need two or more roles to get the 99.95% uptime.
For compute, we guarantee that when you deploy two or more role instances in different fault and upgrade domains your Internet facing roles will have external connectivity at least 99.95% of the time.
On 4 February 2011, Steve Marx from Microsoft asked Roger Jennings to stop publishing his Windows Azure Uptime Report
Please stop posting these. They’re irrelevant and misleading.
To others who read this, in a scale-out platform like Windows Azure, the uptime of any given instance is meaningless. It’s like measuring the availability of a bank by watching one teller and when he takes his breaks.
Think, for a moment, what this means when you run MongoDB in Windows Azure – your MongoDB role is going to be running where the “uptime of any given instance is meaningless”. That makes using a role for persistence really hard. The only way then is to run multiple instances and make sure that the data is on both instances.
Before getting into how this would work on Windows Azure, consider for a moment that MongoDB is unashamedly fast and that speed is gained by committing data to memory instead of disk as the default option. So committing to disk (using ‘safe mode’) or a number of instances (and their disks) goes against some of what MongoDB stands for. The MongoDB api allows you to specify the ‘safe’ option (or “majority” in 2.0, but more about that later) for individual commands. This means that you can fine tune when you are concerned about ensuring that data is saved. So, for important data you can be safe, and in other cases you may be able to put up with occasional data loss.
(Semi) Officially MongoDB supports Windows Azure with the MongoDB Wrapper that is currently an alpha release. In summary, as per the documentation, is as follows:
- It allows running a single MongoDB process (mongod) on an Azure worker role with journaling. It also optionally allows for having a second worker role instance as a warm standby for the case where the current instance either crashes or is being recycled for a normal upgrade.
- MongoDB data and log files are stored in an Azure Blob mounted as a cloud drive.
- MongoDB on Azure is delivered as a Visual Studio 2010 solution with associated source files
There are also some additional screen shots and instructions in the Azure Configuration docs.
What is interesting about this solution is the idea of a ‘warm standby’. I’m not quite sure what that is and how it works, but since ‘warm standby’ generally refers to some sort of log shipping and the role has journaling turned on, I assume that the journals are written from the primary to the secondary instances. How this works with safe mode (and ‘unsafe’ mode) will need to be looked at and I invite anyone who has experience to comment. Also, I am sure that all of this journaling and warm standby has some performance penalty.
It is unfortunate that there is only support for a standalone mode as MongoDB really comes into its own when using replica sets (which is the recommended way of deploying it on AWS). One of the comments on the page suggests that they will have more time to work on supporting replica sets in Windows Azure sometime after the 2.0 release, which was today.
MongoDB 2.0 has some features that would be useful when trying to get it to work on Windows Azure, particularly the Data Centre Awareness “majority” tagging. This means that a write can be tagged to write across the majority of the servers in the replica set. You should be able to, with MongoDB 2.0, run it in multiple worker roles as replicas (not just warm standby) and ensure that if any of those roles were recycled that data would not be lost. There will still be issues of a recycled instance rejoining the replica set that need to be resolved however – and this isn’t easy on AWS either.
I don’t think that any Windows Azure application can get by with SQL Azure alone – there are a lot of scenarios where SQL Azure is not suitable. That leaves Windows Table Storage or some other database engine. Windows Table Storage, while well integrated into the Windows Azure platform, is short on features and cloud be more trouble than it is worth. In terms of other database engines, I am a fan of MongoDB but there are other options (RavenDB, CouchDB) – although they all suffer from the same problem of recycling instances. I imagine that 10Gen will continue to develop their Windows Azure Wrapper and expect that a 2.0 replica set enabled wrapper would be a fairly good option. So at this stage MongoDB should be a safe enough technology bet, but make sure that you use the “safe mode” or “majority” sparingly in order to take advantage of the benefits of MongoDB.
Update: This post was written in 2011 when Windows Azure was PaaS only. The Spring 2012 IaaS features now allow more direct comparisons, but the PaaS principles discussed in this post are still relevant.
AWS is the market leader of cloud computing, by virtue of its early entry, rapid innovation and monopolisation of the definition of cloud computing. Microsoft is a viable contender with their established customers, channels and developers as well as a sound offering backed by a company that is “All in”.
Little surprise then that customers, particularly those with a Microsoft bias, that are looking at public cloud platforms want to compare the de-facto standard of AWS with the apparently familiar world of Windows Azure. Business then turns to technical people for an opinion and a simple explanation of the differences between them.
On the surface, a technical person can draw parallels. Windows Azure (AppFabric) Queues map to Amazon Simple Queue Service. SQL Azure maps to Amazon Relational Database Service and, more recently, Windows Azure AppFabric Caching maps to Amazon ElastiCache, and so on. But those are services offered on the platform, so while you can compare and map individual services, it becomes difficult to provide a technical comparison of the overall platforms.
The fundamental building block of AWS applications is the EC2 instance and on Windows Azure it is the compute role (mostly web and worker roles). EC2 is a fully fledged virtual machine and Azure roles are containers for compute resources. (Drawing parallels between EC2 and Windows Azure VM Roles does a disservice to EC2, as Azure VM roles are not the building block of Azure applications.)
The importance of EC2 as a basic building block breaks down most of the technical comparisons between the two platforms. One may, for example map Windows Azure Table Storage, as a NoSQL database, to Amazon SimpleDB. But, with EC2 as a building block, SimpleDB is not the popular NoSQL choice on AWS, where MongoDB and Redis run well. Also, EC2 allows anything to be run (although some may not be recommended), giving EC2 an unfair edge over Windows Azure. You can, for example, run SOLR on Linux for search on EC2 and it manifests itself as a first class service, whereas no search service exists on the Windows Azure platform.
In turn, Windows Azure roles, as the container for applications and the basic building block, have advantages over EC2. Windows Azure roles require significantly less hand holding, configuration and management than EC2. There is no Chef for Windows Azure. The Windows Azure AppFabric takes care of keeping the roles running smoothly. (AWS is catching up with CloudFormation and the Java oriented Elastic Beanstalk, but it still cannot be compared to the Windows Azure fabric controller).
It would be easy to say that Windows Azure is PaaS and Amazon Web Services is IaaS, so that is where the comparison should be made. But, firstly AWS is not IaaS, and secondly all cloud vendors are moving further up the stack, providing more services that can be consumed instead of the infrastructure services. Amazon’s ElastiCache is a good example of this, where a component that previously had to be hosted in EC2 instances is now made available as a service. AWS will continue, I am sure, to add more services and the infrastructure appearance of their model will be unimportant in a few years’ time. At the same time, Microsoft has the opportunity and engineering resources to add really good services and even entire application suites to their platform — further confusing the comparisons.
So when business asks for a comparison between the two, work with them to do a higher level comparison that makes sense in their particular environment. Don’t cheat by trying to map low level services, features and prices — they look eerily similar at that level and you won’t find the droids you are looking for. Rather try something like this,
- Map out a general architectural picture for each of the platforms
- Decide which components are less architecturally significant, so that compromises can be made (and there are always compromises)
- Factor in the business and internal IT attitude towards Microsoft, anti-Microsoft and Amazon
- Look at existing IT support processes and how they would fit in with each platform.
- Work out, more or less, what the development, hosting and support costs will be for each platform. Comparing EC2 pricing to Azure Role pricing is not enough.
- Look at the application roadmap to determine if it looks like it will add high degrees of specialisation, forcing it down the AWS route, or integration with existing services and applications, tending it more towards Windows Azure.
- Get internal IT on board, they probably won’t like either of them, so you need to get working on them early.