# Wednesday, 25 July 2012

Microsoft recently announced the beta of Service Bus 1.0 for Windows Server. This is the on-premise version of the Azure Service Bus that so many have been asking for. There is a good walkthrough of the new beta in the MSDN documentation here including how to install it.

So why this blog post? Well if you install the beta on a Windows Server 2008 R2 box and you actually develop on another machine then there’s a couple of things not mentioned in the documentation that you need to do (having previously spent long hour glaring at certificate errors allowed me to resolve things pretty quickly)

First a little background into why there is an issue. Certificate validation has a number of steps to it for a certificate to be considered valid:

  1. The issuer of the cert must be trusted. In fact the issuer of the issuer must also be trusted. In fact the issuer of the issuer of the issuer must be trusted. In fact … well you get the picture. In other words, when a certificate is validated all certificates in the chain of issuers must be trusted back to a trusted root CA (normally bodies like Verisign or your corporate certificate server). You will find these in your certificate store in the “Trusted Root Certificate Authorities” section. This mechanism is known as chain trust.
  2. The certificate must be in date (between the valid from and valid to dates)
  3. The usage of the certificate must be correct for the supported purposes of the cert (server authentication, client authentication, code signing, etc)
  4. For a server authentication cert (often known as an SSL cert) the subject of the cert must be the same as the server DNS name
  5. The certificate must not be revoked (if a cert is compromised the issuer can revoke the cert and publishes this in a revocation list) according to the issuers revocation list

For 1. there is an alternative called Peer Trust where the presented certificate must be in the clients Trusted People section of their certificate store

There may be other forms of validation but these are the ones that typically fail

So the issue is that the installer of Service Bus (SB), if you just run through the default install, generates a Certificate Authority cert and puts it into the target machine’s Trusted CA store – it obviously doesn’t put it in the client’s Trusted CA store because it knows nothing about that. It also generates a server authentication cert with the machine name you are installing on. This causes two issues: no cert issued by the CA will be trusted and so an SSL failure will happen the first time you try to talk to SB, the generated SSL cert will have problems generally validating due to, for example, no revocation list

The first of these issues can be fixed by exporting the generated CA cert and importing it into the Trusted CA Store ion the dev machine

CA

The second can be fixed by exporting the SSL cert and importing it into the trusted people store on the dev machine

SSL

With these in place the Getting Started / Brokered Messaging / QueuesOnPrem sample should work fine

.NET | Azure | ServiceBus | WCF
Wednesday, 25 July 2012 17:04:54 (GMT Daylight Time, UTC+01:00)  #    Disclaimer  |   | 
# Friday, 02 October 2009

I just got back from speaking at Software Architect 2009. I had a great time at the conference and thanks to everyone who attended my sessions. As promised the slides and demos are now available on the Rock Solid Knowledge website and you can get them from the Conferences page.

.NET | Azure | REST | RSK | WCF | WF
Friday, 02 October 2009 22:12:18 (GMT Daylight Time, UTC+01:00)  #    Disclaimer  |   | 
# Tuesday, 14 July 2009

It has been one of the bug bears for me as I have given talks on the Azure platform that I have not been able to answer any question that involved commercial details with anything other than “we’ll have to see the prices when they announce them. Finally, today, the team have announced the pricing model and availability

http://blogs.msdn.com/windowsazure/archive/2009/07/14/confirming-commercial-availability-and-announcing-business-model.aspx

.NET | Azure
Tuesday, 14 July 2009 15:46:36 (GMT Daylight Time, UTC+01:00)  #    Disclaimer  |   | 
# Wednesday, 10 June 2009

I am one of the moderators of the MSDN WCF Forum. One of the main areas of questions on the forum is duplex messaging – particularly using the WSDualHttpBinding. So instead of typing long messages repeating the same thing in answer to these questions I’ve decided to write this blog post to give a bit of background about duplex messaging and then discuss the options for bindings and common problems people have.

What is Duplex Messaging?

There are many ways that messages can be exchanged between two parties in a service based system: the client can send messages to the server and never get any back; the client can send a message and wait for a response; the client and service can send eachother messages without any pre-defined pattern; the client can send the service a message but not wait synchronously for a response and then then service can send a message back asynchronously; and there are many others. However, the first three of these are supported natively in WCF and are known as One-way, request/response and duplex.

So Duplex messaging is where, unsolicited, the client and service can send eachother messages. Most commonly this is characterized by the service sending the client “events” or notifications or progress of “interesting things”.

Duplex Contracts in WCF

To send messages to eachother the client and service must have an idea of what operations are available and what messages are sent and received during the communication. In WCF this idea is modelled by the contract. Now normally a contract just determines what functionality is available at the service. However, now the service is going to be sending messages to the client that the client isn’t specifically waiting for so it needs an idea of what messages the client can deal with. So we need a contract that models both directions of the conversation.

A bi-directional contract is modelled using two interfaces bound together with a ServiceContract – like this:

[ServiceContract(CallbackContract=typeof(IPizzaProgress))]
interface IOrderPizza
{
    [OperationContract]
    void PlaceOrder(string PizzaType);
}
 
 
interface IPizzaProgress
{
    [OperationContract]
    void TimeRemaining(int minutes);
 
    [OperationContract]
    void PizzaReady();
}

The import bit here is the CallbackContract that establishes the relationship between the service’s and client’s contracts.

Writing the Service

The service is implemented normally apart from two issues: firstly it needs to access the callback contract to be able to send messages back to the client; secondly the communication infrastructure (modelled by the binding) needs to be able to cope with duplex messaging. Firstly lets look at accessing the callback contract:

class PingService : IOrderPizza
{
    IPizzaProgress callback;
 
    public void PlaceOrder(string PizzaType)
    {
        callback = OperationContext.Current.GetCallbackChannel();
 
        Action preparePizza = PreparePizza;
        preparePizza.BeginInvoke(ar => preparePizza.EndInvoke(ar), null);
    }
 
    void PreparePizza()
    {
        for (int i = 10 - 1; i >= 0; i--)
        {
            callback.TimeRemaining(i);
            Thread.Sleep(1000);
        }
        callback.PizzaReady();
    }
}

The critical line here is calling GetCallbackContract on the OperationContext. This gives the service access to a proxy to call back to the client.

Now the service also needs to use a contract that is compatible with duplex messaging. WSHttpBinding is the default for the built in WCF projects but it does not support duplex messaging. People generally then move to the WSDualHttpBinding which is similar to the WSHttpBinding but does support duplex. I will go into more depth about bindings for duplex shortly but for now lets stick to this for now - it will work in our test rig on a single machine without issue.

Writing the Client

If the client is going to receive these messages it needs to provide an implementation of the callback contract. It can gets its definition from either a shared contract assembly or from metadata. If using metadata the callback contract will be named the same as the service’s contract but with the work Callback appended. It will also need to supply this implementation to the WCF infrastructure and it does this by wrapping an instance in an InstanceContext object and passing it to the proxy constructor. So here is the client:

class Program
{
    static void Main(string[] args)
    {
        InstanceContext ctx = new InstanceContext(new Callback());
 
        OrderPizzaClient proxy = new OrderPizzaClient(ctx);
 
        proxy.PlaceOrder("Pepperoni");
 
        Console.WriteLine("press enter to exit");
        Console.ReadLine();
    }
}
 
class Callback : IOrderPizzaCallback
{
    public void TimeRemaining(int minutes)
    {
        Console.WriteLine("{0} seconds remaining", minutes);
    }
 
    public void PizzaReady()
    {
        Console.WriteLine("Pizza is ready");
    }
}

Running the service and the client will have this working quite happily – so it would seem that duplex messaging and WCF works very well … so why on earth do people keep asking questions about it on the WCF forums?

It Worked on My Machine but Broke when we Deployed It!

Ahh well you probably did the thing that is obvious but almost always a bad idea. You went and chose WSDualHttpBinding as your duplex binding. To understand why this is a bad idea we need to dig a little deeper into how the WSDualHttpBinding works. HTTP is a unidirectional protocol: the client makes a request and the server sends a response. There is no way for a server to initiate an exchange with the client. So how on earth is duplex messaging going to work because it requires exactly this facility? Well the “Dual” in the name is significant, the WSDualHttpBinding actually consists of two connections: one outbound from client to server and one inbound from server to client – this second connection may already be ringing alarm bells with you. The are a two big problems with inbound connections to a client: firewalls very often block inbound connections to clients; the client may not be reachable from the server, it may be using NAT translation behind a router and so cannot be contacted without port forwarding being set up on the router. Both of these issues are showstoppers in real network topologies. You can take some small steps to help – you can specify what port the client should listen on for example by using the clientBaseAddress property of the WSDualHttpBinding. This means the network admin will only have to punch one hole in their firewall (but lets face it, network admins don’t allow any holes to be punched in the firewall).

So if you really shouldn’t use WSDualHttpBinding for duplex, what should you use instead? Well NetTcpBinding supports duplex out of the box and the nice thing about this is that the outbound connection that it establishes can also be used be used for inbound traffic – suddenly we don;t have the inbound connection firewall/NAT issues. “But hold on, isn’t NetTcpBinding for intranet? I’ve read books that tell me that in their ‘which binding should I use?’ flowcharts!” Well it turns out those flowcharts are talking rubbish – NetTcpBinding works very happily over the internet, its just not interoperable by design. “Aha! but I need interop so WSDualHttpBinding is for me!” Well unfortunately not, NetTcpBinding is non-interoperable by design, WSDualHttpBinding is non-interoperable despite its design. From the name it would suggest interoperability but  Arun Gupta from Sun wrote this excellent post describing why it wasn’t.

So now seeing that we really are not talking about interop anyway, NetTcpBinding is far more useful than WSDualHttpBinding. Its not bullet proof, if the firewall only allows outbound port 80 but also allows inbound port 80, then WSDualHttpBinding would work where NetTcpBinding wouldn’t – but in this situation we’re really talking server to server and so I’d argue its probably better to roll your own bidirectional communication with two standard HTTP based connections.

The final option you have for duplex communication is to add a piece of infrastructure into the mix. The .NET Services Service Bus (part of the Azure platform) allows two parties to exchange messages both making outbound connections – potentially even using HTTP port 80. The two outbound connections rendezvous in the Service Bus which mediates their message exchanges. If the receiver has had to use outbound port 80 then it polls to receive message bound for it.

It Worked for the First 10 Clients and then the Rest Timed Out!

Irrespective if which of the standard bindings you are using, duplex assumes a constant relationship between proxy and service. In WCF this idea is modelled by the concept of session. All duplex bindings require session. A while back I wrote in some detail about sessions. You will have to either put up with increasing the session throttle (see the linked article for details) or roll your own custom binding that can do duplex without session – you can find an example of this here.

I Use the Callback While the Client is calling the Service and it Deadlocks!

This is because your client is probably a Rich Client GUI based application (Windows Forms or WPF). To understand why this is a problem we need to step back briefly and look at UI clients, threading and WCF threading. UI applications have a rule: you must only update the UI from the thread that created those UI components. In general a GUI application has one UI thread so anything that changes the UI needs to be done from that thread. .NET 2.0 introduced a new construct to simplify the process of a background thread updating the UI: SynchronizationContext. The idea is that a UI framework creates an implementation of a SynchronizationContext derived class that handles the mechanics of marshalling a call on to the UI thread. An instance of this implementation is then made available on the UI and accessible via the SynchronizationContext.Current.

WCF adds more complexity into the mix by enforcing a rule that says “unless you tell me otherwise I will only allow one thread at a time into an object that I control”. You see this with singleton services that will only allow one call at a time by default. The same is also true of the callback implementation object – so WCF will only allow one active thread in the client at a time. So while WCF is performing an outbound call it will not allow an inbound call into the object. This causes the initial problem with the deadlock that the service’s callback cannot be dispatched while the client’s outbound call is in progress. To solve this we use the “unless you tell me otherwise” part of the above rule. You do this by annotating the callback implementation class with a [CallbackBehavior] attribute like this:

[CallbackBehavior(ConcurrencyMode=ConcurrencyMode.Reentrant)]
class Callback : IOrderPizzaCallback
{
    public void TimeRemaining(int minutes)
    {
        Console.WriteLine("{0} seconds remaining", minutes);
    }
 
    public void PizzaReady()
    {
        Console.WriteLine("Pizza is ready");
    }
}

But now there is another problem: by default WCF will attempt to dispatch using an available SynchronizationContext. The problem with this callback is the UI thread is already blocked in an outbound call. SO for the call to dispatch we need to tell WCF not to use the SynchronizationContext – again using the CallbackBehavior attribute:

[CallbackBehavior(ConcurrencyMode=ConcurrencyMode.Reentrant, UseSynchronizationContext=false)]
class Callback : IOrderPizzaCallback
{
    ...
}

Now the issue is of course that the call is going to be processed on a non UI thread so you would have to manually marshal any UI interaction using the SynchronizationContext.Post method.

Duplex messaging can be a useful message exchange pattern but in WCF there can be some unexpected issues. Hopefully this blog post clarifies those issues and demonstrates workarounds for them.

.NET | Azure | WCF
Wednesday, 10 June 2009 07:54:13 (GMT Daylight Time, UTC+01:00)  #    Disclaimer  |   | 
# Wednesday, 25 February 2009

I’ve finally got round to pushing the code for BlobExplorer to CodePlex. You can find the CodePlex site here. If you want to contribute to the project, let me know at richard at nospam dotnetconsult dot co dot uk and I’ll add you to the contributors list. The releases will continue being pushed to blob storage http://dotnetconsult.blob.core.windows.net/tools/BlobExplorer.zip

Wednesday, 25 February 2009 14:59:12 (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Friday, 06 February 2009

I've just dropped a new release of Blob Explorer for managing Windows Azure Blob Storage

Two new features:

  • Blob metadata can now be added and viewed
  • There is a status bar to show that long running uploads and downloads are in progress

As always you can get it from blob storage here

http://dotnetconsult.blob.core.windows.net/tools/BlobExplorer.zip

Friday, 06 February 2009 22:39:17 (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Tuesday, 20 January 2009

Christian found a bug in BlobExplorer when the blob name has pseudo directory structure in it (common prefixes)

Fixed version is available here

Tuesday, 20 January 2009 10:10:25 (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Saturday, 10 January 2009

I'm really excited to announce that I've joined thinktecture as a consultant. I've known Ingo, Christian and Dominick for some time now and its great to take that relationship on to a new level. I have huge respect for the abilities of the thinktecture team and so it was fantastic when they asked me if I was interested in working with them.

I'll be focusing on all things distributed at thinktecture - so that includes WCF, Workflow, BizTalk, Dublin and Azure. And I guess I'll have to learn to speak German now ...

Azure | BizTalk | Dublin | Life | WCF | WF
Saturday, 10 January 2009 18:27:22 (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Monday, 15 December 2008

I've added drag and drop support to the BlobExplorer (announced here) so you can drag files from windows explorer into cloud blob storage

Rather than maintain the application in multiple places all releases will now go to the cloud here

 

.NET | Azure | WPF | BlobExplorer
Monday, 15 December 2008 15:52:11 (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 

I will finally submit to Dominick's harrassment and stop running Visual Studio elevated. I started doing it because I couldn't be bothered to mess about with HttpCfg and netsh when writing self-hosted WCF apps exposed over HTTP. Registering part of the HTTP namespace with http.sys requires admin privilege. I could have sorted it out with a wildcard registration but never got round to it.

Since writing the BlobExplorer, Christian has been my major source of feature requests. His latest was he wanted drag and drop. Now I'm not an expert in WPF so I hit google to find out how to do it. And it seemed really simple - just set AllowDrop=true on the UIElement in question and handle the Drop event. But I just couldn't get the app to accept drag events from Windows Explorer. After much frustration I finally realised that the problem had nothing to do with my code but was rather a security issue. the BlobExplorer was running elevated because I started it from Visual Studio which was also running elevated. Elevated applications will not accept drag events from non-elevated applications.

So I'm setting VS to run normally again and will do the proper thing with http.sys for my WCF applications.

.NET | Azure | WCF | WPF
Monday, 15 December 2008 12:49:41 (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Monday, 08 December 2008

I've uploaded an updated version of the BlobExplorer (first blogged about here). It now uses a decent algorithm for determining MIME type (thanks Christian) and the blob list now retains a single entry when you update an existing blob

Downloadable from my website here

BlobExplorer12.zip (55.28 KB)

downloadable from Azure Blob Storage here

.NET | Azure | BlobExplorer | REST | WPF
Monday, 08 December 2008 09:33:00 (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Sunday, 07 December 2008

The Windows Azure Storage infrastructure has three types of storage:

  • Blob - for storing arbitrary state as a blob (I recently blogged about writing an explorer for blob storage)
  • Table - for storage structured or semi-structured data with the ability to query it
  • Queue - primarily for communication between components in the cloud

This post is going to concentrate on queue storage as its usage model is a bit different traditional queuing products such as MSMQ. The code samples in this post use the StorageClient library sample shipped in the Windows Azure SDK.

Lets start with the basics: you authenticate with Azure storage using your account name and key provided by the Azure portal when you create a storage project. If you're using the SDK's deveopment storage the account name, key and uri are fixed. Here is the code to set up authentication details with development queue storage:

string accountName = "devstoreaccount1";
string accountKey = "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==";
string address = "http://127.0.0.1:10001";

StorageAccountInfo info = new StorageAccountInfo(new Uri(address), null, accountName, accountKey);

QueueStorage qs = QueueStorage.Create(info);

Queues are named within the storage so to create a queue (or address an existing queue) you use the collowing code:

MessageQueue mq = qs.GetQueue("fooq");

mq.CreateQueue();

The CreateQueue method will not recreate an already existing queue and has an overload to tell you that the queue already existed. You can then create message objects and push them into the queue as follows:

Message msg = new Message(DateTime.Now.ToString());

mq.PutMessage(msg);

Hopefully none of this so far is particularly shocking. Where things start to get interesting is on the receive side. We can simply receive a message so:

msg = mq.GetMessage(5);
if (msg != null
)
{
    Console
.WriteLine(msg.ContentAsString());
}

So what is interesting about this? Firstly notice that the GetMessage takes a parameter - this is a timeout in seconds. Secondly, GetMessage doesn't block but will return null if there is no message to receive. Finally, although you can't see it from the above code, the message is still on the queue. To remove the message you need to delete it:

Message msg = mq.GetMessage(5);
if (msg != null
)
{
    Console
.WriteLine(msg.ContentAsString());
    mq.DeleteMessage(msg);
}

This is where that timeout comes in: having received a message, that message is locked for the specified timeout. You must called DeleteMessage within that timeout otherwise the message is unlocked and can be picked up by another queue reader. This prevents a queue reader trying to process a message and then dying leaving the message locked. It, in effect, provides a loose form of transaction around the queue without having to formally support transactional sematics which tend not to scale at web levels.

However, currently we have to code some polling logic as the GetMessage call doesn't block. The MessageQueue class in StorageClient also provides the polling infrastructure under the covers and delivers the message via eventing:

mq.MessageReceived += new MessageReceivedEventHandler(mq_MessageReceived);
mq.PollInterval = 2000; // in milliseconds
mq.StartReceiving(); // start polling

The event handler will fire every time a message is received and when there are no more messages the client will start polling the queue according to the PollInterval

Bear in mind that the same semantics apply to receiving messages. It is beholden on the receiver to delete the message within the timeout otherwise the message will again become visible to other readers. Notice here we don't explicitly set a timeout and so a default is picked up from the Timeout property on the MessageQueue class (this defaults to 30 seconds).

Using the code above it is very easy to think that everything is running with a nicely tuned .NET API under the covers. Remember, however, that this infrastructure is actually exposed via internet standard protocols. All we have here in reality is a convenient wrapper over the storage REST API. And so bear in mind that the event model is just a convenience provided by the class library and not an inherent feature built into queue storage. Similarly the reason that GetMessage doesn't block is simply that it wraps an HTTP request that returns an empty queue message list if there are no messages on the queue. If you use GetMessage or eventing API then the messages are retrieved one at a time. You can also use the GetMessages API which will allows you to receive multiple messages at once.

.NET | Azure | REST
Sunday, 07 December 2008 11:04:49 (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Saturday, 06 December 2008

I've been digging around in Azure Storage recently and as a side project I decided to write an explorer for Blob Storage. My UI skills are not my strongest suit but I had fun dusting off my WPF knowledge (I have to thank Andy Clymer, Dave "no blog" Wheeler and Ian Griffiths for nursemaiding me through some issues)

Here is a screen shot of the explorer attached to development storage. You can also attach it to your hosted Azure storage in the cloud

In the spirit of cloud storage and dogfooding I used the Blob Explorer to upload itself into the cloud so you can find it here

However, as Azure is a CTP environment I thought I would also upload it here

BlobExplorer1.zip (55.08 KB)

as, on the web, URIs live forever and my storage account URIs may not after the CTP. I hope someone gets some mileage out of it - I certainly enjoyed writing it. Comments, bug reports and feature requests are appreciated

.NET | Azure | BlobExplorer | REST | WPF
Saturday, 06 December 2008 22:00:19 (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Monday, 01 December 2008

DevelopMentor UK ran a one day post PDC event on Friday at the Microsoft Offices in London. Myself and Dominick covered WF 4.0, Dublin, Geneva, Azure and Oslo

You can find the materials and demos here

.NET | Azure | Oslo | WCF | WF | Dublin
Monday, 01 December 2008 11:30:39 (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Sunday, 02 November 2008

.NET Services is a block of functionality layered on top of the Azure platform. This project has had a couple of names in the past – BizTalk Services when it was an incubation project and then Zurich as it was productionized.

 

.NET Services consists of three services:

1)      Identity

2)      The Service Bus

3)      Workflow

 

Lets talk about identity first. Officially named the .NET Access Control Service, this provides you a Security Token Service (STS) that you can use to configure how different forms of authentication map into claims (it leverages the claims construct first surfaced in WCF and now wrapped up in Geneva). It supports user ids/passwords, Live ID, Certs and full blown federation using WS-Trust. It also supports rule based authorization against the claim sets.

 

The Service Bus is a component that allows you to both listen for and send messages into the “service bus”. More concretely it means that from inside my firewall I can listen to an internet based endpoint that others can send messages into. It supports both unicast and multicast. The plumbing is done by a bunch of new WCF Bindings (the word Relay in the binding indicates it is a Service Bus binding) although there an HTTP based API too. Sending messages to the internet is fairly obvious how that happens, its the listening that is more interesting. The preferred way is to use TCP (NetTcpRelayBinding) to connect. This parks a TCP session in the service bus which it then sends messages to you down. However they also support HTTP although the plumbing is a bit more complex. There they construct a Message Buffer that they send messages into then the HTTP relay listener polls it for messages. The message buffer is not a full blown queue although it does have some of the same characteristics of a queue. There is a very limited size and TTL for the messages in it. Over time they may turn it into a full blown queue but for the current CTP it is not.

 

So what’s the point of the Service Bus? It enables to be subscribe to internet based events (I find it hard to use the word “Cloud” ;-)) to allow loosely coupled systems over the web. It also allows the bridging of on-premises systems to web based ones through the firewall

 

Finally there is the .NET Workflow Service. This is WF in the Cloud (ok I don’t find it that hard). They provide a constrained set of activities (currently very constrained although they are committed to providing a much richer set. They provide HTTP Receive and Send and Service bus Send plus some flow control and some XPath ones that allow content based routing. You deploy your workflow into their infrastructure and can create instances of it waiting for messages to arrive and to route them, etc. With the toolset you basically get one-click deployment of XAML based workflows. They currently only support WF 3.5 although they will be rolling out WF 4.0 (which is hugely different from WF 3.5 – thats the subject of another post) in the near future.

 

So what does .NET Services give us? It provides a rich set of messaging infrastructure over and above that of Windows Azure Services

.NET | Azure | WCF | WF
Sunday, 02 November 2008 15:10:04 (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Saturday, 01 November 2008

Having spent the last week at PDC - with very spotty Internet access. I thought I'd take some time to reflect on what I thought about the various announcements and technologies that I dug around in. This post is about the big announcement: Windows Azure

 

Azure (apparently pronounced to rhyme with badger) is an infrastructure designed to control applications installed in a huge datacenter. Microsoft refer to it as a “cloud based operating system” and talk about it being where you deploy your apps for Internet  scale.

 

So I guess we have to start with: what problem are Microsoft trying to solve? There are two answers here:

1)      Applications that need Internet scale are really hard to deploy due to the potentially huge hardware requirements and cost of managing that infrastructure. Most organizations that go through rapid growth experience a lot of pain as they try to scale for the 10,000s to many millions of users (Pinku has some great slides on the pain MySpace went through and how they solved it). This normally requires rearchitecting of the app to more loosely couple, etc.

2)      Microsoft have a serious threat from both Amazon and Google in the high scaling world as both of those companies already have “cloud solutions” in place. They had to do something to ensure they were not left behind.

 

So Microsoft started the project to create an infrastructure to allow customers to deploy applications into Microsoft’s datacenter that would seamlessly scale as their requirements grew – Azure is the result.

 

The idea then is that you write your software and tell Microsoft what facilities you require – this is in the form of a manifest config file: “I require 20 instances of the web front end and 5 instances of the data processing engine”. The software and config is then deployed in to Microsoft’s infrastructure and a component called the Fabric Controller maps the software on to virtual machines under the control of a hypervisor. They also put a load balancer in front of the web  front end. The web site runs as a “web role” that can accept requests from the Internet and the processing engine gets mapped to a “worker role” that cannot be accessed externally.

 

The Fabric Controller has responsibility to ensure that there are always a number of running instances your components even in the face of hardware failures. It will also ensure that you can perform rolling upgrades without taking your app offline if that what you require.

 

The problem that apps then face is that even though they may run on a specific machine at one point, the next time they are initialized they may be on a completely separate machine and so storing anything locally is pointless apart from as a localized cache. That begs the question: where do I store my state? Enter the Azure Storage Service. This is a massively scalable, fault tolerant, highly available storage system. There are three types of storage available

 

Blob: Unstructured data with sizes up to 50Gb

Table: Structured tabular data with up to 252 user defined properties (think columns)

Queue: queue based data for storing messages to be passed from one component to another in the azure infrastructure

 

Hopefully Blob and Queue are fairly familiar constructs to most people. Table probably needs a little clarification. We are not talking about a relational database here. There is no schema for the table based data so, in fact, every row could be shaped completely differently (although this would be pretty ugly to try to read back out again). There are no relations and therefore no joins. There are also no server managed indexes – you define a pseudo index with the idea of a partition ID – this ID can be used to horizontally partition the data across multiple machine clusters but the partition ID is something you are responsible for managing. However, each row must be uniquely identifiable so there is also a concept of a row id and the partition id/row id combination make up the primary key of the table. There is also a system maintained version number for concurrency control. So this is where the strange number of 252 user defined properties comes from 255 – 3 (partition id, row id, version)

 

So in the above example, the web front end passes the data processing engine the data by enqueing it into queue storage. The processing engine then stores the data further (say in a blob or table) or just processes and pushes the results back on another queue. It can also send messages out to external endpoints.

 

All components run under partial trust (a variant similar to ASP.NET medium trust) so Azure developers will need some understanding of CAS.

 

The API for talking to Azure is REST based which can be wrapped by ADO.NET Data Services if appropriate (e.g. for table storage)

 

To get started you need an account provisioned (to get space reserved in the datacenter). You can do this via http://www.azure.com. There are other services built on top of Azure, which I will cover in subsequent posts, which get provisioned in the same place.

 

There is an SDK and VS2008 SP1 integration. This brings a development fabric to your machine that has the same services available as the cloud based one so you can test without deploying into the cloud. There are also VS project templates which in fact create multiple projects: the application one(s) and another to specify the deployment configuration.

 

So where does that leave Microsoft. They have created an offering that in its totality (which is much more than I have talked about here) is beyond what both Amazon and Google have created in terms of functionality. But they are left with two issues:

1)      Will companies trust Microsoft to run their business critical applications? Some definitely will but others will reserve judgement for some time until they have seen in practice that this infrastructure is sound. Microsoft say they will also have an SLA in the contract that will have financial penalty clauses if they fail to fulfil it in some currently unspecified way

2)      Microsoft have not yet announced any pricing model. This leaves companies in a difficult position – do they throw resources at a project with a lot of potential and bear the risk that when Microsoft unveil the pricing their application is not economically viable? Or do they wait to start investing in this technology until Microsoft announce pricing. Unfortunately this is a chicken and egg situation – Microsoft cannot go live commercially  until the infrastructure has been proven in practice by companies creating serious app on it, and yet they do not want to announce pricing until they are ready for commercial release. Hopefully they will be prepared to discuss pricing for any organization that it serious about building on the infrastructure on a case by case basis before full commercial release.

 

Azure definitely has huge potential for those companies that need a very flexible approach to scale or who will require scale over time but that time cannot yet be determined. It also has some challenges for how you build applications - there are design constraints you have to cater for (failure is a fact of life and you have to code to accept that for example).

 

Definitely interesting times

.NET | Azure | REST
Saturday, 01 November 2008 02:39:23 (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Monday, 27 October 2008
I'm sitting here in the PDC 08 Keynote. Ray Ozzie has just announced Windows Azure - a new Windows platform "in the cloud". In other words a set of services hosted in Microsoft's datacenters that you can deploy your apps into. As a platform it has a whole systems management side to it.

The service model uses services, endpoints - contracts ... seems familiar. You deploy the code and a model describing the app so the systems management can support your app.

Storage system is highly available with storage for blobs, tables and queues. Supports dynamic load balancing and caching

Azure development is done in Visual Studio and supports both managed and unmanaged code. New "cloud" project templates give you a pair of projects - one is a standard familiar .NET project and the other is the one is configuation that describes the app.

The Azure portal lets you change the configuration dynamically to scale up as required. Currently you have to edit the XML but they will be providing a UI for the configuration.

This all looks pretty exciting - looking forward to getting hold of the bits tomorrow

.NET | Azure
Monday, 27 October 2008 16:24:25 (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   |