# Friday, January 23, 2009

Sometimes you have to wonder if this subject will ever go away …

A few weeks ago I posted on using OO constructs in your DataContracts. It’s one of those things that is understandable when .NET developers first start building message based systems. Another issue that raises its head over and over again goes along the lines of “I’m trying to send a DataSet back to my client and it just isn’t working properly”. This reminds me of the old joke:

Patient: Doctor, doctor it hurts when I do this (patient raises his arm into a strange position over his head)
Doctor: Well don’t do it then

So what is the issue with using DataSets as parameters or return types in your service operations?

Lets start off with interoperability – or the total lack thereof. If we are talking about interoperability we have to think about what goes into the WSDL for the DataSet – after all it is a .NET type. In fact DataSets, by default serialize as XML so surely it must be ok! Here’s what a DataSet looks like in terms of XML Schema in the WSDL

  <xs:element ref="xs:schema" /> 
  <xs:any />

In other words I’m going to send you some … XML – you work out what do with it. But hold on – if I use Add Service Reference, it *knows* its a DataSet so maybe it is ok. Well WCF cheats; just above that sequence is another piece of XML

       <ActualType Name="DataSet" Namespace="http://schemas.datacontract.org/2004/07/System.Data"
="http://schemas.microsoft.com/2003/10/Serialization/" /> 

So WCF cheats by putting an annotation only it understands into the schema so it knows to use a DataSet. If you really do want to pass back arbitrary XML as part of a message then use an XElement.

So how about if I have WCF on both ends of the wire? Well then you’ve picked a really inefficient way to transfer around the data. You have to remember how highly functional a DataSet actually is. Its not just the data in the tables that support that functionality, there is also : change tracking data to keep track of what rows have been added, updated and removed since the DataSet was filled; relationship data between tables; a schema  describing itself. DataSets are there to support disconnected processing of tabular data, not as a general purpose data transfer mechanism.

Then you may say – “hey we’re running on an intranet – the extra data is unimportant”. So the final issue you get with a DataSet is tight coupling of the service and the consumer. Changes to the structure of the data on one side of the wire cascade to the other side to someone who may not be expecting the changes. Admittedly not all changes will be breaking ones but are you sure you know which ones will be and which ones won’t. As long as the data you actually want to pass isn’t changing why are you inflicting this instability on the other party in the message exchange. The likelihood that you will have to make unnecessary changes to, say, the client when the service changes is increased with DataSets 

So what am I suggesting to do instead? Instead model the data that you do want to pass around using DataContract (or XElement if you truly want to be able to pass untyped XML). Does this mean you have to translate the data from a DataSet to this DataContract when you want to send it? Yes it does, but that code can be isolated in a single place. When you receive the data as a DataContract and want to process it as a DataSet, does this mean you have to recreate a DataSet programmatically? Yes it does, but again you can isolate this code in a single place.

So what does doing this actually buy you if you do that work? You get something which is potentially interoperable, that only passes the required data across the wire and that decouples the service and the consumer.

Friday, January 23, 2009 4:49:20 PM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Thursday, January 22, 2009

I’ve been writing a lab on Workflow Services in 4.0 recently. Part of what I was showing was the new data orientated correlation (the same kind of mechanism that BizTalk uses for correlation). So I wanted to have two operations that were correlated to the same workflow instance based on data in the message (rather than a smuggled context id as 3.5 does it). As I was writing this lab I suddenly started getting an InvalidOperationException stating DispatchOperation requires Invoker every time I brought the .xamlx file up in a browser. It appeared that others had seen this as well but not really solved it. So I dug around looking at the XAML (workflow services can be written fully declaratively now) and the config file and could see no issues there. I asked around but no one I asked knew the reason.

So I created a simple default Declarative Workflow Service project and that worked ok. I compared my lab workflow and the default one and it suddenly dawned on me what was wrong. The default project has just one operation on the contract and has a ServiceOperationActivity to implement it. My contract had my two operations but I had, so far, only bound one ServiceOperationActivity. So in other words I had not implemented the contract. This is obviously an issue and looking back I’m annoyed I didn’t see it sooner.

However, the problem is that this is a change in behavior between 3.5 and 4.0. In 3.5 if I didn’t bind a ReceiveActivity to every operation I got a validation warning but I could still retrieve metadata; in 4.0 you get a fatal error. Its not hugely surprising that the behavior has changed – after all the whole infrastructure has been rewritten.

On the whole its a good thing that implementation of the contract is enforced – although it would be nice if the validation infrastructure caught this at compile time rather than it being a runtime failure.

.NET | BizTalk | WCF | WF
Thursday, January 22, 2009 10:10:15 AM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Wednesday, January 21, 2009

A forum question was asking how to create a host that dynamically loaded up services deployed in DLLs. I told the poster to use reflection but he wasn't familiar with it so I thought I would quickly knock up a sample

The host looks in a folder under the directory in which it is executing. The service must be annotated with a [ServiceClass] attribute (defined in a library that accompanies the host). Then you set up the config file for the service ( [ServiceBehavior(ConfigurationName="<some name>")] helps decouple the config from the actual implementing type nicely

The sample is here

DynamicHost.zip (77.07 KB)

Wednesday, January 21, 2009 11:00:56 AM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Tuesday, January 20, 2009

Christian found a bug in BlobExplorer when the blob name has pseudo directory structure in it (common prefixes)

Fixed version is available here

Tuesday, January 20, 2009 10:10:25 AM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Friday, January 16, 2009

Hanging out on the WCF Forums, it appears one in five answers I give seem to be to do with sessions in WCF. As a result, rather than write the same thing over and over again I decided to distil my forum posts into a blog post that I can link to.

WCF has the concept of a session. A session is simply an identifier for the proxy making the call. Where does this ID come from? Well it depends on the binding: with NetTcpBinding, the session is inherent in the transport – TCP is a connected session based transport; with WSHttpBinding session is an artifice piggybacking either WS-SecureConversation or WS-ReliableMessaging. So different bindings go via different mechanisms to achieve the concept of session. However, not all bindings support session – BasicHttpBinding, for example, has no mechanism for maintaining a session. Be aware that every mechanism for maintaining a session in WCF has issues with load balancing. The relationship between proxy and service is a machine-bound one. If you want to use load balancing you will have to use sticky sessions.

So what does having a session do for me? Well some parts of the infrastructure rely on sessions – callbacks for example. WCF also supports the ability to associate a specific instance of the service implementation class with a session (called PerSession instancing). Having a specific instance for the session can seem very attractive in that it allows you to make multiple calls to a service from a specific proxy instance and the service can maintain state in member variables in that service instance in behalf of the client. In fact, if the binding supports session, this is the default model for mapping service instances to requests. WCF also supports two other instancing models (controlled by the service behavior InstanceContextmode): PerCall – each request gets its own instance; Single – all calls use a single instance.

Session has an impact beyond the service – client proxy behavior is affected by the idea of session. Say the client crashes, what happens to the session? Again this depends on the binding. NetTcpBinding is connection orientated so the service side gets torn down if the client crashes. WSHttpBinding on the othe hand rides on top of a connectionless protocol (HTTP) and so if the client crashes the service has no idea. In this situation the session will eventually time out depending on the configuration of WS-SecureConverstion or WS-ReliableMessaging – the default is 10 minutes. To tear down the session the proxy has to call back to the service to say that it is done. This means that closing the proxy in the client is crucial to efficient session management and that Close with WSHttpBinding will actually roundtrip to the service. If the service has died, Close, in this situation will throw an exception and you must call Abort to clean up the proxy correctly.

Session also has effects that are often hidden during development. WCF has an inbuilt throttle to control concurrent operations. There are three throttles: concurrent requests (self explanatory); concurrent objects (remember that there are options as to how many service instances service requests – in reality this one is very unlikely to affect you); and most importantly for the discussion here, concurrent sessions – this defaults to 10. So if you do not clean up your proxy by default your service will support up to 10 concurrent proxies which is generally way too low for many systems.

Sometimes your service will require the ability to maintain per client state. However, due to the complexity of dealing with sessions it is often best to avoid them (unfortunately impossible with the NetTcpBinding) and use another, applkication defined, session identifier and put the session state in a database. However, for example with callbacks, sometimes you require session in your service. In this case you should annotate your contract with the modified ServiceContract


If you do not need session and you do not need to use a session orientated protocol like named pipes or tcp then I would turn off session explicitly by using the following on the contract


In my experience it is rarely useful to use sessions and they introduce overhead, load balancing restrictions and complexity for the client.

Friday, January 16, 2009 11:18:54 PM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Saturday, January 10, 2009

I'm really excited to announce that I've joined thinktecture as a consultant. I've known Ingo, Christian and Dominick for some time now and its great to take that relationship on to a new level. I have huge respect for the abilities of the thinktecture team and so it was fantastic when they asked me if I was interested in working with them.

I'll be focusing on all things distributed at thinktecture - so that includes WCF, Workflow, BizTalk, Dublin and Azure. And I guess I'll have to learn to speak German now ...

Azure | BizTalk | Dublin | Life | WCF | WF
Saturday, January 10, 2009 6:27:22 PM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Thursday, January 08, 2009

I’ve recently started hanging out on the MSDN WCF forum. Its interesting seeing the questions that are coming up there. One fairly common one is along the lines of:

I have a service that I want to return a list of Person objects but it doesn’t seem to be working when I pass Employees and Managers in the list.

At some point someone will reply

You should annotate the contract with the ServiceKnownType attribute for each type that the Person could be.

Now in a strict sense this is true – however, I think there is a much deeper question that needs discussion.

To understand where the basic problem is we need to take a quick detour into the architecture of WCF. WCF has two layers: the channel layer and the service model layer.

The channel layer deals with Message objects passing them through a composable stack of channels which add or process message headers, handle mapping the Message object to and from a stream of bytes and moving those bytes from one place to another. Notice here we have no notion of ServiceContracts, etc. All messages are untyped in .NET terms are are expressed in terms of an XML Infoset.

The service model layer sits on top of the channel layer and deals with endpoints, behaviors, dispatchers and other bits of plumbing – but the phrase “sits on top of the channel layer” is hugely important. An endpoint is made up of three elements: an address (where), a binding (how) and a contract (what). It is the contract that often causes confusion. In WCF code it takes the form of an annotated interface (can be a class but an interface is more flexible)

interface ICalc
     int Add( int x, int y);

But what do the parameters and return types really mean? What you are in fact doing here is specifying the request and response message bodies. This information will be translated into XML by the time the Message hits the channel layer. Now here we have only used simple types. The data that we generally want to move around is more complex and so we need another way to describe it – for that we use a .NET type and a serializer that translates an object into XML and back again. There are two main serializers that can be used: the long standing XmlSerializer and the DataContractSerializer that was introduced with WCF; the DataContractSerializer is the default one. You normally take a .NET class and annotate is with attributes (although .NET 3.5 SP1 removed the need to do this its generally a good idea anyway as you retain control over XML namespaces, etc)

class Person
     public string Name{ get; set; }
     public int Age{ get; set; }

interface IMakeFriends
     void AddFriend( Person p );

So far so good. So where is the problem? Well now we’re in .NET type system world so it starts to become tempting to pass types that derive from person to the AddFriend method – after all the C# compiler doesn’t doesn’t complain. And at this point when you call the AddFriend method it doesn’t work and so you post the above question to the WCF forum. Unfortunately the way that contracts are expressed in WCF makes is very easy to forget what their purpose is: to define the messages send to the operation and being sent back from the operation. In reality you have to think “how would I express this data in XML?”. XML doesn’t support inheritance so whatever you put in the contract is going to have to have some way of mapping to XML. The data contracts used to define the messages are simply a .NET typed convenience for generating the XML for the data you want to pass – if you view them any other way you are destined for a world of pain. So think about the data you want to pass, not how it may happen to be represented in your business layer and design your DataContracts accordingly.

“But wait!” you may say “I may want to pass and Employee or a Manager here and I need the data to get to the receiver”. Again it comes back to “how would I express that in XML?” as that is what will be happening at the end of the day. If you have a discrete set of types that could be passed then you have something similar to a XML Schema Choice Group. In this case ServiceKnownType is a reasonable approach (although it doesn’t use a Choice Group under the covers). Remember you will need to change the contract if new types need to be passed – as with Choice Groups. However, if you require looser semantics where an extensible number of things may be passed without wanting to change the contract then you require a different way of expressing this in XML and therefore a different way of expressing this in .NET code. Something like:

public class KeyValuePair
public string Key { get; set; }
public string Value { get; set; }

public class FlexibleData
public string Discriminator { get; set; }
public List<KeyValuePair> Properties { get; set; }

In this case the receiver will be responsible for unpicking the Properties based on the Discriminator. The other thing you could do is to pass the data as an XElement and work explicitly at the XML level, with LINQ to XML this is not very onerous. Is this all really convenient to use in .NET? Well not as easy as inheritance, but then we’re actually talking about XML here not .NET. Again, the .NET typed contract is simply a convenience for creating the XML message.

Thursday, January 08, 2009 4:40:18 PM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   |