# Friday, February 26, 2010

Thanks to everyone who attended my two sessions at BASTA! – another thoroughly enjoyable conference. I’ve uploaded the demos

What’s new in Workflow 4.0 – this includes the application with the rehosted designer

Building WCF services with Workflow 4.0

.NET | WCF | WF | WF4
Friday, February 26, 2010 10:02:42 AM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Sunday, February 14, 2010

I my last post I showed that creating a custom composite activity (one that can have one or more children) requires deriving from NativeActivity. The Retry activity that I showed was fairly simple and in particular didn’t try to share data with its child. There appears to be a catch-22 in this situation when it comes to overriding CacheMetadata: if I add a Variable to the metadata (AddVariable) then it can be used exclusively by its children – i.e. the activity itself can’t manipulate the state; if I add a variable as implementation data to the metadata (AddImplementationVariable) then the children cant see it as its seen as purely used for this activities implementation. How then do we create data that can be both manipulated by the activity and accessed by the parent?

The secret to achieving this is a feature called ActivityAction - Matt Winkler talks about it here. The idea is that I can bind an activity to one or more parent defined pieces of data then schedule the activity where it will have access to the data. Its probably best to show this with an example so I have written a ForEachFile activity that you give a directory and then it passes its child the file name of each file in the directory in turn. I’ll show the code in its entirety and then walk through each piece in turn

   1: [ContentProperty("Body")]
   2: [Designer(typeof(ForEachFileDesigner))]
   3: public class ForEachFile : NativeActivity, IActivityTemplateFactory
   4: {
   5:     public InArgument<string> Directory { get; set; }
   7:     [Browsable(false)]
   8:     public ActivityAction<string> Body { get; set; }
  10:     private Variable<IEnumerator<FileInfo>> files = new Variable<IEnumerator<FileInfo>>("files");
  12:     protected override void CacheMetadata(NativeActivityMetadata metadata)
  13:     {
  14:         metadata.AddDelegate(Body);
  16:         RuntimeArgument arg = new RuntimeArgument("Directory", typeof(string), ArgumentDirection.In);
  17:         metadata.AddArgument(arg);
  19:         metadata.AddImplementationVariable(files);
  20:     }
  21:     protected override void Execute(NativeActivityContext context)
  22:     {
  23:         DirectoryInfo dir = new DirectoryInfo(Directory.Get(context));
  25:         IEnumerable<FileInfo> fileEnum = dir.GetFiles();
  26:         IEnumerator<FileInfo> fileList = fileEnum.GetEnumerator();
  27:         files.Set(context, fileList);
  29:         bool more = fileList.MoveNext();
  31:         if (more)
  32:         {
  33:             context.ScheduleAction<string>(Body, fileList.Current.FullName, OnBodyComplete);
  34:         }
  35:     }
  37:     private void OnBodyComplete( NativeActivityContext context, ActivityInstance completedInstance)
  38:     {
  39:         IEnumerator<FileInfo> fileList = files.Get(context);
  40:         bool more = fileList.MoveNext();
  42:         if (more)
  43:         {
  44:             context.ScheduleAction<string>(Body, fileList.Current.FullName, OnBodyComplete);
  45:         }
  47:     }
  49:     #region IActivityTemplateFactory Members
  50:     public Activity Create(DependencyObject target)
  51:     {
  52:         var fef = new ForEachFile();
  53:         var aa = new ActivityAction<string>();
  54:         var da = new DelegateInArgument<string>();
  55:         da.Name = "item";
  57:         fef.Body = aa;
  58:         aa.Argument = da;
  60:         return fef;
  61:     }
  62:     #endregion
  63: }

Ok lets start with the core functionality then we’ll look at each of the pieces that help make this fully usable. As you can see the class derives from NativeActivity and overrides CacheMetadata and Execute – we’ll look at their implementations in a minute. There are three member variables in the class: the InArgument<string> for the directory; an implementation variable to hold the iterator as we move through the files in the directory; the all important ActivityAction<string> which we will use to pass the current file name to the child activity.

Lets look at CacheMetadata more closely. Because we want to do interesting things with some of the state the default implementation won’t work so we override it. As we don’t call the base class version we need to specify how we use all of the state. We add the ActivityAction as a delegate, we bind the Directory argument to a RuntimeArgument and specify the iterator as an implementation variable – we don’t want the child activity to have access to that directly.

Next lets look at Execute. The first couple of lines are nothing unusual – we get the directory and get hold of the list of files. Now we have to store the retrieved iterator in the implementation variable, fileList. We move to the first file in the iteration, if it returns false the list was empty so as long as MoveNext returned true we want to schedule the child activity passing the current file. To do this we use the ScheduleAction<T> member on the context. However, because we’re going to process each one sequentially, we also need to know when the child is finished so we pass a completion callback, OnBodyComplete.

Now in OnBodyComplete we simply get the iterator, move to the next item in the iteration and as long as we’re not finished iterating re-schedule the child again passing the new item in the iteration.

That really is the “functional” part of the activity – everything else is there to support the designer. So why does the designer need help? Well lets look at what we would have to do to build this activity correctly if we were going to execute it from main

   1: Activity workflow = new ForEachFile
   2: {
   3:   Body = new ActivityAction<string>
   4:   {
   5:     Argument = new DelegateInArgument<string>
   6:     {
   7:       Name = "item",
   8:     },
   9:     Handler = new WriteLine
  10:     {
  11:       Text = new VisualBasicValue<string>("item")
  12:     }
  13:   },
  14:   Directory = @"c:\windows"
  15: };

As you can see, its not a matter of simply creating the ForEachFile, we have to build the internal structure too – something needs to create that structure for the designer - this is the point of IActivityTemplateFactory. When you drag an activity on to the design surface normally it just creates the XAML for that activity. However, before it does that it does a speculative cast for IActivityTemplateFactory and if the activity supports that it calls the interface’s Create method instead and serializes the resulting activity to XAML.

So going back to the ForEachFile activity, you can see it implements IActivityTemplateFactory and therefore, as far as the designer is concerned, this is the interesting functionality – lets take a look at the Create method.We create the ForEachFile and wire an ActivityAction<string> to its Body property. ActivityAction<string> needs a slot to store the data to be presented to the child activity. This is modelled by DelegateArgument<string> and this gets wired to the Argument member of the ActvityAction. We also name the argument as we want a default name for the state so the child activity can use it. Notice, however, we don’t specify the child activity itself (it would be a pretty useless composite if we hard coded this). The child will be placed on the Handler property of the ActivityAction but that will be done in the ActivityDesigner using data binding.

Before we look at the designer lets highlight a couple of “polishing” features of the code: the class declared a ContentProperty via an attribute – that just makes the XAML parsing cleaner as the child doesn’t need to be specified using property element syntax; the Body is set to non-browsable – we don’t want the activities user to try to set this value in the property grid.

OK on to the designer. If you read my previous article there are a couple of new things here. Lets look at the markup – again there is no special code in the code behind file, everything is achieved using data binding

   1: <sap:ActivityDesigner x:Class="ActivityLib.ForEachFileDesigner"
   2:     xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
   3:     xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
   4:     xmlns:s="clr-namespace:System;assembly=mscorlib"
   5:     xmlns:conv="clr-namespace:System.Activities.Presentation.Converters;assembly=System.Activities.Presentation"
   6:     xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation"
   7:     xmlns:sapv="clr-namespace:System.Activities.Presentation.View;assembly=System.Activities.Presentation"
   8:     xmlns:me="clr-namespace:ActivityLib">
   9:     <sap:ActivityDesigner.Resources>
  10:         <conv:ArgumentToExpressionConverter x:Key="expressionConverter"/>
  11:     </sap:ActivityDesigner.Resources>
  13:     <Grid>
  14:         <Grid.RowDefinitions>
  15:             <RowDefinition Height="Auto"/>
  16:             <RowDefinition Height="*"/>
  17:         </Grid.RowDefinitions>
  19:         <StackPanel Orientation="Horizontal" Margin="2">
  20:             <TextBlock Text="For each file "/>
  21:             <TextBox Text="{Binding ModelItem.Body.Argument.Name}" MinWidth="50"/>
  22:             <TextBlock Text=" in "/>
  23:             <sapv:ExpressionTextBox Expression="{Binding Path=ModelItem.Directory, Converter={StaticResource expressionConverter}}"
  24:                                     ExpressionType="s:String"
  25:                                     OwnerActivity="{Binding ModelItem}"/>
  26:         </StackPanel>
  27:         <sap:WorkflowItemPresenter Item="{Binding ModelItem.Body.Handler}"
  28:                                    HintText="Drop activity"
  29:                                    Grid.Row="1" Margin="6"/>
  31:     </Grid>
  33: </sap:ActivityDesigner>

Here’s what this looks like in the designer


So the TextBox at line 21 displays the argument name that our IActivityTemplateFactory implementation set up. The ExpressionTextBox is bound to the directory name but allows VB.NET expressions to be used. The WorkflowItemPresenter is bound to the Handler property of the Body ActivityAction, This designer is then associated with the activity using the [Designer] attribute.

So as you can see, ActivityAction and IActivityTemplateFactory work together to allow us to build rich composite activities with design time support

.NET | WF | WF4
Sunday, February 14, 2010 1:54:30 PM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Tuesday, February 09, 2010

I’m writing Essential Windows Workflow Foundation 4.0 with Maurice for DevelopMentor. One of the things that I think is less than obvious is the behavior of NativeActivity.

What is NativeActivity I hear you ask? Well there are a number of models for building custom activities in WF4. Most “business” type custom activities will be built using a declarative model in XAML by assembling building blocks graphically. However, what if you are missing a building block? At this point you have to fall back to writing code and there are three options for your base class when writing an activity in code:

You use CodeActivity when you have a simple synchronous activity. All work happens in Execute and it has no child activities

This is new to WF4. Here you have the ability to implement the async pattern (BeginExecute / EndExecute) to perform short lived async operations where you do not want the workflow persisted (e.g. an async WebRequest)

This gives you full access to the power of the workflow execution engine. However, in the words of Spiderman’s Uncle, “with great power comes great responsibility”. NativeActivity can be a bit tricky so that is what this article is about

I’m going to walk through the code for a Retry activity – where a child activity can be rerun a number of times upon failure. The activity has two InArguments: MaxRetries and RetryDelay. MaxRetries says how many times you will retry the child before giving up. RetryDelay says how long to wait between retries. We could use a Thread.Sleep to do the delay but this would not be good for the workflow engine: we block a thread it could use and there is no way for the engine to persist the workflow – what if we wanted to retry in 2 days? So instead, as part of our implementation, we’ll use a Delay activity.

Now to explain the code we have to take a slight diversion and talk about the relationship between Activity, ActivityInstance and ExecutionContext. I talked about is a while back here when the PDC CTP first came out (that’s what the reference to some base class called WorkflowElement is about) but to expand a little: The Activity is really just a template containing the code to execute for the activity. The ActivityInstance is the actual thing that is executing. It holds the state for this instance of the activity template. Now we need a way to bind the template code to the currently running instance of the activity and this is the role of the ExecutionContext. If you are using a CodeActivity base class then most of this is hidden from you except that you have to access arguments by passing in the ExecutionContext. However, with NativeActivity you have to get more directly involved with this model.

Now how does the workflow engine know what data you need to store in the ActivityInstance? Well it turns out you need to tell it. NativeActivity has a virtual method called CacheMetadata (this post talks about it to some degree). The point being that the activity has to register all of the “stuff” that it wants to use during its execution. Now the base class implementation will do some fairly reasonable default actions but it cannot know, for example, that part of your functionality is there purely for implementation details and should not be public. Therefore, you will often override this when you create a NativeActivity

So without more ado – here’s the code for the Retry activity. I’ll then walk through it

   1: [Designer(typeof(ForEachFileDesigner))]
   2: public class Retry : NativeActivity
   3: {
   4:   public InArgument<int> MaxRetries { get; set; }
   5:   public InArgument<TimeSpan> RetryDelay { get; set; }
   6:   private Delay Delay = new Delay();
   8:   public Activity Body { get; set; }
  10:   Variable<int> CurrentRetry = new Variable<int>("CurrentRetry");
  12:   protected override void CacheMetadata(NativeActivityMetadata metadata)
  13:   {
  14:     metadata.AddChild(Body);
  15:     metadata.AddImplementationVariable(CurrentRetry);
  17:     RuntimeArgument arg = new RuntimeArgument("MaxRetries", typeof(int), ArgumentDirection.In);
  18:     metadata.Bind(MaxRetries, arg);
  19:     metadata.AddArgument(arg);
  21:     Delay.Duration = RetryDelay;
  23:     metadata.AddImplementationChild(Delay);
  24:   }
  26:   protected override void Execute(NativeActivityContext context)
  27:   {
  28:     CurrentRetry.Set(context, 0);
  30:     context.ScheduleActivity(Body, OnFaulted);
  31:   }
  33:   private void OnFaulted(NativeActivityFaultContext faultContext, Exception propagatedException, ActivityInstance propagatedFrom)
  34:   {
  35:     int current = CurrentRetry.Get(faultContext);
  36:     int max = MaxRetries.Get(faultContext);
  38:     if (current < max)
  39:     {
  40:       faultContext.CancelChild(propagatedFrom);
  41:       faultContext.HandleFault();
  42:       faultContext.ScheduleActivity(Delay, OnDelayComplete);
  43:       CurrentRetry.Set(faultContext, current + 1);
  44:     }
  45:   }
  47:   private void OnDelayComplete(NativeActivityContext context, ActivityInstance completedInstance)
  48:   {
  49:     context.ScheduleActivity(Body, OnFaulted);
  50:   }
  51: }

So lets look at the code: first the class derives from NativeActivity (we’ll come to the designer later). This means I want to do fancy things like have child activities or perform long running asynchronous work. Next we see the two InArguments that are passed to the Retry. The Delay is an implementation detail of how we will pause between retry attempts and the Body property is where the activity we are going to retry lives. Finally we have a Variable, CurrentRetry, where we store how many retry attempts we have made. That is the data in the class but remember we need to tell the workflow engine about what we need to store and why – this is the point of CacheMetadata (I read this method name as “here is the metadata for the cache” rather than “I am going to cache some metadata”)

In CacheMetadata the first thing we do is specify that the Body is our child activity. Next we tell the engine that we want to be able to get hold of the CurrentRetry but that its only there for our implementation – we’re not expecting the child activity to try to make use of it. The next 3 lines (17-19) seem a little strange but essentially we’re saying that the MaxRetries argument needs to be accessed over the whole lifetime of the ActivityInstance so we need a slot for that. We next configure out Delay activity passing the RetryDelay as its duration (this is how long we want to wait between retries). Finally we add the Delay, not as a normal child, but as an implementation detail.

OK, Execute is pretty simple – we initialize the CurrentRetry and then schedule the Body. But, because we want to retry on failure, we also pass in a fault handler (OnFaulted)

OnFaulted does the main work. It fires if the child fails. So it checks to see if we have exceeded the retry count and if not retries the Body activity. However, it doesn’t do this directly, first it tells the context that it has handled the error – it also, strangely has to cancel the current child (which is odd as its already faulted) – Maurice talks about this oddity here. Next it schedules the Delay as we have to wait for the retry delay and wires up a completion handler (OnDelayComplete) so we know when the delay is finished. Finally it updates the CurrentRetry.

OnDelayComplete simply reschedules the Body activity, remembering to pass the fault handler again in case the activity fails again.

Oh one thing I said I’d come back to – the designer. I have written a designer to go with this (designers in WF4 are WPF based). The XAML for this is here:

   1: <sap:ActivityDesigner x:Class="WordActivities.ForEachFileDesigner"
   2:     xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
   3:     xmlns:s="clr-namespace:System;assembly=mscorlib"
   4:     xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
   5:     xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation"
   6:     xmlns:sapv="clr-namespace:System.Activities.Presentation.View;assembly=System.Activities.Presentation"
   7:     xmlns:conv="clr-namespace:System.Activities.Presentation.Converters;assembly=System.Activities.Presentation" 
   8:     mc:Ignorable="d" 
   9:     xmlns:d="http://schemas.microsoft.com/expression/blend/2008" 
  10:     xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" >
  11:     <sap:ActivityDesigner.Resources>
  12:         <conv:ArgumentToExpressionConverter x:Key="expressionConverter"/>
  13:     </sap:ActivityDesigner.Resources>    
  14:   <Grid>
  15:         <Grid.RowDefinitions>
  16:             <RowDefinition Height="Auto"/>
  17:             <RowDefinition Height="Auto"/>
  18:             <RowDefinition Height="*"/>
  19:         </Grid.RowDefinitions>
  20:         <Grid.ColumnDefinitions>
  21:             <ColumnDefinition Width="Auto"/>
  22:             <ColumnDefinition Width="*"/>
  23:         </Grid.ColumnDefinitions>
  25:             <TextBlock Grid.Row="0" Grid.Column="0" Margin="2">Max. Retries:</TextBlock>
  26:             <sapv:ExpressionTextBox Expression="{Binding Path=ModelItem.MaxRetries, Converter={StaticResource expressionConverter}}"
  27:                                     ExpressionType="s:Int32"
  28:                                     OwnerActivity="{Binding ModelItem}"
  29:                                     Grid.Row="0" Grid.Column="1" Margin="2"/>
  31:             <TextBlock Grid.Row="1" Grid.Column="0" Margin="2">Retry Delay:</TextBlock>
  32:         <sapv:ExpressionTextBox Expression="{Binding Path=ModelItem.RetryDelay, Converter={StaticResource expressionConverter}}"
  33:                                     ExpressionType="s:TimeSpan"
  34:                                     OwnerActivity="{Binding ModelItem}"
  35:                                     Grid.Row="1" Grid.Column="1" Margin="2"/>
  37:             <sap:WorkflowItemPresenter Item="{Binding ModelItem.Body}"
  38:                                        HintText="Drop Activity"
  39:                                        Margin="6"
  40:                                        Grid.Row="2"
  41:                                        Grid.Column="0"
  42:                                        Grid.ColumnSpan="2"/>
  44:     </Grid>
  45: </sap:ActivityDesigner>

There is no special code behind, everything is done via databinding.

So as you can see, there are a lot of pieces that need to be put into place for something that seems fairly simple. The critical issue is getting the implementation of CacheMetadata correct – once you have identified all of the pieces of data you need stored the rest falls out nicely.

.NET | WF | WF4
Tuesday, February 09, 2010 8:14:03 PM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Friday, February 05, 2010

A while back I wrote a blog post about DataSets and why you shouldn’t use them on service boundaries. The fundamental issues are:

  1. No non-.NET client has any idea what the data looks like you are sending them
  2. Hidden, non essential data is being passes up and down the wire
  3. Your client gets coupled to the shape of your data (usually a product of the Data Access Layer)

So when I saw that Entity Framework 4.0 supports Self Tracking Entities (STEs) I was interested to see how they would work – after all, automated change tracking is one of the reasons people wanted to use DataSets in service contracts. The idea is that as you manipulate the state the object itself tracks the changes to properties and whether it has been created new or marked for deletion. It does this by implementing an interface called IObjectWithchangeTracker which is then used by the ObjectContext to work out what needs to be done in terms of persistence. There is an extension method on the ObjectContext called ApplyChanges which does the heavy lifting.

The Entity Framework team has released a T4 Template to generate these STEs from an EDMX file and the nice thing is that the generated entities themselves have no dependency on the Entity Framework. Only the generated context class has this dependency and, so the story goes, the client needs to know nothing about Entity Framework, only the service does. The client, and the entities, remain ignorant of the persistence model. For this reason STEs have been touted as a powerful tool in n-tier based architectures

All of this seems almost too good to be true … and unfortunately it is.

To understand what the issue is with STEs we have to remember what two of the main goals of Service based systems are:

  1. Support heterogeneous systems – the service ecosystem is not bound to one technology
  2. Decoupling – to ensure changes in a service do not cascade to consumers of the service for technical reasons (there may obviously be business reasons why we might want a change in a service to effect the consumers such as changes in law or legislation)

So to understand the problem with STEs we need to look at the generated code.Here’s the model I’m using:


Now lets look at the T4 Template generated code for, say, the OrderLine

[DataContract(IsReference = true)]
public partial class OrderLine: IObjectWithChangeTracker, INotifyPropertyChanged
    #region Primitive Properties
    public int id
        get { return _id; }
            if (_id != value)
                ChangeTracker.RecordOriginalValue("id", _id);
                _id = value;
    private int _id;
    // more details elided for clarity

A few things to notice: firstly this is a DataContract and therefore is designed to be used on WCF contracts – that is its intent; secondly a bit of work takes place inside the generated property setters. The property setters check to see if the data is actually changed, then it records the old value and raises calls OnPropertyChanged to raise the PropertyChanged event (defined on INotifyPropertyChanged). Lets have a look inside OnPropertyChanged:

protected virtual void OnPropertyChanged(String propertyName)
     if (ChangeTracker.State != ObjectState.Added && ChangeTracker.State != ObjectState.Deleted)
         ChangeTracker.State = ObjectState.Modified;
     if (_propertyChanged != null)
         _propertyChanged(this, new PropertyChangedEventArgs(propertyName));

Ok so maybe a bit more than raising the event. It also marks the object as modified in the ChangeTracker. The ChangeTracker state is partly how the STE serializes its self tracked changes. It is this data that is used by the ApplyChanges extension method to work out what has changed. So the thing to remember here is that the change tracking is performed by code generated into the property setters.

Well the T4 Template has done its work so we create our service contract using these conveniently generated types and the client uses metadata and Add Service Reference to build its proxy code. It gets the OrderLine from the service, updates the quantity and sends it back. The service calls ApplyChanges on the context and then saves the changes and … nothing changes in the Database. What on earth went wrong?

At this point we have to step back and think about what those types we use in service contracts are actually doing. Those types are nothing more than serialization helpers – to help us bridge the object world to the XML one. The metadata generation uses the type definition and the attribute annotations to generate a schema (XSD) definition of the data in the class. Notice we’re only talking about data – there is no concept of behavior. And this is the problem When the Add Service Reference code generation takes place its based on the schema in the service metadata – so the objects *look* right, just the all important code in the property setters is missing. So you can change the state of entities in the client and the service will never be able to work out if the state has changed – so changes don’t get propagated to the database.

There is a workaround for this problem. You take the generated STEs and put them in a separate assembly which you give to the client. Now the client has all of the change tracking code available to it, changes get tracked and the service can work out what has changed and persist it.

But what have we just done? We have forced the client to be .NET. Not only that, we’ve probably compiled against .NET 4.0 and so we are requiring the client to be .NET 4.0 aware – we might as well have taken out a dependency on Entity Framework 4.0 in the client at this point. In addition, changes I make to the EDMX file are going to get reflected in the T4 generated code – I have coupled my client to my data access layer. Lets go back to the problems with using DataSets on service boundaries again. We’re back where we pretty much started – although the data being transmitted is more controlled.

So STEs at first glance look very attractive, but in service terms they are in fact similar to using DataSets in terms of the effect on the service consumer. So what is the solution? We’ll we’re back with our old friends DTOs and AutoMapper. To produce real decoupling and allow heterogeneous environments we have to explicitly model the *data* being passed at service boundaries. How this is patched this into our data access layer is up to the service. Entity Framework 4.0 certainly improves matters here over 1.0 as we can use POCOs which aid testability and flexibility of our service

.NET | EF4 | WCF
Friday, February 05, 2010 12:48:08 PM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   | 
# Thursday, February 04, 2010

I’ve just noticed that Andy has blogged about our drop in clinic we are running at DevWeek 2010. All of the Rock Solid Knowledge guys will be at the conference so come over and let us help solve your design and coding problems – whether it be WCF, Workflow, Multithreading, WPF, Silverlight, ASP.NET MVC,  Design Patterns or whatever you are currently wrestling with. We’re also doing a bunch of talks (detailed on our Conferences page)


.NET | RSK | WCF | WF | WF4
Thursday, February 04, 2010 1:19:44 PM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |   |