In my last post I talked about the base class I created for creating custom composites that you want control over the execution logic. Firstly Josh pointed out that I ought to state its limitations - it only executes (and therefore cancels) one child activity at a time so it would not be useful for building, for example, the ParallelActivity.

With that limitation in mind though, there are a number of useful applications. I had a think about a different execution model and came up with prioritized execution. So I built this on top of my base class which was an interesting exercise as I could use an attached property for the Priority which meant I didn't limit the children to a special child activity type. I also had a think about the design time and realised that the sequential and parallel models for composite design time wasn't appropriate so I stole^H^H^H^H^H leveraged the designer that the Conditional Activity Group uses which seemed to work well.

I packaged up the code and told Josh about it

"Oh yeah, thats an interesting one - Dharma Shukla and Bob Schmidt used that in their book"

...

It turns out that pretty much half the world has decided to build a prioritised execution composite.

Heres the code anyway

PrioritizedActivitySample.zip (76.36 KB)

But then I got to thinking - actually there is a problem with this approach - elegant as it may be. If you cannot change the priorities at runtime then all you are building is an elaborate sequence. So it turns out that attached properties cannot be data bound in WF - you have to use a full DependencyProperty created with DependencyProperty.Register rather than DependencyProperty.RegisterAttached. So I reworked the example, creating a special child activity DynamicPrioritizedSequenceActivity that has a priority that is a bindable dependency property. I also had to create a validator to make sure only this type of child could be added to the composite.

This neat thing now is you have a prioritized execution composite whose children can dynamically change their priority using data binding. The code for this is here

DynamicPrioritySample.zip (78.83 KB)

So between these two samples we have prioritized execution, designer support, attached property support and validator support

As usual all comments welcome

There are two types of custom composites you may end up writing:

1. a form of UserControl with a set of activities whose composite functionality you want to reuse. This is what you get when you add a new activity to a workflow project using the menu item. You end up with a class that derives from SequenceActivity and is the one occasion with a custom activity where you don;t really have to override Execute
2. a parent that wants to take control over the execution model of its children (think of the execution model of the IfElseActivity, WhileActivity and the ParallelActivity)

To me the second of these is more interesting (although no doubt less common) and involves writing a bit of plumbing. In general, though, there is a common pattern to the code and so I have created a base class that wraps this plumbing.

CustomCompositeBase.zip (55.6 KB)

There are two fundemental overridable methods:

LastActivityReached: returns whether there are more child activities to execute
GetNextActivity: returns the next activity to be scheduled

Inside the zip there is also a test project that implements a random execution composite (highly useful I know).

Feedback, as always, very welcome.

Yesterday I posted a base class for Async Activities that buried all the plumbing. However, looking at the zip I only had the binary for the base class and the test harness. Here is the zipped code.

AsyncActivityBaseClass12.zip (69.71 KB)

In addition Rod gave me some feedback (for example making the base class abstract messed up the design time view for the derived activity). In the light of that and other ideas I have made some changes:

• The base class is now concrete and so designer view now works for derived classes
• If you don't supply a queue name then the queue name defaults to the QualifiedName of the activity
• Cancellation support is now embedded in the base class.
• There is a new override called DoAsyncTeardown to allow you to deallocate any resources you allocated in DoAsyncSetup

I hope someone finds this useful

A while back Joe Duffy announced that they were changing the default max number of worker threads in the CLR threadpool in CLR 2.0 SP1. The change was to increase the limit 10 fold from 25 threads per processor per process to 250 per processor per process.

The basis of the change is that it is possible to deadlock the threadpool if you spawn and then wait for threadpool work from a threadpool thread (e.g. in processing an ASP.NET request). Increasing the size makes this occurance less likely (although doesn't remove the risk).

Although its not obvious, CLR 2.0 SP1 ships with .NET 3.5. So I was teaching Guerrilla .NET a couple of months ago and thought I'd demo the change by running  a project built under VS2005 then rebuild it under VS2008. What I hadn't expected (although on reflection its not that surprising) is that when I ran under 2005 on a machine with 2008 *installed* then 2005 also has the new behavior. In other words it patches the 2.0 CLR rather than being something that only affects new applications.

Why should we care? Well there are two main reasons we use threadpools. Firstly to remove the repeated cost of creating and destroying threads by repeatedly using a set of already created threads; secondly, it makes sure that we don't drown the machine in spawned threads when the application comes under load. Extra threads consume resources: the scheduler has to give them time and they default to 1Mb stack space.

And here is where I think the problem is. If you create a server side application, on 32 bit Windows, deploy it on an 8-way machine and put in under load, before this change you will cap out at 200 threads by default which is a reasonable (maybe a little high) number for IO bound processing. However, after the change this will cap out by default at 2000 threads. 2000 threads with 1Mb stack space means 2Gb of memory consumed by the threads. On 32 bit windows this is the total amount of addressable memory address space ... ouch! Say hello to Mr OutOfMemoryException.

So for existing applications this is a potentially breaking change. Your applications will have to take control of the max number of worker threads to ensure correct processing. ASP.NET already does this and you can adjust it via the <processModel> configuration element. In a windows service based application you have to make a call to ThreadPool.SetMaxThreads to control the threadpool.

To his credit Joe doen't pretend this change was the ideal resolution to a real-world problem but it is a change that will affect existing applcations as it is a patch to CLR 2.0 rather than a feature of a new version of the CLR

One of the most powerful aspects of Windows Workflow is that it can manage a workflow instance's state when it has no work to do. Obviously to do this it needs to know that your activity is waiting for something to happen. A number of activities in the Standard Activity Library run asynchronously like this -e.g. the DelayActivity and the new ReceiveActivity. However, most real WF projects require you to write your own custom activities. To perform long running operations or wait for external events to occur, these too should be able to run asynchronously.

To be usable in all situations, an async activity needs to do more than just derive from Activity and return ActivityExecutionStatus.Executing from the Execute override. It also needs to implement IActivityEventListener<QueueEventArgs> and IEventActivity (the latter allows it to be recognised as an event driven activity). We end up with 3 interesting methods

Execute
IEventActivity.Subscribe
IEventActivity.Unsubscribe

Depending where your activity is used these get called in a different order

Sequential Workflow: Execute
State Machine: Subscribe, Execute, Unsubscribe
ListenActivity: Subscribe, Unsubscribe, Execute

So getting the behavior correct in where you create the workflow queue you are going to listen on, set up the event handler on the queue, process the results from the queue and close the activity can be a little tricky. So I have created a base class for async activities that buries the complexity of the plumbing and leaves you with three responsiblities:

1. Tell the base class the name of the queue you wish to wait on
2. Optionally override a method to set up any infrastructure you need before you start waiting on the queue
3. Override a method to process the data that was passed on to the queue and decide whether you are still waiting for more data or you are finished

You can download the code here AsyncActivityBaseClass.zip (38.36 KB). Any feedback gratefully received

Finally, thanks to Josh and Christian for sanity checking the code

Really as a note to myself as I always forget how to do this. To turn off JIT debugging which takes an age to try to attach the debugger under the covers, you set this value in the registry

HKLM\SOFTWARE\Microsoft\.NETFramework\DbgJITDebugLaunchSetting=1

At least I now know where to go to find that information

The release of BizTalk 2006 has brought a lot of love to the messaging engine in terms of handling failures. Probably the biggest change was messaging errors like “no subscription” are now resumable. However, there are other changes such as being able to subscribe to failed messages using the ErrorReport context properties and enabling routing for failed messages on a receive or send port.

Another change was added in the Xml and Flat File disassemblers – that of recoverable interchange processing. When a disassemble is breaking apart a composite message either in the form of a flat file or an envelope schema what happens if something is wrong about one of the child messages? In BizTalk 2004 the entire batch would fail. In BizTalk 2006 this is still the default behaviour; however, you can now set RecoverableInterchange to true on the disassembler which will cause all successful messages to be processed and only the incorrect ones to fail.

This seems like a great idea but in this article I want to look at some of the subtleties in the XmlDisassembler that make this technology less useful than may first appear.

First, here is the setup. I have two schemas: an envelope that is simply a container for repeated records; a person schema with child elements firstName and lastName as part of a sequence group.

This is the person schema

<xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003"    

           xmlns="http://dotnetconslt.co.uk/schema/person"

           targetNamespace=http://dotnetconslt.co.uk/schema/person

           xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xs:element name="person">

    <xs:complexType>

      <xs:sequence>

        <xs:element name="firstName" type="xs:string" />

        <xs:element name="lastName" type="xs:string" />

      </xs:sequence>

    </xs:complexType>

  </xs:element>

</xs:schema>

And here is the envelope

<xs:schema xmlns:ns0="http://dotnetconslt.co.uk/schema/person"

           xmlns:b="http://schemas.microsoft.com/BizTalk/2003"

           xmlns="http://dotnetconsult.co.uk/schema/envelope"

     targetNamespace="http://dotnetconsult.co.uk/schema/envelope"

           xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xs:import schemaLocation=".\person.xsd"

            namespace="http://dotnetconslt.co.uk/schema/person" />

  <xs:annotation>

    <xs:appinfo>

      <b:schemaInfo is_envelope="yes"                 

         xmlns:b=http://schemas.microsoft.com/BizTalk/2003 />

      <b:references>

        <b:reference

           targetNamespace="http://dotnetconslt.co.uk/schema/person" />

      </b:references>

    </xs:appinfo>

  </xs:annotation>

  <xs:element name="envelope">

    <xs:annotation>

      <xs:appinfo>

        <b:recordInfo body_xpath="/*[local-name()='envelope' and

namespace-uri()='http://dotnetconsult.co.uk/schema/envelope']" />

      </xs:appinfo>

    </xs:annotation>

    <xs:complexType>

      <xs:sequence>

        <xs:element minOccurs="1"

                    maxOccurs="unbounded"

                    ref="ns0:person" />

      </xs:sequence>

    </xs:complexType>

  </xs:element>

</xs:schema>

Notice the emboldened pieces that the XmlDisassembler uses to break apart the envelope.

Now, I deploy these two schemas into Biztalk, and then create a Receive Port with a Receive Location using the XmlReceive pipeline. I make sure that error reporting is turned on.

Next I create two send ports: one with a filter set to the BTS.ReceivePortName of the receive port; one using a filter set for error reporting like this

So with the infrastructure in place we can try submitting a message. Let’s start with one that works just to ensure that the envelope is working correctly. Here is the good message:

<ns0:envelope xmlns:ns0="http://dotnetconsult.co.uk/schema/envelope">

  <ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">

    <firstName>Rich</firstName>

    <lastName>Blewett</lastName>

  </ns1:person>

  <ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">

    <firstName>Niels</firstName>

    <lastName>Bergland</lastName>

  </ns1:person>

  <ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">

    <firstName>Jason</firstName>

    <lastName>Whittington</lastName>

  </ns1:person>

</ns0:envelope>

And as expected, dropping this into the receive folder results in three messages in the non-error send folder.

Now let’s use this message

<ns0:envelope xmlns:ns0="http://dotnetconsult.co.uk/schema/envelope">

  <ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">

    <firstName>Rich</firstName>

    <lastName>Blewett</lastName>

  </ns1:person>

  <ns1:personbad xmlns:ns1="http://dotnetconslt.co.uk/schema/person">

    <firstName>Niels</firstName>

    <lastName>Bergland</lastName>

  </ns1:personbad>

  <ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">

    <firstName>Jason</firstName>

    <lastName>Whittington</lastName>

  </ns1:person>

</ns0:envelope>

The problem here is that we don’t have a schema deployed describing the personbad element. We drop this into the receive folder and this time we get the default behaviour that the whole batch gets abandoned and the single envelope message ends up in the error send folder.

OK let’s turn on recoverable interchange now. We could deploy a new pipeline with the XmlDisassembler configured differently. However in BizTalk 2006 there is a UI for the per-instance pipeline data so we can actually reconfigure this receive location’s XmlReceive pipeline. The button to get to it is shown here

Now we can change the Recoverable Interchange setting to true

Now when we deploy the last message we get two child messages into the non-error send folder and one into the error send folder – the one that could not be processed.

Well that seems to work pretty well for an incorrect well-formed message – lets try a non-well-formed message:

<ns0:envelope xmlns:ns0="http://dotnetconsult.co.uk/schema/envelope">

  <ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">

    <firstName>Rich</firstName>

    <lastName>Blewett</lastName>

  </ns1:person>

  <ns1:personbad xmlns:ns1="http://dotnetconslt.co.uk/schema/person">

    <firstName>Niels</firstName>

    <lastName>Bergland</lastName>

  </ns1:person>

  <ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">

    <firstName>Jason</firstName>

    <lastName>Whittington</lastName>

  </ns1:person>

</ns0:envelope>

Notice that the second person record the start and end element names do not match. If we drop this into the receive folder we end up with the single envelope in the error send folder. This is because the XmlDisassembler requires well-formed XML to work. OK so that’s not going to work – what we need is another type of error to test. Lets make one of the child person records schema invalid.

<ns0:envelope xmlns:ns0="http://dotnetconsult.co.uk/schema/envelope">

  <ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">

    <firstName>Rich</firstName>

    <lastName>Blewett</lastName>

  </ns1:person>

  <ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">

    <lastName