The release of BizTalk 2006 has brought a lot of love to the messaging engine in terms of handling failures. Probably the biggest change was messaging errors like “no subscription” are now resumable. However, there are other changes such as being able to subscribe to failed messages using the ErrorReport context properties and enabling routing for failed messages on a receive or send port.
Another change was added in the Xml and Flat File disassemblers – that of recoverable interchange processing. When a disassemble is breaking apart a composite message either in the form of a flat file or an envelope schema what happens if something is wrong about one of the child messages? In BizTalk 2004 the entire batch would fail. In BizTalk 2006 this is still the default behaviour; however, you can now set RecoverableInterchange to true on the disassembler which will cause all successful messages to be processed and only the incorrect ones to fail.
This seems like a great idea but in this article I want to look at some of the subtleties in the XmlDisassembler that make this technology less useful than may first appear.
First, here is the setup. I have two schemas: an envelope that is simply a container for repeated records; a person schema with child elements firstName and lastName as part of a sequence group.
This is the person schema
<xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003"
xmlns="http://dotnetconslt.co.uk/schema/person"
targetNamespace=http://dotnetconslt.co.uk/schema/person
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="person">
<xs:complexType>
<xs:sequence>
<xs:element name="firstName" type="xs:string" />
<xs:element name="lastName" type="xs:string" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
And here is the envelope
<xs:schema xmlns:ns0="http://dotnetconslt.co.uk/schema/person"
xmlns:b="http://schemas.microsoft.com/BizTalk/2003"
xmlns="http://dotnetconsult.co.uk/schema/envelope"
targetNamespace="http://dotnetconsult.co.uk/schema/envelope"
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:import schemaLocation=".\person.xsd"
namespace="http://dotnetconslt.co.uk/schema/person" />
<xs:annotation>
<xs:appinfo>
<b:schemaInfo is_envelope="yes"
xmlns:b=http://schemas.microsoft.com/BizTalk/2003 />
<b:references>
<b:reference
targetNamespace="http://dotnetconslt.co.uk/schema/person" />
</b:references>
</xs:appinfo>
</xs:annotation>
<xs:element name="envelope">
<xs:annotation>
<xs:appinfo>
<b:recordInfo body_xpath="/*[local-name()='envelope' and
namespace-uri()='http://dotnetconsult.co.uk/schema/envelope']" />
</xs:appinfo>
</xs:annotation>
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="1"
maxOccurs="unbounded"
ref="ns0:person" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
Notice the emboldened pieces that the XmlDisassembler uses to break apart the envelope.
Now, I deploy these two schemas into Biztalk, and then create a Receive Port with a Receive Location using the XmlReceive pipeline. I make sure that error reporting is turned on.
Next I create two send ports: one with a filter set to the BTS.ReceivePortName of the receive port; one using a filter set for error reporting like this
So with the infrastructure in place we can try submitting a message. Let’s start with one that works just to ensure that the envelope is working correctly. Here is the good message:
<ns0:envelope xmlns:ns0="http://dotnetconsult.co.uk/schema/envelope">
<ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">
<firstName>Rich</firstName>
<lastName>Blewett</lastName>
</ns1:person>
<ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">
<firstName>Niels</firstName>
<lastName>Bergland</lastName>
</ns1:person>
<ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">
<firstName>Jason</firstName>
<lastName>Whittington</lastName>
</ns1:person>
</ns0:envelope>
And as expected, dropping this into the receive folder results in three messages in the non-error send folder.
Now let’s use this message
<ns0:envelope xmlns:ns0="http://dotnetconsult.co.uk/schema/envelope">
<ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">
<firstName>Rich</firstName>
<lastName>Blewett</lastName>
</ns1:person>
<ns1:personbad xmlns:ns1="http://dotnetconslt.co.uk/schema/person">
<firstName>Niels</firstName>
<lastName>Bergland</lastName>
</ns1:personbad>
<ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">
<firstName>Jason</firstName>
<lastName>Whittington</lastName>
</ns1:person>
</ns0:envelope>
The problem here is that we don’t have a schema deployed describing the personbad element. We drop this into the receive folder and this time we get the default behaviour that the whole batch gets abandoned and the single envelope message ends up in the error send folder.
OK let’s turn on recoverable interchange now. We could deploy a new pipeline with the XmlDisassembler configured differently. However in BizTalk 2006 there is a UI for the per-instance pipeline data so we can actually reconfigure this receive location’s XmlReceive pipeline. The button to get to it is shown here
Now we can change the Recoverable Interchange setting to true
Now when we deploy the last message we get two child messages into the non-error send folder and one into the error send folder – the one that could not be processed.
Well that seems to work pretty well for an incorrect well-formed message – lets try a non-well-formed message:
<ns0:envelope xmlns:ns0="http://dotnetconsult.co.uk/schema/envelope">
<ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">
<firstName>Rich</firstName>
<lastName>Blewett</lastName>
</ns1:person>
<ns1:personbad xmlns:ns1="http://dotnetconslt.co.uk/schema/person">
<firstName>Niels</firstName>
<lastName>Bergland</lastName>
</ns1:person>
<ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">
<firstName>Jason</firstName>
<lastName>Whittington</lastName>
</ns1:person>
</ns0:envelope>
Notice that the second person record the start and end element names do not match. If we drop this into the receive folder we end up with the single envelope in the error send folder. This is because the XmlDisassembler requires well-formed XML to work. OK so that’s not going to work – what we need is another type of error to test. Lets make one of the child person records schema invalid.
<ns0:envelope xmlns:ns0="http://dotnetconsult.co.uk/schema/envelope">
<ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">
<firstName>Rich</firstName>
<lastName>Blewett</lastName>
</ns1:person>
<ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">
<lastName>Bergland</lastName>
<firstName>Niels</firstName>
</ns1:person>
<ns1:person xmlns:ns1="http://dotnetconslt.co.uk/schema/person">
<firstName>Jason</firstName>
<lastName>Whittington</lastName>
</ns1:person>
</ns0:envelope>
This time the second person record has the first and last name element transposed. As this is part of a sequence group this is not valid according to the schema.
We drop this message into the receive folder and we get three messages in the non-error send folder ... what?! It turns out that by default BizTalk with the XmlReceive pipeline does not perform validation.
However, we can force the XmlDisassembler to validate messages – once again lets use the per-instance pipeline data to achieve this
Now we drop the last message into the receive folder again and we get the single envelope message in the error send folder. Hmmm, so apparently the XmlDisassembler validates the entire envelope which is not what we want at all. The envelope validation is failing before we ever take the envelope apart.
All is not lost. Let’s validate the messages after disassembly. To do this we can use the XmlValidator component. But this component is not in the XmlReceive pipeline so we have to deploy a new pipeline with the XmlDisassembler in the disassemble stage and the XmlValidator in the validation stage. We’ll configure the XmlValidator to use the person schema to validate the message as this will be the message type after the disassember has run. Remember the XmlDisassembler will need to use recoverable interchange but not perform validation. So its pipeline data will look something like this:
So now we drop our envelope, with the schema invalid child, into the receive folder and out the other end we get ... the envelope message in the error send folder. The problem here is that Recoverable Interchange is only valid during the disassemble stage and so again the validation error causes the whole batch to fail and we are back to square one.
OK, so validation will have to take place in the disassemble stage. Unfortunately even if we make the envelop “dumb” with the child elements weakly typed as xs:any, the actual message validation uses the deployed schema and before the disassembly takes place – so the whole message is validated, not the ones resulting from disassembly.
So it would appear that as long as the envelope message has a matching pair on invalid child root node elements then Recoverable Interchange adds value. However, the likelihood that this is the only sort of invalid message you will get is, to me, pretty unlikely so based on that I’d say that Recoverable Interchange is of limited value with the XmlDisassembler. The frustrating thing is all it would take to make it a very powerful tool is to allow validation to only run on the disassembled messages and not on the envelope as a hole. In that case the failures would be within the scope of recoverable interchange and, therefore, the good messages would get through and the bad ones be caught.