cXML (commerce eXtensible Markup Language) is a XML based standard for communication of data related to electronic commerce. The problem from a BizTalk perspective is that they don’t publish any XML schemas (XSD),  only Document Type Definition (DTD).

When trying to generate a schema based on a DTD using the functionality in BizTalk (via Add Generated Items) one ends up with a schema split of three files that really doesn’t make any sense (XmlSpy doesn’t do a very good job either …). So after a while I found Nick Heppleston schema repository! After some tweaking I actually had a cXML Order schema in the version I was looking for! Thanks Nick!

The next set of problems was to handle the lack of XML namespace and the DOCTYPE declaration that messages validating against DTD carries on top.

<?xml version="1.0" standalone="no"?>
<!DOCTYPE cXML SYSTEM "http://xml.cxml.org/schemas/cXML/1.2.014/cXML.dtd">
<cXML xml:lang="en-US" payloadID="2007117.25919@Contempus" timestamp="2007-11-07T11:06:16+01:00">

To handel these two issues I set up a receive pipeline that looked like the one below.

Remove the DOCTYPE declaration

First I created a pipeline component to remove the DOCTYPE node. It’s simple code using regular expression to find the DOCTYPE node, replace it with an empty string and return the message.

public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)
{
   string messageString = new StreamReader(inmsg.BodyPart.Data).ReadToEnd();
   Regex doctypePattern = new Regex("<!DOCTYPE.+?>");
   messageString = doctypePattern.Replace(messageString, string.Empty);
   MemoryStream memStream = new MemoryStream();
   byte[] data = Encoding.UTF8.GetBytes(messageString);
   memStream.Write(data, 0, data.Length);
   memStream.Seek(0, SeekOrigin.Begin);
   inmsg.BodyPart.Data = memStream;
   return inmsg;
}

Set an XML namespace

Secondly I used Richard Seroter’s post on how to change the SetNSForMsg component to add a XML namespace. That’s the second component showing in the decode stage of the pipeline.

Arrow number 3 shows how the SetMsgNS exposes a property that allows us to set the namespace that we can configure per pipeline. In this case I’ve set it to http://schemas.modhul.com/cXML/1.2.014/OrderRequest which is the namespace of the cXML schema I’m currently working agains.

In the end we’ll have a message with the following declaration and root node.

<?xml version="1.0" encoding="utf-16" standalone="no"?>
<cXML xml:lang="en-US" payloadID="2007117.25919@Contempus" xmlns="http://schemas.modhul.com/cXML/1.2.014/OrderRequest" timestamp="2007-11-07T11:06:16+01:00">

Now we’re ready to start mapping!

Ordered Delivery is a great feature of BizTalk 2006. The problem is however that as soon as one introduce a orchestration in the process it doesn’t work. The Ordered Delivery option on ports in BizTalk 2006 ensures two things today:

1. Messages are published to the MessageBox in the same order as they were consumed by the Receive Port.
2. Messages are consumed from the MessageB0ox and published to the end destination by the send port in the same order as they were persisted in the MessageBox.

What’s the problem?

Fine. But if we introduce an orchestration each message being persisted in the  the MessageBox will start it’s own orchestration instance. These instances could (and in many cases will) finish in different order than they started. This means that we loose the correct order of the messages one they are persisted back to the MessageBox from the orchestration instance.

The above “limitation” is well known and the solution to this has always been a Sequential Orchestration pattern (also called Singleton Orchestration). Basically this correlates messages from for example a specific Receive Port of a particular Message Type. This ensures that only one instance of the orchestration will be started and we can keep the order of the messages. We’ve experienced a lot of problems using this this kind of solution for Ordered Delivery - all from Zombies to them using huge amounts of memory and so on.

Microsoft’s new solution

Now it finally looks like this problem is about to be solved! Microsoft released a white paper recently (download it here - found it via Richard Seroter’s blog) to solve these problems in a other way. It basically means that a pipeline will stamp the messages with a sequence number as they enter BizTalk (the messages are then in order as we’re then using Ordered Delivery on the port). Then one can have as many orchestration as one like processing these messages and publish them back to the MessageBox (now possibly out of order but with a sequence number stamped on them). The last business orchestration will set a “destination” property that will route the message to a resequence orchestration. This orchestration resequences the messages and decides by checking the last sequence number it sent out if the current message should be sent out or put back on queue.

Conclusion

My biggest concern in this solution is that is still based on a singleton. We’ve had cases where where the send procedure been extremely slow (for example when we used SOAP and had a slow Web Service in the other end), then the orchestration has built up in memory as it queued messages internally. However it looks like this solution is well thought through with a “flush queue” functionality, stop message, some ideas on how to handle errors (remember that if the singleton fails and you don’t handle this your processes stop) and so on.

Read it and please let me know what you think!