Archive for the ‘BizTalk 2006’ Category
Posted by Richard »
5 Comments »The XPath function that’s available directly inside BizTalk orchestration is a powerful little tool. However I’ve seen a couple of project where developers just grown tired of it and started creating their own little libraries instead. I’ll be the first to admit that the XPath function isn’t perfect, and it sure doesn’t work like most of the other XPath engines (which is the biggest problem) but it’s still inside the orchestration and you can use it to both read and assign values to a message which is super useful! Basically I don’t see a valid reason for bringing more complexity into your solution by adding another library – as long as you’re just going to read or set value using XPath.
However there is one trick that you should know of when it comes to reading a text value from a node. Basically you have to use both the string() and
text() XPath functions. Both Charles Young and Yossi Dahan has good post on this subject. Also if your new to writing XPath expressions for complex schemas with loads of namespaces and stuff (like schemas in BizTalk) this post could be useful for you.
Finally a nice tool for writing and testing small XPath expression inside Visual Studio (if you don’t want to spend x minutes waiting for XmlSpy to start up …) is XPathmania. Read about it here – I use it all the time!
Posted by Richard »
Add Comment »This post shows how it’s possible to create your own .NET class and then use this within your orchestration (as a variable) to work with typed lists - something that unfortunately isn’t supported out of the box in BizTalk orchestrations. Below is the class used to create a typed list for System.String objects.
using System;
using System.Collections.Generic;
using System.Text;
namespace Sample.BizTalkCollections
{
[Serializable]
public class StringList
{
private List<string> _list = new List<string>();
public void Add(string item)
{
_list.Add(item);
}
public int Count()
{
return _list.Count;
}
public bool Remove(string item)
{
return _list.Remove(item);
}
public void RemoveAt(int index)
{
_list.RemoveAt(index);
}
public override string ToString()
{
StringBuilder builder = new StringBuilder();
foreach (string item in _list)
{
builder.AppendLine(item);
}
return builder.ToString();
}
}
}
Notice that the class is serializable so BizTalk can serialize when dehydrating the orchestration to the database. We’ve also chosen not to extend the IList interface as we don’t want to expose all methods of that interface, but only the once we’ll really going use within our orchestrations.
This StringList class exists within the BizTalkCollections namespace where we have other classes for lists types with different datatypes (for example ObjectList, Int32List and so on). Below is a picture showing the structure of the Visual Studio project containing all the collections we current have implemented. This project and the BizTalk testproject is available for download here.
In the orchestration we then have set up a variable and type this to our StringList class within the Sample.BizTalkCollections namespace.
Also make sure the assembly with your collection class is installed in the GAC, otherwise you’ll end up with a FileNotFound exception saying
Could not load file or assembly ‘Sample.BizTalkCollections, Version=1.0.0.0, Culture=neutral, PublicKeyToken=7109528d5dbf986b’ or one of its dependencies. The system cannot find the file specified.
Then you can use the following code within your orchestration! Download the sample project and test it.
A really generic class
I’d really like for it be possible to create my own generic classes and then initialize these using something like the below.
genericList = Sample.BizTalkCollections.GenericList<System.String>();
However this isn’t possible as BizTalk orchestrations don’t seem to understand generic classes … If someone found a solution to this please let me/us know via the comments to this post.
Posted by Richard »
37 Comments »Update 2007-07-05: The example project used in the post can be downloaded from here.
The operations dll (Micrsoft.BizTalk.Operations) is one of the new bits in BizTalk 2006 that at least I haven’t heard much about (most of you probably find in the C:\Program Files\Microsoft BizTalk Server 2006 folder). However it’s a library with a lot of useful functionality.
In this post I’ll focus on how it’s possible to use the library to get hold of message parts and message context from the BizTalk tracking database (usually called the BizTalkDTADb). If you’re unfamiliar with the architecture of the tracking in BizTalk this article is a good place to start.
What used to be the problem?
In BizTalk 2004 one soon got into problems when trying to read the message parts and the message context from the tracking database. The problem is that the the Xml is compressed (something that makes totally sence – Xml is a perfect candidate for compression). To my knowledge no one has found a good way to decompress the message context. There’s however a way to decompress the message parts (and only the parts) that Rob posted on in the BizTalk newsgroup. But this method doesn’t work on the compressed context of the same message that the parts belong to! One could assume that the same method could be used for both compression and decompression but I haven’t got it to work for the context of the message (read about my frustration here)
What’s wrong with WMI and MSBTS_TrackedMessageInstance?
So the option that was left to us before BizTalk 2006 for reading all the message parts as well as the message context was to use the MSBTS_TrackedMessageInstance WMI scrtipt and save everything down to file. The problem with this approach is that it’s slow and ugly! Say that we like to save a couple of thousand messages from the tracking database. When forced to save the messages to file we’ll end up with a slow and ugly solution where writing to file takes ages, then we have to read the content of the different files before cleaning up and deleting the files.
Micrsoft.BizTalk.Operations to the rescue
A quick view in Lutz Roeder’s Reflector shows us a couple of interesting methods in the operations dll. However I’ll only use the GetTrackedMessage method in this post.
I’ve written a tiny test application that uses the library to read the body part of the message (the meat of the message) and a couple of properties I’m interested in from the context of the message. It looks something like this. If your interested drop me an email and I’ll send it. The first screenshot below shows an example of reading the InterchangeID from the context of a tracked message.
This screenshot shows an example of reading the full body of a tracked message from the tracking database. Try doing this with less then 10 lines of code using the MSBTS_TrackedMessageInstance script!
There is really nothing to the application, reading a tracked message using the MessageID will return an object implementing IBaseMessage and we basically read the BodyPart property of - it really couldn’t be any easier.
public static string GetMessageBodyByMessageID(string dbServer, string dbName, Guid messageID)
{
TrackingDatabase dta = new TrackingDatabase(dbServer, dbName);
BizTalkOperations operations = new BizTalkOperations();
IBaseMessage message = operations.GetTrackedMessage(messageID, dta);
string body = string.Empty;
using(StreamReader streamReader = new StreamReader(message.BodyPart.Data))
{
body = streamReader.ReadToEnd();
}
return body;
}
I’d be very interested in how people use the operations library both when it comes to read and use tracked messages, but also in other ways! Use the comments to tell me and other readers how you use it your solutions!
Posted by Richard »
3 Comments »Ever had a “use of unconstructed message” error message in your orchestrations? Then you know that messages has to be constructed before one can use them within orchestrations. Basically messages in orchestrations are created in ports or when executing a map in a construct shape. However there is also the possibility of creating messages in expression shapes using code. Matt Meleski does a great job explain the different approaches in his Constructing BizTalk 2004 XML Messages (In an Orchestration) – Choices blog post.
However there are some pitfalls when constructing messages from scratch within orchestration (a part from the fact that there really isn’t a clean way of doing it … Scott Colestock has some ideas in this post).
A common problem is XPathUpdateException. Say one has a mapping that looks like the one below where one node isn’t mapped. This will create a message without the ElementB element. This doesn’t have to be in a mapping, it could also happend when constructing a message using code in a expression shape. A common scenario is that we add code for constructing the bone structure of the XML message, then change the schema without updating the construct code. Ergo a message will be constructed without that element (could be both valid or invalid XML then depending on the Min Occurs value)!
When we then try to add a value to the element that hasn’t been created the exception will be thrown!
One way around when using a map is to make sure to have a default value on every element to force them be constructed. Adding the following to map will make it work.
It’s a one time mistake but it’s good to know what to look for when the XPathUpdateException in Microsoft.XLANGs.Core.XSDPart.SetDistinguishedField (if it’s a distinguished field that is) shows up in the event log.
Posted by Richard »
3 Comments »Recently I had to insert a record using a stored procedure and the SQL Adapter in BizTalk 2006. There are lots of examples on both how to insert records and how to select a number of record using this adapter. However I had problems finding how to insert a record and receiving the new id of the inserted row in return (the SCOPE_IDENTITY()). In my scenario I needed the id to insert into another database further down in the orchestration.
I ended up with a stored procedure looking like the below.
ALTER PROCEDURE [dbo].[TestInsertParty]
@partyName nchar(30) = NULL
AS
BEGIN
SET NOCOUNT ON;
Insert Into Party ([name], chain_idx) Values(@partyName, NULL)
Select Scope_Identity() As Id For Xml Raw ('Response')
END
The trick was to use the XML RAW Mode. This mode transforms the result set into a generic identifier as <row>. It is however possible to provide a element name, as <Response>. Basically this will insert the new value and return something like this from the stored procedure.
After return via the send port the orchestration will receive something like the below.
<TestInsertResponse xmlns="TestInsert">
<Response Id="1054" />
</TestInsertResponse>
The schema that I use to both handling the response and request against the SQL Adapter is shown below. First I set the type of the Id-attribute to xs:int but this gave me some problems when using the promoted value in the orchestration, everything worked fine when switching back to xs:string.
The same technique would be used for receiving a code from the stored procedure (say 1 for success and 0 for failure or whatever) and then to make a logical decision in the orchestration.
Posted by Richard »
Add Comment »Convoying is one of BizTalk’s real strengths but it has a lot of pitfalls (I’m talking performance, zombies etc). This article is really good at explaining the different patterns used for creating convoys. It also makes some deep dives into how the subscriptions are solved, why zombies are created and how to deal with them.
Even if you feel you understand the convoy patters from before the part about how BizTalk solves the subscriptions for correlations (the part called Basic Convoy Theory) is great.
Using sequential convoy to handle ordered delivery
A big selling point in BizTalk 2006 is it’s ability to handle Ordered Delivery. It’s important however to understand that this setting (on the Receive and Send port) only works in a pure messaging scenario (a scenario without orchestration, just passing messages between ports). To get order delivery in a scenario using orchestrations one has to use the sequential convoy pattern (basically forcing the orchestration to only one instance on one thread). This Webcast explains this in depth and also deals with some of the different problems that are related to the issue. Problems like performance of course, but also the requirement for receive adapter (on the send side all adapters support ordered delivery) for enabling ordered delivery (you’ll have to use MSMQ, MQSeries or specific scenarios of File, SOAP or HTTP).
Both convoying and ordered delivery are important concepts to understand in depth to be able to make the right decisions in a BizTalk 2006 solution.
Posted by Richard »
1 Comment »I’ll start trying to post a weekly sum up of BizTalk, .NET topics I’ve come across during the week. We’ll see how goes as I’m not the most constant blogger …
However I worked with mainly three things this week.
Error handling and General Exception
I’ve tested some ideas relating to an error handling pattern we like to implement in a one of our major integration projects. We’ve looked at how the new BizTalk 2006 Failed Message Routing fits in with our project. The idea is to have and Orchestration listening to the Error Message, log these and then put the failing message parts to file. We’ll then have a system that picks up the failed messages, launches them in an editor and makes it possible to resend the message after it’s been edited.
Problem arises when we get to handling exceptions in Orchestrations. Orchestrations doesn’t have the possible to use Failed Message Routing as ports does (I guess that ok, we might want handle different errors in different ways and so on.). However that means that we have to catch an error and create our own Error Message to submit to the message data box for further routing to the Orchestration handling errors (the one that logs and puts the message to file.) So far so good as long as we don’t get an General Exception! Usually when getting an General Exception, and we don’t catch it, the message gets suspended and we get a some error details giving us an idea of what went wrong. However as soon as one catches a General Exception we loose all information about the error … We’ve still haven’t found a good way of handling these errors. We just can’t afford loosing that error information!
General Exceptions are still a mystery to me. I guess that are there so BizTalk has a chance in catching exception that aren’t raised from .NET based code. But what kind of errors within an orchestration raises these kind of exceptions? I know a failure in a mapping does, what else? I’d really like this scenario to better documented. The error handling patter is based on a chapter in the Pro BizTalk 2006 book (which is a excellent book in my opinion - buy it!) but the General Exception and how to deal with it isn’t discussed there either. Any ideas, book, articles are highly appreciated!
Validation
One important thing to think of in a BizTalk solution is to not let anything either in or out that doesn’t validate (ok, there are exception to this, but generally speaking)! BizTalk has great built in support for this in the XML Validator Pipeline Component (you might also want to have a look at this code). However when working with this it’s important to understand what one is validating against and it suddenly even more important to understand every details in the schemas (external system owners usually like some technical explanation when your telling them that their messages don’t validate in your pipeline ;)). Things we run into this week is the elementFormDefault attribute. The following reading help me understanding what the attribute does:
Another thing we ran into was white space handling and the xml:space attribute. Apparently if one likes to have a node with a space in it BizTalk removes this if the element doesn’t have the xml:space attribute. so <node> </node> will come out as <node></node> – sound familiar?
But if the schema doesn’t declare that the node will have a xml:space attribute the validation will fail! To get this working one has to get the schema declaring the the xml:space attribute and then reference it like this example. The schema from w3c (if you don’t write your own like in the article) is located here.
End-to-end tracing in a SOA
I’ll make this short even if it should be the longest part in this post. Basically we’re trying to archive the following:
Our solution send loads of different messages types between five different BizTalk servers. The client likes to be able to have full text search within these messages and to also be able to see all the messages in each interchange within a server.
So say for example that we receive and Order message in a flat file format. This is transformed in to a Xml message that is then routed down to two other different BizTalk servers. It should then be possible for the client to start an application, click on the Order message type, enter for example the order number (an element i in the message) and see the the full content of the different files within the interchange (in this example that would be the flat file and the Xml file) where that Order number is found.
Basically it’s does what the HAT does (with full text search and a custom GUI). In the first phase of the project we’ll have it work per server, but in the final solution this should perform over all servers … I’ll come back to this in a later post. In the mean time listen to this episode of Hanselminutes on end-to-end tracking. This will be a challenge in every service orientated architecture …
Finally
I’ve sold my iPod Nano and ordered the Creative Vision Zen:M. Now I’ll catch up on all those dnrTV Webcasts! I’ll also make sure to watch everyone of the BizTalk 2006 Webcasts during my commute to work.
Let me know if you found some of this information useful and I’ll try harder to post some like this every Friday.
Posted by Richard »
3 Comments »The version redirection in BizTalk do NOT work as one is used to coming from a ordinary .NET developer background (so I was wrong in this post …). Say that we made a reference to a code library in one of our orchestrations. This code library is in version 1.0.0.0 when we build and deploy the orchestration to BizTalk and to the GAC. The setup in VS 2005 looks something like this.
Then we make some minor changes in the code library, we fix them and set the version to 1.0.1.0. Build and deploy it. This means that we now have one 1.0.0.0 and one 1.0.1.0 version side-by-side in the GAC (as shown in the figure below).
Coming from .NET the CLR should now load the 1.0.1.0 version as it identifies assemblies based on the assembly name, major or minor version (NOT the build and revision version), public key token and culture (more about this here).
However, this does NOT work in BizTalk. BizTalk loads assemblies by fullname it has stored in the management database and the build and revision number are part of the fullname … This means we have to build and redelopy EVERY part where we like to use new version of the code library.
There is one way around this for emergency use. Say that one has a code library that is used in loads of BizTalk artifacts (A place where we store all base functionality for the solution). Rebuilding all those parts and redeploying it, just to be able to update the version number, is not going to happened! It’s to much work and to risky. So it’s possible to make a change in the BTSBTSvc.exe.config file instead. In my case the change would be something like the below (read more about here).
<dependentAssembly>
<assemblyIdentity name="StaticCodeLibrary" publicKeyToken="32ab4ba45e0a69a1" culture="neutral" />
<bindingRedirect oldVersion="1.0.0.0" newVersion="1.0.1.0"/>
</dependentAssembly>
Posted by Richard »
1 Comment »UPDATE: This does not apply to BizTalk … I’ve made an update post here. Sorry.
We’re working with a lot of code libraries that we use in different parts of our BizTalk solutions. As the are used on several servers and by loads of different “BizTalk parts” (both in orchestrations and maps) it’s important that we always keep the version number of the assemblies up to date. That means that every little change should increase the current version number. But as they are used in so many places people have started to skip this step as they thought they had to compile all parts that should use the new code (say it’s a bug fix and you’d like all “using parts” of the assembly to load the updated version). This is where GAC loading policy comes to the resource!
First we have to understand that every .NET assembly is identified using four characteristics:
- Assembly name
- Major or minor version
- Public key token
- Culture
Then we need to know that the first version number in for example version 1.1.2.1 is the major version. The second is the minor version and the third and fourth are build, revision version number. So this means that if you have 1.1.2.1 installed and make a minor change the easiest way to use the new assembly is to change the one of the build or revision numbers (the third or fourth number). Then the CLR will load the new assembly without any other changes!
But sometimes we have to change the minor or major version - and we still don’t have to recompile a thing! We can use a publisher policy file. This is an example of such a file defined for version 1.0.0.0 moving to 2.0.0.0.
<configuration>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<dependentAssembly>
<assemblyIdentity name="BaseHelper" publicKeyToken="18517ea673f8584b" culture="neutral" />
<bindingRedirect oldVersion="1.0.0.0" newVersion="2.0.0.0"/>
</dependentAssembly>
</assemblyBinding>
</runtime>
</configuration>
This kb article describes what to do next:
- Change the version and recompile. The first step is to create the new version of your component. After you’ve done that, you will need to modify the version number in the AssemblyInfo file for your component.
Create the publisher policy file. Create the publisher policy file for the assembly using the format shown above.
Use Assembly Linker (Al.exe) to create the publisher policy assembly. The Assembly Linker is included with the .NET Framework SDK. To create the publisher policy assembly that redirects a binding from version 1.0 of Website.dll to version 2.0 using a publisher policy file called website.config, run the following command:
al /link:BaseHelper.config /out:policy.1.0.BaseHelper.dll /keyfile:c:\keyfile.snk
This command will create a new assembly called policy.1.0.BaseHelper.dll. This naming convention is important, as indicated in the “What Is a Publisher Policy Assembly?” section.
- Install the publisher policy assembly into the Global Assembly Cache. The publisher policy assembly is installed into the GAC. It will be used by the .NET runtime when any application attempts to bind to version 1.0 of the BaseHelper.dll, and it will force the application to bind to the new version automatically.
- Install the new version into the Global Assembly Cache. Install the new version of the component into the GAC. After the new version has been installed, the old version can be safely removed.
So no more excuses for not updating the version number!
Posted by Richard »
Add Comment »I found this post from Richard Seroter’s blog on persistence points interesting. The post deals with the basics in persistence points and how to tweak them to get a resumed orchestration to resume at an expected step. Not understanding persistence point and not thinking about them when planing and building an orchestration might cause some strange bugs and errors.
