Posted by Richard »
7 Comments »Update 2010-04-13
Grant Samuels commented and made me aware of the fact that inline scripts might in some cases cause memory leaks. He has some further information here and you’ll find a kb-article here.
I’ve posted a few times before on how powerful I think it is in complex mapping to be able to replace the BizTalk Mapper with a custom XSLT script (here’s how to). The BizTalk Mapper is nice and productive in simpler scenarios but in my experience it break down in more complex ones and maintaining a good overview is hard. I’m however looking forward to the new version of the tool in BizTalk 2010 – but until then I’m using custom XSLT when things gets complicated.
Custom XSLT however lacks a few things once has gotten used to have – such as scripting blocks, clever functoids etc. In some previously post (here and here) I’ve talked about using EXSLT as a way to extend the capabilities of custom XSLT when used in BizTalk.
Bye, bye external libraries – heeeello inline scripts 
Another way to achieve much of the same functionality even easier is to use embedded scripting that’s supported by the XslTransform class. Using a script block in XSLT is easy and is also the way the BizTalk Mapper makes it possible to include C# snippets right into your maps.
Have a look at the following XSLT sample:
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet
version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:msxsl="urn:schemas-microsoft-com:xslt"
xmlns:code="http://richardhallgren.com/Sample/XsltCode"
exclude-result-prefixes="msxsl code"
>
<xsl:output method="xml" indent="yes"/>
<xsl:template match="@* | node()">
<Test>
<UniqueNumber>
<xsl:value-of select="code:GetUniqueId()" />
</UniqueNumber>
<SpecialDateFormat>
<xsl:value-of select="code:GetInternationalDateFormat('11/16/2003')" />
</SpecialDateFormat>
<IncludesBizTalk>
<xsl:value-of select="code:IncludesSpecialWord('This is a text with BizTalk in it', 'BizTalk')" />
</IncludesBizTalk>
</Test>
</xsl:template>
<msxsl:script language="CSharp" implements-prefix="code">
//Gets a unique id based on a guid
public string GetUniqueId()
{
return Guid.NewGuid().ToString();
}
//Formats US based dates to standard international
public string GetInternationalDateFormat(String date)
{
return DateTime.Parse(date, new System.Globalization.CultureInfo("en-US")).ToString("yyyy-MM-dd");
}
//Use regular expression to look for a pattern in a string
public bool IncludesSpecialWord(String s, String pattern)
{
Regex rx = new Regex(pattern);
return rx.Match(s).Success;
}
</msxsl:script>
</xsl:stylesheet>
All one has to do is to define a code block, reference the xml-namespace used and start coding! Say goodbye to all those external library dlls!
It’s possible to use a few core namespaces without the full .NET namespace path but all namespaces are available as long as they are fully qualified. MSDN has a great page with all the details here.
Posted by Richard »
3 Comments »Recently there’s been a few really good resources on streaming pipeline handling published. You can find some of the here and here.
The Optimizing Pipeline Performance MSDN article has two great examples of how to use some of the Microsoft.BizTalk.Streaming.dl classes. The execute method of first example looks something like below.
public IBaseMessage Execute(IPipelineContext context, IBaseMessage message)
{
try
{
...
IBaseMessageContext messageContext = message.Context;
if (string.IsNullOrEmpty(xPath) && string.IsNullOrEmpty(propertyValue))
{
throw new ArgumentException(...);
}
IBaseMessagePart bodyPart = message.BodyPart;
Stream inboundStream = bodyPart.GetOriginalDataStream();
VirtualStream virtualStream = new VirtualStream(bufferSize, thresholdSize);
ReadOnlySeekableStream readOnlySeekableStream = new ReadOnlySeekableStream(inboundStream, virtualStream, bufferSize);
XmlTextReader xmlTextReader = new XmlTextReader(readOnlySeekableStream);
XPathCollection xPathCollection = new XPathCollection();
XPathReader xPathReader = new XPathReader(xmlTextReader, xPathCollection);
xPathCollection.Add(xPath);
bool ok = false;
while (xPathReader.ReadUntilMatch())
{
if (xPathReader.Match(0) && !ok)
{
propertyValue = xPathReader.ReadString();
messageContext.Promote(propertyName, propertyNamespace, propertyValue);
ok = true;
}
}
readOnlySeekableStream.Position = 0;
bodyPart.Data = readOnlySeekableStream;
}
catch (Exception ex)
{
if (message != null)
{
message.SetErrorInfo(ex);
}
...
throw ex;
}
return message;
}
We used this example as a base when developing something very similar in a recent project. At first every thing worked fine but after a while we stared getting an error saying:
Cannot access a disposed object. Object name: DataReader
It took us a while to figure out the real problem here, everything worked fine when sending in simple messages but as soon as we used to code in a pipeline were we also debatched messages we got the “disposed object” problem.
It turns out that when we debatched messages the execute method of the custom pipeline ran multiple times, one time for each sub-messages. This forced the .NET Garbage Collector to run.
The GC found the XmlTextReader that we used to read the stream as unreferenced and decided to destoy it.
The problem is that will also dispose the readOnlySeekable-Stream stream that we connected to our message data object!
It’s then the BizTalk End Point Manager (EPM) that throws the error as it hits a disposed stream object when trying to read the message body and save it to the BizTalkMsgBox!
ResourceTracker to the rescue!
Turns out that the BizTalk message context object has a nice little class connected to it called the ResourceTracker. This object has a “AddResouce”-method that makes it possible to add an object and the context will the hold a reference to this object, this will tell the GC not to dispose it!
So when adding the below before ending the method everything works fine – even when debatching messages!
context.ResourceTracker.AddResource(xmlTextReader);
Posted by Richard »
Add Comment »Update 2010-09-30: BizMon is now owned and developed by Communicate Norway. They have renamed, and further developed the product, to IPM (Integration Monitoring Platform) – check it out here.
I’ll start this post by clarifying two important things
- I am involved in the development and marketing of “BizMon”. Therefore I am biased and you have to decide for yourself if that affects the content of the post. As always it is best to try it for yourself and see if it is useful for you.
- I have talked about BizTalk monitoring tools in a previous post and my goal then was then to start a an open source project. That did not happened and you can read why in the update to that post.
Why “monitoring” for BizTalk?
I have worked as a BizTalk developer for many years but it was not until I really got in to maintaining a large integration solution that I realized that the tools I really needed was not there. I found myself using the following “tools” and techniques over and over again.
- Open the BizTalk Administration Console and query for suspended messages, running instances, routing errors etc, etc.
But as I had to pull for this information it took time and discipline (two things I’m short of) to quickly find out when errors occurred.
- I used the HAT to try and find out when the last messages was sent and received on the different applications. This gave me a “guarantee” that things worked as I accepted and that the solution had a “pulse” – messages at least moved back and forward.
The problem is that the HAT tool is bad and it is hard to find what one is looking for (It is a bit better in BizTalk 2009 but it is still tricky to get useful information out of it.)
- Some of the integrations in our environment used BAM to track messages and their state.
The problem was that all solutions was developed by either myself or different consultants. This made it hard to get everyone to use the same tracking. It was also hard to convince management to go back and try and “instrument” old working integrations with BAM tracking.
At the same time as we had the “tools” and techniques mentioned above available, management had the following requirements for us.
- Start working on fixing an error within 10 minutes after it occurred 24/7 all 365 days …
- Be able to delegate simple monitoring task to support personnel (a help desk).
- Not have to actively “pull” for information but be quickly altered of errors and get the information “pushed” to us.
The idea was that this would would save time as people don’t have to look for errors when everything is working fine. Time that people can use for other tasks …
- Enable reporting so we can provide systems owners and other interested people with information on how much data has been sent received to the systems and parties they care about.
All the above lead up to the realization that we needed some sort of tool.
What are the existing options for BizTalk monitoring tooling?
At the time we started looking for options all we could find was System Center Operations Manager (SCOM). We looked at SCOM BizTalk Management Pack and decided that for us this was not the right solution. It was too big, too complicated and it would be to hard to get it to the what we wanted to do.
The decision to not use SCOM I think was right for us. We wanted something leaner and more specialized. I am however not saying that it is the right decision for you.
If you are successfully suing SCOM to monitor BizTalk I would love to hear about it!
What we ended up with
We ended up building BizMon. It does what we need and our help desk can now basically monitor about 100 different BizTalk application themselves. At the same time they do all the other support task they have to do. When something happens (and it does …) they are the first to know. Some easy tasks they can solve themselves, otherwise they make sure to notify the users and quickly call the developer that knows more and can help them.
Support personnel can now also setup custom reports that users can subscribe to, all based on BAM that they now easily can interject tracking points in existing solutions – both new and old ones.
As I said. This worked out out good and helped us. If you think that it could work for you as well – give it a try.
I am also really interested to how you have solved similar requirements as we had with your own tool or other solutions.
What else is there?
Recently FRENDS released a beta version of their FRENDS Helium product that looks promising could potentially solve a lot of the same issues that BizMon does and that I have discussed in this post.
Check it out and let us know what you think.
