1. I am involved in the development and marketing of “BizMon”. Therefore I am biased and you have to decide for yourself if that affects the content of the post. As always it is best to try it for yourself and see if it is useful for you.
2. I have talked about BizTalk monitoring tools in a previous post and my goal then was then to start a an open source project. That did not happened and you can read why in the update to that post.

Why “monitoring” for BizTalk?

I have worked as a BizTalk developer for many years but it was not until I really got in to maintaining a large integration solution that I realized that the tools I really needed was not there. I found myself using the following “tools” and techniques over and over again.

1. Open the BizTalk Administration Console and query for suspended messages, running instances, routing errors etc, etc.

   But as I had to pull for this information it took time and discipline (two things I’m short of) to quickly find out when errors occurred.

2. I used the HAT to try and find out when the last messages was sent and received on the different applications. This gave me a “guarantee” that things worked as I accepted and that the solution had a “pulse” – messages at least moved back and forward. 

   The problem is that the HAT tool is bad and it is hard to find what one is looking for (It is a bit better in BizTalk 2009 but it is still tricky to get useful information out of it.)

3. Some of the integrations in our environment used BAM to track messages and their state.

   The problem was that all solutions was developed by either myself or different consultants. This made it hard to get everyone to use the same tracking. It was also hard to convince management to go back and try and “instrument” old working integrations with BAM tracking.

At the same time as we had the “tools” and techniques mentioned above available, management had the following requirements for us.

1. Start working on fixing an error within 10 minutes after it occurred 24/7 all 365 days …
2. Be able to delegate simple monitoring task to support personnel (a help desk).
3. Not have to actively “pull” for information but be quickly altered of errors and get the information “pushed” to us.

   The idea was that this would would save time as people don’t have to look for errors when everything is working fine. Time that people can use for other tasks …

4. Enable reporting so we can provide systems owners and other interested people with information on how much data has been sent received to the systems and parties they care about.

All the above lead up to the realization that we needed some sort of tool.

What are the existing options for BizTalk monitoring tooling?

At the time we started looking for options all we could find was System Center Operations Manager (SCOM). We looked at SCOM BizTalk Management Pack and decided that for us this was not the right solution. It was too big, too complicated and it would be to hard to get it to the what we wanted to do.

The decision to not use SCOM I think was right for us. We wanted something leaner and more specialized. I am however not saying that it is the right decision for you.

If you are successfully suing SCOM to monitor BizTalk I would love to hear about it!

What we ended up with

We ended up building BizMon. It does what we need and our help desk can now basically monitor about 100 different BizTalk application themselves. At the same time they do all the other support task they have to do. When something happens (and it does …) they are the first to know. Some easy tasks they can solve themselves, otherwise they make sure to notify the users and quickly call the developer that knows more and can help them.

Support personnel can now also setup custom reports that users can subscribe to, all based on BAM that they now easily can interject tracking points in existing solutions – both new and old ones.

As I said. This worked out out good and helped us. If you think that it could work for you as well – give it a try.

I am also really interested to how you have solved similar requirements as we had with your own tool or other solutions.

What else is there?

Recently FRENDS released a beta version of their FRENDS Helium product that looks promising could potentially solve a lot of the same issues that BizMon does and that I have discussed in this post.

Check it out and let us know what you think.

I worked with the tool before but now I wanted to schedule the tool and to have MBV-reports e-mailed to relevant persons within the company on a weekly basis. This is quite easy to accomplish as MBV comes in two version. One GUI-based (shown below) version and one command-line based.

The command-line version is of course perfect for scheduling using the Windows Task Scheduler.

If you feel uncomfortable running all the queries (there is a lot of them) on a schedule you can pick some you find important and configure the tool to only run those. Jean-Pierre has a post on how to do just that here.

After MBV has completed all its queries and done its magic it will produce a html-report in the working folder (that’s the folder in the “Start in” field in the scheduled task example above).

We then use a tool called AutoMailer NT (cost €20 – there is a 30 days trial) to:

1. Poll the working folder for a *.html report file.
2. Compress the file (using zip).
3. Send the report to a configured list of recipients.
4. Delete the report file.

The AutoMailer NT installation is a bit rough (don’t miss to the separate download (!) of the trial certificate). But once you have everything working it’s great to have a fresh MBV report in you inbox every Monday telling you how your BizTalk environment is doing and possible issues to attend to.

One of the last episodes of dnrTV hosted Dom Demsak (Don XML), the creator of XPathmania. The show is 30% about XPathmania and 70% about XPath and XPath syntax in general. It’s kind of basic XPath but I think it can be useful for someone who feels they haven’t got full control of the language. 

During the show they touch on XML namespaces and XML default namespaces. However they don’t really explain the difference between them and how it effects the document. Something that’s sad as I feel that XML namespaces (and especially default namespaces) is something that most people haven’t fully understood.

Anyway, if you got some spare time watch it or forward it someone you think should watch it.

DebugView

Sysinternals (Windows Sysinternal now - Microsoft bought them last year) DebugView is a wonderful little tool and is especially useful when it comes to figure out what’s actually going on inside an orchestration. Basically the tool listens to system wide debug output. From an orchestration it’s possible to write debug information using the .NET System.Diagnostics namespace and the Debug or Trace class.

Decide on how to filter

There are a couple of handy little tricks that makes DebugView a even better in BizTalk development. First one should try and have something in the debug messages that makes it possible to filter and distinct one’s own (as DebugView listens system wide debug output all running applications debug info will show up). Our team decided on “Sogeti” (our company name) for all our development and to have a method in our BaseLibrary component that outputs something like the below (the BaseLibrary is a small little .NET component with a couple of very useful classes we use company wide in our BizTalk related development).

System.Diagnostics.Debug.WrtieLine("Sogeti, your debug/trace message here")

This make is possible to have a filter in DebugView and to for example have it look something like this.

Trace full context of messages

Another little useful trick is to trace the full context of messages. This is done be storing the message in a XmlDocument typed variable and get the OuterXml property of that variable. The below code is and example of this.

tempXml = msgFindPartyRequest.parameters;
System.Diagnostics.Trace.Write(System.String.Concat("Sogeti, msgFindPartyRequest: ", tempXml.OuterXml));

Example of a full message trace.

To Trace or Debug – that’s the question

As stated earlier both System.Diagnostics.Debug and System.Diagnostics.Trace has methods (Write, WriteLine and so on) for outputting debug information. However there is only one that stays in your compiled code when switching from Development to Deployment compilation mode (guess which one ). So make sure you choose the right class for the right information. I like to have some critical messages left using Trace and be able to trace these even on the test and production server.

DebugView on a remote desktop

When running DebugView on an other server (say a test or a production server) using Remote Desktop I’ve found that ones has to use the console user on the server. This kind of makes sense as if we’re connection “normally” we’re creating a virtual session and that’s not were the debug information is written to.

Microsoft SOAP Toolkit 3.0

This is a totally other tool than DebugView but I thought it fit here any way. It’s a handy tool when working with SOAP based messages. Without it’s very hard to actually figure out how the raw request and response message look and why your orchestration web service is acting the way it does.

The trace tool is placed as a reverse proxy between BizTalk and the Internet. It’s setup by telling the tracing tool which localhost port to listen at (for example 9091 as in the example below) then we’ll redirect to that port by changing the setting in the BizTalk send port.

Finally we’ll set up the trace tool to listen to port 9091 and redirect all traffic to our web service URL at port 80 in this case. So basically the trace tool will catch all the traffic hitting the 9091 port and forward it.

That’s it! This is probably basic stuff for most of you but hopefully it’s useful for someone! 

I’ve also noticed that the SOAP Toolkit is deprecated by Microsoft and I’d like to hear if anyone used something else (like Fiddler example) for tracing SOAP messages. I’d also love some other tips, tools and methods you use for debugging BizTalk orchestrations.

However there is one trick that you should know of when it comes to reading a text value from a node. Basically you have to use both the string() and
text() XPath functions. Both Charles Young and Yossi Dahan has good post on this subject. Also if your new to writing XPath expressions for complex schemas with loads of namespaces and stuff (like schemas in BizTalk) this post could be useful for you.

Finally a nice tool for writing and testing small XPath expression inside Visual Studio (if you don’t want to spend x minutes waiting for XmlSpy to start up …) is XPathmania. Read about it here – I use it all the time!

The application has easy access to almost all of the features that the ordinary web interface has. There was (and probably still are) an mobile version of the web interface but it didn’t have the easy access to all the different menus as this application has. I really like it and use it several times each day.

The supported devices are listed here and to get on your phone all you do is to visit this URL http://gmail.com/app with your device.

Another cool little application is the Opera Mini browser. These a full feature list here. Basically it transforms the pages so that they fit the mobile screen, and it’s fast! It also has some special features for navigation on small screens. And it supports RSS bookmarks!

The application then figures out all the dependencies that the selected artifacts has (In our solution an Orchestration for example might have > 20 dependencies to different schemas, pipelines, C# libraries etc, etc). 

All DLL:s of the different artifacts are then extracted from the local GAC (where they exist when deployed to the local BizTalk). These DLL:s are then packed in to one single deployment package also containing a single XML file that keeps track of the dependencies and the order of witch they have to be deployed in BizTalk (the most depending schemas first and so on). The XML file also contains other meta data information such as ports the artifacts use etc.

This package is then loaded into another part of the application were it’s possible to point out the different servers one like to deploy to. This view then shows information about what has to be done on the server to make it possible to deploy without conflicts (One might have running Orchestration or suspended messages for example that has to be stopped or terminated.). At this stage we also check the naming of the different artifacts. These have to comply with the naming conventions that are configured in the config file of the applications (a warning is shown if the artifact doesn’t validate towards these).

When no warnings (its possible to override a warning) or conflicts are shown one can deploy. We then move the deploy scripts to the servers and use the MSBuild tasks in the SBF to deploy everything for us. We really saved some serious time with this approach and even if it took us a while to get everything working it’s been well worth it!

We have a huge feature list for the next phase of the tool and are planing to and support for BizUnit tests in the DLL:s (so that one gets a warning when deploying a Orchestration without etc). We also like to add more meta data about the port and make it possible to configure new ports as a part of the deployment process. Etc, etc …

Feel free to comment or write we line for further information about this approach. It would also be interesting to hear about other approaches for deployment and what kind of features these solutions have.

The PowerToys package contains several small XP add-ins. For example the Virtual Desktop Manager! 

As the image above show the manager makes it possible to have four separate desktops running at one – nice! A few configurations tips that I found useful was turn of the shared desktops option and to set the shortcut for showing the desktops to alt-§ (that’s close to the alt-tab option).

An other cool application in the PowerToys package is the Alt-Tab Replacement that shows a small preview of the different windows one is multi tasking between (as shown in the image above).

Validation

I’ll work my way down the list when I have some more time. However I found this, seems like it could have saved my a lot if I’d known about earlier. It’s a package with validation controls for ASP.NET for every situation. Looks really good.

Date

This seems like another usefull resource. Loads and loads of different date controls. I’ll have to remeber this site the next time I’ll nedd one.

It saved me a lot of time. It even generates .NET code.