Archive for the ‘.NET’ Category
Posted by Richard »
2 Comments »As all of you know the number one time consuming task in BizTalk is deployment. How many times have you worked your way through the steps below (and even more interesting – how much time have you spent on them …)
- Build
- Create application
- Deploy schemas
- Deploy transformations
- Deploy orchestration
- Deploy components
- Deploy pipelines
- Deploy web services
- Create the external databases
- Change config settings
- GAC libraries
- Apply bindings on applications
- Bounce the host instances
- Send test messages
- Etc, etc …
Not only is this time consuming it’s also drop dead boring and therefore also very prone – small mistakes that takes ages to find and fix.
The good news is however that the steps are quite easy to script. We use a combination of a couple of different open-source MsBuild libraries (like this and this) and have created our own little build framework. There is however the BizTalk Deployment Framework by Scott Colescott and Thomas F. Abraham that looks great and is very similar to what we have (ok, ok, it’s a bit more polished …).
Binding files problem
Keeping the binding files in a source control system is of course a super-important part of the whole build concept. If something goes wrong and you need to roll back, or even rebuild the whole solution, having the right version of the binding file is critical.
A problem is however that if someone has done changes to the configuration via the administration console and missed to export these binding to source control we’ll deploy an old version of the binding when redeploying. This can be a huge problem when for example addresses etc have changed on ports and we redeploy old configurations.
“- What!? If fixed that configuration issue in production last week and now it back …”
So how can we reassure that the binding file is up-to-date when deploying?
One solution is to do and export of the current binding file and compare that to one we’re about to deploy in a “pre-deploy”-step using a custom MsBuild target.
Custom build task
Custom build task in MsBuild are easy, a good explanation of how to write one can be found here. The custom task below does the following.
- Require a path to the binding file being deployed.
- Require a path to the old deployed binding file to compare against.
- Using a regular expression to strip out the time stamp in the files as this is the time the file was exported and that will otherwise differ between the files.
- Compare the content of the files and return a boolean saying if they are equal or not.
public class CompareBindingFiles : Task
{
string _bindingFileToInstallPath;
string _bindingFileDeployedPath;
bool _value = false;
[Required]
public string BindingFileToInstallPath
{
get { return _bindingFileToInstallPath; }
set { _bindingFileToInstallPath = value; }
}
[Required]
public string BindingFileDeployedPath
{
get { return _bindingFileDeployedPath; }
set { _bindingFileDeployedPath = value; }
}
[Output]
public bool Value
{
get { return _value; }
set { _value = value; }
}
public override bool Execute()
{
_value = GetStrippedXmlContent(_bindingFileDeployedPath).Equals(GetStrippedXmlContent(_bindingFileToInstallPath));
return true; //successful
}
private string GetStrippedXmlContent(string path)
{
StreamReader reader = new StreamReader(path);
string content = reader.ReadToEnd();
Regex pattern = new Regex("<Timestamp>.*</Timestamp>");
return pattern.Replace(content, string.Empty);
}
}
Using the build task in MsBuild
After compiling the task above when have to reference the dll in <UsingTask> element like below.
<UsingTask AssemblyFile="My.Shared.MSBuildTasks.dll" TaskName="My.Shared.MSBuildTasks.CompareBindingFiles"/>
We can then do the following in out build script!
<!--
This target will export the current binding file, save as a temporary biding file and use a custom target to compare the exported file against the one we’re about to deploy.
A boolean value will be returned as IsValidBindingFile telling us if they are equal of not.
-->
<Target Name="IsValidBindingFile" Condition="$(ApplicationExists)=='True'">
<Message Text="Comparing binding file to the one deployed"/>
<Exec Command='BTSTask ExportBindings /ApplicationName:$(ApplicationName) "/Destination:Temp_$(BindingFile)"'/>
<CompareBindingFiles BindingFileToInstallPath="$(BindingFile)"
BindingFileDeployedPath="Temp_$(BindingFile)">
<Output TaskParameter="Value" PropertyName="IsValidBindingFile" />
</CompareBindingFiles>
<Message Text="Binding files is equal: $(IsValidBindingFile)" />
</Target>
<!--
This pre-build step runs only if the application exists from before. If so it will check if the binding file we try to deploy is equal to one deployed. If not this step will break the build.
-->
<Target Name="PreBuild" Condition="$(ApplicationExists)=='True'" DependsOnTargets="ApplicationExists;IsValidBindingFile">
<!--We'll break the build if the deployed binding files doesn't match the one being deployed-->
<Error Condition="$(IsValidBindingFile) == 'False'" Text="Binding files is not equal to deployed" />
<!--All other pre-build steps goes here-->
</Target>
So we now break the build if the binding file being deployed aren’t up-to-date!
This is far from rocket science but can potentially save you from making some stupid mistakes.
Posted by Richard »
Add Comment »Traditional SOAP Web Services might feel kind of old as more and more people move over to WCF. But a lot of integration projects still relay heavily on old fashion SOAP Web Services.
Using BizTalk generated Web Services however has a few issues and one needs to add a few extra steps and procedures to make them effective and easy to work with. This post aims to collect, link and discuss all those issues and solutions.
1. Building and deploying
BizTalk Server includes the “BizTalk Web Services Publishing Wizard” tool that integrates with Visual Studio. This is basically a tool to generate a DSL based script for generating web services.
The wizard collects information about what schema or a orchestration to expose, namespaces, names of service and method, where on IIS to publish the service etc, etc.
The output of the tool is then a xml file (a “WebServiceDescription” file) that has all the collected values in it.
As a final step Visual Studio uses the newly created description file as input to a class called WebServiceBuilder in the .NET Framework. It is this class that is responsible for interpreting the description, configure and generate the service.
A common procedure is to use the wizard and click thru it and input values for every single deployment. This is of course slow, error prone and stupid.
What is much better is to take a copy of the generated “WebServiceDescription” file, save it to your source control system and then programmatically pass the file to the WebServiceBuilder class as part of your deployment process. Possible changes to the details of the service can then be done directly in the description file.
I have seen this approach save lots of time and problems related to deployment.
2. Fixing namespace
Another annoying problem (I’d would actually go so far as calling it a bug) is the problem with the bodyTypeAssemblyQualifiedName value in the generated Web Service class.
This causes BizTalk to skip looking up the actual message type for the incoming message. As no message type exists for the message is in BizTalk mapping and routing on message types etc will fail. It is a know problem and there are solutions to it. I would also recommend take the extra time need to make this small “post process step” be part of your deployment process (see how here).
3. Pre-compiling
By default the “WebServiceBuilder” class generates a web service without pre-compiling it. Usually this is not a problem. But in some cases were one really on the web service being online and give a quick response-message the performance problems in this approach can be a huge problem.
When generating the web service without pre-compiling it IIS has to compile the service and then keep the compiled service in memory. That means that when IIS releases the service from memory there is a latency before IIS re-compiled the service, loaded it into memory and executed it. This is a known problem and I have seen this “slow first hit” issue been a frequent question the different forums.
The solution is to use the aspnet_compiler.exe tool and pre-compile the service and the use those pre-compiled dlls as the service. IIS then never has to recompile it and will serve initial hits much faster.
Here is an example of how we defined a target to do this as part of our deployment process using MSBuild.
- Pre-compile the service into a new folder
- Clean out the “old” not compile service folder.
- Copy the pre-complied service into the service folder
<Target Name="CompileWeb">
<Message Text="Uses aspnet compiler to compile the service into a new folder. Then copies the compiled content back into its original place" />
<AspNetCompiler
PhysicalPath="$(WebSiteServicePath)InitiateProjectService\"
VirtualPath="/WebServiceName"
TargetPath="$(WebSiteServicePath)$(WebServiceName)Compiled\"
Force="true" />
<Folder.CleanFolder Path="$(WebSiteServicePath)$(WebServiceName)\"/>
<Folder.CopyFolder
Source="$(WebSiteServicePath)$(WebServiceName)Compiled\"
Destination="$(WebSiteServicePath)$(WebServiceName)\" />
</Target>
Posted by Richard »
Add Comment »This is a very specific problem but I’m sure some of you stumbled over it. When disassembling a XML message in a SOAP port BizTalk can’t read the message type. This causes problems when for example trying to handle an envelope message and split it to smaller independent messages in the port. It’s a known problem discussed here and here (you also find information about it in the BizTalk Developer’s Troubleshooting Guide) and the solution is to make a small change in the generated web service class. Below is a small part of he generated class.
//[cut for clarity] ...
Microsoft.BizTalk.WebServices.ServerProxy.ParamInfo[] outParamInfos = null;
string bodyTypeAssemblyQualifiedName = "XXX.NO.XI.CustomerPayment.Schemas.r1.CustomerPayments_v01, XXX.NO.XI.CustomerPaym" +
"ent.Schemas.r1, Version=1.0.0.0, Culture=neutral, PublicKeyToken=ac564f277cd4488" +
"e";
// BizTalk invocation
this.Invoke("SaveCustomerPayment", invokeParams, inParamInfos, outParamInfos, 0, bodyTypeAssemblyQualifiedName, inHeaders, inoutHeaders, out inoutHeaderResponses, out outHeaderResponses, null, null, null, out unknownHeaderResponses, true, false);
}
}
}
Basically the problem is that the generated code puts the wrong DocumentSpecName property in the message context. I’ll not dicusses the problem in detail here but Saravana Kumar does thorough dissection of the problem in his post on it.
The solution is to update the bodyTypeAssemblyQualifiedName to set a null value. That will cause the XmlDiassasemler to work as we’re used to and expect.
If the value null is passed instead of bodyTypeAssemblyQualifiedName, SOAP adapter won’t add the DocumentSpecName property to the context. Now, when we configure our auto-generated SOAPReceiveLocation to use XmlReceive pipeline, the XmlDisassembler component inside XmlReceive will go through the process of automatic dynamic schema resolution mechanism, pick up the correct schema and promotes all the required properties (distinguished and promoted) defined in the schema and it also promotes the MessageType property.
From: http://www.digitaldeposit.net/saravana/post/2007/08/17/SOAP-Adapter-and-BizTalk-Web-Publishing-Wizard-things-you-need-to-know.aspx
//[cut for clarity] ...
Microsoft.BizTalk.WebServices.ServerProxy.ParamInfo[] outParamInfos = null;
string bodyTypeAssemblyQualifiedName = null;
// BizTalk invocation
this.Invoke("SaveCustomerPayment", invokeParams, inParamInfos, outParamInfos, 0, bodyTypeAssemblyQualifiedName, inHeaders, inoutHeaders, out inoutHeaderResponses, out outHeaderResponses, null, null, null, out unknownHeaderResponses, true, false);
}
}
}
But if you have an automated deployment process you probably use MSBuild to generate your Web Services. Then is soon becomes very annoying to remember to update the .cs-file again and again for every deployment. So how can we script that update?
First we need to find a regular expression to find the right values. With some help from StackOverflow (let’s face it, there are some crazy regular expressions skills out there …) I ended up on the following.
(?<=string\sbodyTypeAssemblyQualifiedName\s=\s)(?s:[^;]*)(?=;)
If you’re not a RegEx ninja the line above does something like this:
- After the string “string bodyTypeAssemblyQualifiedName = ”
- turn on single line (treat “\r\n” as any other character) ( this is what “(?s: )” does)
- match every character that is not a semicolon
- until a single semicolon is reached.
Then I used a task from the SDC Task library (you probably already use this if you’re using MSBuild and BizTalk). More specially we use the File.Replace
<Target Name="FixSOAPServiceCode">
<File.Replace
Path="$(WebSiteServicePath)CustomerPaymentService\App_Code\CustomerPaymentService.asmx.cs"
Force="true"
NewValue="null"
RegularExpression="(?<=string\sbodyTypeAssemblyQualifiedName\s=\s)(?s:[^;]*)(?=;)">
</File.Replace>
</Target>
Now this task is part of the build script and called right after the tasks that generates the web service. This saves me a lot of manual work and potential errors!
Posted by Richard »
1 Comment »
It seems like there more build script one writes, the more often one wants to run them and it’s always a bit annoying (and time consuming) having to leave Visual Studio and start MSBuild from the command line. Brennan Stehling has a cool solution to that problem here were he sets up MSBuild as an external tool and runs it.
One problem for us was the we had our solution files in one place on the file system and our build files in a totally different place. The solution was to add the build file for the current solution as a Solution Folder (as shown in the figure below) and then set MSBuild to use $(ItemDir) as its Initial Directory. That will kick of MSBuild from the directory that the current selected Solution Folder points to and in our case that’s were the XXX.Build.Article.proj file exists.
Posted by Richard »
Add Comment »Yesterday I presented the MasterData Management using BizTalk 2006 R2 talk (I’ll soon have a post out with the presentation in English) I recently held at Developer Summit at the local .NET user group in Karlstad (KNUG).
KNUG is a new .NET user group that I actually helped start a couple of months ago. This meeting was the second meeting for the group. The meeting was attended by about 20 persons and we had two presentations on the agenda. Besides my own Thomas Heder showed the group some LINQ and how he and his colleagues uses LINQPad to develop and test there queries.
We also discussed future subjects, possible speakers and moving information on the group over to a Community Server driven site.
Does anyone have any experience on Community Server and how the feature set matches those need for running a user group (managing users, blogs, email lists, calendar etc)?
Posted by Richard »
1 Comment »I’ve blogged about the Visual Studio extension XPathmania before. It’s a very simple little tool that lets you write and test XPath inside of Visual Studio 2005. No big deal if you already have tools like XMLSpy or XML Notepad but still. I like not having to start another application, opening the XML document I’m working with and so on. Doing stuff inside of Visual Studio just feels right and saves some time anyway.
One of the last episodes of dnrTV hosted Dom Demsak (Don XML), the creator of XPathmania. The show is 30% about XPathmania and 70% about XPath and XPath syntax in general. It’s kind of basic XPath but I think it can be useful for someone who feels they haven’t got full control of the language.
During the show they touch on XML namespaces and XML default namespaces. However they don’t really explain the difference between them and how it effects the document. Something that’s sad as I feel that XML namespaces (and especially default namespaces) is something that most people haven’t fully understood.
Anyway, if you got some spare time watch it or forward it someone you think should watch it. 
Posted by Richard »
Add Comment »I’ve just found out that I get to go to TechEd in November! I haven’t been there before. I’ve been to a lot of conferences but not the Microsoft conference. Do I have to say that I’m looking forward to it?
If you’re going and feel like meeting up for a BizTalk lunch, dinner or whatever don’t hesitate to drop me an e-mail.
Posted by Richard »
Add Comment »Ever seen this alert when trying to debug a project using Visual Studio 2005?
The following module was build with optimizations enabled or without information:
C:\Windows\assembly\GAC_MSIL\ …
To debug this module, change its build configuration to debug mode. TO suppress this message, disable the ‘Warn if no user code on launch’ debugger option.
Basically this means that Visual Studio can’t find the debug file (the .pdb file) at the same location as the dll is loaded from. Reading the error message tells us that the dll been loaded from the GAC and there aren’t any pdb files there! Just make sure to uninstall the dll from the GAC and Visual Studio should be able to load it from your Debug folder and you’ll probably be good to go.
Posted by Richard »
1 Comment »As I don’t use regular expressions that often I always forget the syntax. So I thought I just put a basic replace pattern up here.
This method takes a schema, finds all places where is says schemaLocation=”whatever” in a text and changes this to schemaLocation=”whatever.xsd“ and then returns the schema.
private XmlSchema FixSchemaLocation(XmlSchema schema)
{
System.Text.RegularExpressions.Regex locationReplacePattern = new System.Text.RegularExpressions.Regex("schemaLocation=\"(?<location>.*?)\"");
string locationReplaceValue = "schemaLocation=\"${location}.xsd\"";
//Puts .xsd after the schemaLocation. We need this find the imported schemas
StringWriter sw = new StringWriter();
schema.Write(sw);
XmlSchema formatedSchema = XmlSchema.Read(new StringReader(locationReplacePattern.Replace(sw.ToString(), locationReplaceValue)),null);
return formatedSchema;
}
Posted by Richard »
1 Comment »UPDATE: This does not apply to BizTalk … I’ve made an update post here. Sorry.
We’re working with a lot of code libraries that we use in different parts of our BizTalk solutions. As the are used on several servers and by loads of different “BizTalk parts” (both in orchestrations and maps) it’s important that we always keep the version number of the assemblies up to date. That means that every little change should increase the current version number. But as they are used in so many places people have started to skip this step as they thought they had to compile all parts that should use the new code (say it’s a bug fix and you’d like all “using parts” of the assembly to load the updated version). This is where GAC loading policy comes to the resource!
First we have to understand that every .NET assembly is identified using four characteristics:
- Assembly name
- Major or minor version
- Public key token
- Culture
Then we need to know that the first version number in for example version 1.1.2.1 is the major version. The second is the minor version and the third and fourth are build, revision version number. So this means that if you have 1.1.2.1 installed and make a minor change the easiest way to use the new assembly is to change the one of the build or revision numbers (the third or fourth number). Then the CLR will load the new assembly without any other changes!
But sometimes we have to change the minor or major version - and we still don’t have to recompile a thing! We can use a publisher policy file. This is an example of such a file defined for version 1.0.0.0 moving to 2.0.0.0.
<configuration>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<dependentAssembly>
<assemblyIdentity name="BaseHelper" publicKeyToken="18517ea673f8584b" culture="neutral" />
<bindingRedirect oldVersion="1.0.0.0" newVersion="2.0.0.0"/>
</dependentAssembly>
</assemblyBinding>
</runtime>
</configuration>
This kb article describes what to do next:
- Change the version and recompile. The first step is to create the new version of your component. After you’ve done that, you will need to modify the version number in the AssemblyInfo file for your component.
Create the publisher policy file. Create the publisher policy file for the assembly using the format shown above.
Use Assembly Linker (Al.exe) to create the publisher policy assembly. The Assembly Linker is included with the .NET Framework SDK. To create the publisher policy assembly that redirects a binding from version 1.0 of Website.dll to version 2.0 using a publisher policy file called website.config, run the following command:
al /link:BaseHelper.config /out:policy.1.0.BaseHelper.dll /keyfile:c:\keyfile.snk
This command will create a new assembly called policy.1.0.BaseHelper.dll. This naming convention is important, as indicated in the “What Is a Publisher Policy Assembly?” section.
- Install the publisher policy assembly into the Global Assembly Cache. The publisher policy assembly is installed into the GAC. It will be used by the .NET runtime when any application attempts to bind to version 1.0 of the BaseHelper.dll, and it will force the application to bind to the new version automatically.
- Install the new version into the Global Assembly Cache. Install the new version of the component into the GAC. After the new version has been installed, the old version can be safely removed.
So no more excuses for not updating the version number!
