Introduction to BizTalk 2004
This paper takes a mainly functional look at BizTalk. However, some hardware and software requirements are specified. It is meant for those who know little or nothing about BizTalk and its uses, but assumes some knowledge about the problems that are inherent to application (and especially data) integration. It does not purport to champion the product in any way. As a matter of fact, on several occasions it will contradict or at least qualify the standard Microsoft line, the idea behind it all being not to convert, but to inform.
The great personal revolution in computerland gave us the freedom to take automation into our own hands. And so we did: all in our own, personal way. As a result, worldwide, there must be many thousands, if not millions of tables with client data, or stock item, order or invoice data.
When all these personal computers were connected, however, could these tables easily exchange data? They could not. Not only did the hardware differ from system to system, the software also came in almost as many versions as there are systems. And that’s only technology, for on top of this we have the Babylonian confusion that we software developers created ourselves. How, for instance, do you describe a customer: customer, client, debtor, buyer or maybe even another term? And how do you uniquely identify a customer: client code, GUID, social security number? Further suggestions are all equally valid. And what do you call this attribute: CustNo, ClientNo, Number, Code? The possibilities seem almost limitless and we have probably used most of them. And right now we are not even considering the ways in which you could refer to a name, an address, a town, a phone or fax number, the name of the head purchase, and so on. As a result, we often talk about the same things, but still cannot understand each other, simply because we keep using different labels.
How, then, to communicate? Over the years, many bridges have been built, most of them for a very specific task to meet a very specific need. However, as more and more bridges were required, the need arose to make these bridges more generic, so that they could be used for more communication channels. A branch of this integration software (a type of so-called ‘middleware’) is the subject of this paper: Microsoft BizTalk.
Just a Tool
Let’s not beat about the Microsoft marketing bush: BizTalk is not an out-of-the-box product with which the end user is just a few mouse clicks away from the solution to all of his ICT business problems; BizTalk is a tool with which a professional system integrator can make data from two or more different systems (e.g. a file based system and a database) ‘talk’ to each other. On top of that, he can use the data contained in such a table to route data between two or more possible channels or maybe send an acknowledgement. Although all of this sounds (and, more or less, is) fairly straightforward, the realization is complex enough for a simple end user to bow out at this stage. BizTalk is for the system administrator at least, but better still for someone who is well grounded in the ins and outs of BizTalk development: a software developer. After initially positioning BizTalk as a pure server product, Microsoft now seems to have recognized this fact by incorporating BizTalk 2004, the latest version, into its Visual Studio.Net development suite.
A Functional Look
Every developer will probably have encountered the problem of getting two or more sets of related data to ‘talk’ to each other at least once during his career. If it was only that once, then the solution would have been a one-off problem-specific affair, but the second or third time around, the idea of a generic solution would have crept up.
What, then, would that generic approach have entailed? For starters, the incoming and outgoing data structures would have to be described, followed by the mapping between these structures, possibly with a conversion mechanism to make properties ‘fit’ each other. Lastly, there would have been the location of the incoming and outgoing data structures.
If all this has sounded logical, then the structure of BizTalk itself should come across as fairly logical as well, for BizTalk approaches the problem in much the same way.
Schema
First of all the data structure that should be integrated must be identified and described in a so-called Schema. This is always done in XML terms, although the structure itself can be anything from flat file via database table to actual XML. All this is done in the BizTalk Schema Editor. Next, in the BizTalk Mapper, incoming and outgoing data structures are mapped onto each other at attribute level in a so-called Mapping. Data conversion can be straightforward one-on-one, but also fairly complex, in which tiny applets called functoids are used to convert data from one data type to another or from one structure to another. As these functoids can themselves be .Net applications (as well as scripts), some highly complex interdependencies are possible.
Pipeline
Next comes a non-functional component of the BizTalk structure: the pipeline. As, internally, BizTalk uses XML to move its data around in, data from outside first needs to be converted into that XML, while data going out must be converted again to whatever structure the outside world expects.
Scenario
Once all the data has been described thus, we can define ports in which the source and the target locations of the data are given. This will then complete our Scenario. Once the ports are activated, the BizTalk Server is operational and any data structure (Message, in BizTalk parlance) that meets the criteria for a source will be picked up and processed into a target.
Files
BizTalk accepts files in XML, EDI (X12 en EDIFACT), SQL*Server and flat file format, from locations as diverse as a File Location, FDP or HTTP, but will always interpret them in terms of XML. Internally, XML is used for messages as well as mappings. Output is also in XML, EDI, SQL*server or flat file format. There are, however, a great many third party adapters that open up other data structures.
Orchestration
On top of this, we can use the BizTalk Orchestrator to carry out certain actions according to the data contained within a message, like routing. This may then be used as a basis for Human Workflow Management, but this will not be discussed in this paper.
Pros and Cons
The BizTalk messaging system works asynchronously: an incoming message is placed in a queue, which is then read as soon as the BizTalk Server has the resources available to process it. Is the BizTalk Server down, the message will just wait, so in a sense there is a guaranteed delivery. A message with an accepted name that does not meet structural requirements (as laid down in the Schema Editor), is still picked up, but retained in the BizTalk internal database, upon which the event log is fed with an appropriate error message. As this event log is a passive medium rather than an active one and the appropriateness of the error message still leaves lots to be desired for, this is one of the weak points of BizTalk. Another weak point of BizTalk is its dependency on other software (SQL*Server, MS Exchange), which often renders it incapable of functioning with the reasons beyond its control. The error messages, finally, are often highly cryptic and there for not much use beyond the conclusion that ‘something’ has gone wrong.
Hardware and Software Requirements
BizTalk Server 2004 alone is not enough. First of all, Windows 2003 Server is required (Windows 2000 Professional could suffice, but only for a very limited number of features), but on top of this the following software must be installed before BizTalk 2004 can be installed:
· .Net Framework (should come as part and parcel of Windows 2003)
· Internet Information Server
· SQL*Server 2000
· Visual Studio (for development)
BizTalk itself does not make any high hardware demands. The software that is a prerequisite for BizTalk 2004 to function at all makes the demands. In general, any machine that is suitable for Windows 2003 Server is suitable for BizTalk 2004.
Uses
B2B – Business to Business Commerce
Microsoft thinks highly of the B2B possibilities that BizTalk has to offer, but for Europe this may well not be the case. Using BizTalk, it would indeed be very simple to automate an order sequence from stock taking via order and invoice to payment, but this requires all participants to
· have a BizTalk Server system up and running,
· always place orders with the same business partner,
· have no objection to giving the business partner (indirect) access to its database and
· have no objection to sharing IT expertise
Now, get a European Head Purchase and ask if he wants his system to order new supplies without any human intervention. No matter if, using his own historical details, you could prove that he always orders a particular item with the same supplier, his reaction to your audacious automational suggestion will almost invariably be “I’ll be the judge of that”. In short: European resistance to B2B is not much a matter of technology as it is one of culture and we had better learn to live with that instead of trying to toe the Microsoft US line. For there is a much more interesting and, in Europe, much more rewarding field waiting: Enterprise Application Integration.
EAI – Enterprise Application Integration
The other BizTalk stomping ground is a more probable use in the Old World. Over the years, many long-established companies have grown “hysterically” through mergers and takeovers, while for different reasons the various subsidiaries retained their own systems. These different systems make it very difficult to develop one company-wide system and hence a company-wide view. Connecting these systems with middleware like BizTalk makes this company-wide view possible.
A Few Common Misconceptions
There are some common misconceptions about BizTalk, which need to be addressed before they cause problems.
“BizTalk is a Workflow Management Tool”
Microsoft still do not have a Workflow Management tool and what with all the hypes around that one, this omission must hurt all the way to Redmond. So, over the past few years we have seen Sharepoint Portal Server being touted as a WfM tool. People pretty soon got wise about that one, though: only if your WfM is concerned with documents does the epithet WfM make any sense.
Next, BizTalk was suggested. Granted: there is something to say for this: at heart, WfM is really nothing more than a stringing together of various (heterogeneous) processes, covered by any number of applications, by means of data mappings. Now, this stringing together of heterogeneous applications by means of mappings is exactly what BizTalk does. The glee of the Microsoft marketing department is in some ways justified. However, this stringing together is all that BizTalk does and WfM, as anyone who is worth his salt in this area will have screamed by now, goes much further. If we put it in hardware terms: if a WfM system is seen as a network, then BizTalk is only the wires between the servers, clients, printers and such. The rest you wilol have to come up with yourself. Conclusion: BizTalk can be a tool with which you may create WfM solutions, but it is not such a tool in itself
“BizTalk 2004 is just an upgrade of BizTalk 2002”
The current BizTalk incarnation is version 2004 and it is a rather radical departure from the previous versions, 2000 and 2002. So much so, that a BizTalk 2002 scenario cannot be converted to its BizTalk 2004 counterpart. Any upgrade will mainly consist of rebuilding the entire scenario.
A closer look at the differences will make this quite understandable:
· Technically: BizTalk 2004 is wholly .Net based (indeed, BizTalk itself was built using C#.Net, making it the largest single-assembly .Net app to date), whereas BizTalk 2000 and 2002 were COM based. Just forget the old slogan about BizTalk 2002 being part of the .Net family of servers: that was just sales talk. Consequently, a 2004 scenario can be subdivided into an assembly and a configured part, whereas 2000/2002 was wholly configurable.
· Structurally, BizTalk 2004 tackles the whole mapping process very differently:
Figure 1: BizTalk 2004
All components can be created and/or configured using Visual Studio.Net. Now compare this to BizTalk 2000/2002.
Figure 2: BizTalk 2000/2002
Components are created in BizTalk Editor (schemas), BizTalk Mapper (maps), BizTalk Messaging Manager (document definitions, envelopes, organizations, channels and ports) and BizTalk Service Administration (Receive Function).
From these two figures, it can be concluded that 2004 is somewhat easier, more logical that 2000/2002, although die-hard 2000/2002 developers will doubtlessly beg to disagree with that.
· Usage – BizTalk 2004 is part and parcel of Visual Studio.Net, with which Microsoft has more or less admitted that BizTalk is a development tool rather than an out-of-the-box server product.
Prices and Licences
There are a number of cheaper versions available, but the BizTalk version that you will usually need is the Enterprise, which will set you back some 20 (Euro)grand per CPU. Add to this the price of an SQL*Server 2000 and Visual Studio.Net for development and the final tag may seem quite hefty. However, compared to other middleware programs such as Cloverleaf, which comes at a higher price, you are still in for a bargain if you consider the fact that beyond the initial license, there are no other (annual) costs involved and developing in BizTalk 2004 has a fairly steep learning curve.
Conclusion
This being the age of software integration, it should come as no surprise that marketing savvy Microsoft has come up with an excellent tool for doing exactly that. Moreover, BizTalk neatly integrates with Microsoft’s own development tools (from C# to InfoPath) on the one hand and its other server products like SPS or the Host Integration Server on the other. What makes BizTalk stand out from the competition, though, is its relatively low price tag and its well constructed, almost natural approach to the whole problem, making it quite easy to master. This in turn makes for a very interesting ROI for the developer and a tantalizingly low TCO for the user.