Microsoft Devices & Studios: Using Windows Azure Biztalk Services for Supply Chain Orders

Microsoft Devices & Studios: Using Windows Azure BizTalk Services for Supply Chain Orders

Audience

The intended audience for this white paper includes anyone who develops or uses the application. The primary audience includes:

·  Business representatives, including product management

·  Development team, including development manager

·  Test team, including test manager

·  Application support

·  Program management

Prerequisites

To benefit from this white paper, you should have knowledge in the following areas:

·  Enterprise Application Integration (EAI), business-to-business (B2B), and application-to-application (A2A) concepts and technologies.

·  Microsoft Windows Azure BizTalk Services (WABS) including account privileges, administration, and configuration.

·  Microsoft BizTalk Server 2013 including setup, administration, and configuration.

·  Microsoft Visual Studio 2012 including creating and deploying BizTalk Services (WABS) projects to Windows Azure.

·  B2B formats including Electronic Data Interchange (EDI) and OAGIS (Open Applications Group Integration Specification).

Background

The Manufacturing, Supply Chain, and Information Services (MSCIS) group is a Global Supply Chain Management group at Microsoft. Every year, Microsoft launches new products. MSCIS is responsible for bringing these new products to market. To support these products, new partners are added to the Supply Chain, including Supplier, Manufacturer, Distributor, Retailer, Service Center, Carrier, and so on.

The number of partner transactions increases yearly. This increase brings challenges that include volume, capacity, performance, scalability, and cost.

One of the core applications is Order Management. During peak sales, the Order Management application processes 25,000 transactions per hour. In the future, the Order Management application is expected to process 50,000–70,000 transactions per hour. This number will grow in the coming years.

In the on-premise BizTalk hub, there are over 25 applications across four BizTalk Groups using the Enterprise Edition. There are six main applications that interact directly with different business partners. Each application contains multiple transactions. During high load, there are issues related to throughput. While the Supply Chain BizTalk hub processes more transactions (increased volume), the end system or partner system is not able to keep up.

A requirement of the solution is the ability to scale up for peak sales and scale down for normal sales.

Opportunity

The current challenge is to prepare the business-to-business (B2B) systems for the holiday season. The requirement is to provide flawless execution from end-to-end. Teams meet months in advance to evaluate the launch volumes and perform system capacity planning for infrastructure scale-up/scale-out options. These options lead to increased deployment cycles and infrastructure cost.

As part of Technology Adoption Program (TAP), MSCIS is exploring Platform as a Service (PaaS) using Windows Azure BizTalk Services (WABS). WABS provides key capabilities that can solve the scale situation. MSCIS created a Proof of Concept (POC) involving a hybrid solution that uses Windows Azure BizTalk Services (WABS) and BizTalk Server 2013.

MSCIS focused on the Order Management application. The application receives purchase orders from upstream systems, validates the messages, and then posts the messages to an internal LOB SAP system. The purchase orders are OAGIS XML from some partners and EDI AS2 messages for other partners.

As part of the POC, the focus is on two types of transactions:

·  Synchronous OAGIS (Open Applications Group Integration Specification) to process purchase orders

·  Asynchronous EDI 850 over AS2 to process purchase orders

Most of the transactions in the BizTalk hub use one of these communication patterns with partners.

Proposed Solutions

There are two proposed solutions based on a hybrid architecture that use Windows Azure BizTalk Services and on-premise BizTalk Server 2013. In both solutions, messages are routed to Service Bus Topics using business filters like SKU, receipt timing, partner type, status of messages, and so on. The on-premise BizTalk Server application polls the Service Bus Topics for the messages, performs the on-premise workflow activities, and sends the messages to the downstream ERP application.

The solutions differ in how communication is handled with the partners. The next sections explain the different communication methods:

Proposed Solution: Synchronous OAGIS XML Purchase Orders

Proposed Solution: Asynchronous EDI 850 over AS2 Purchase Orders

Proposed Solution: Synchronous OAGIS XML Purchase Orders

“Request-Reply”

This solution consists of the following components:

·  A BizTalk Service provisioned on Windows Azure.

·  In Visual Studio, a BizTalk Service application that consists of a XML Request-Reply Bridge, a Transform (converts the incoming message to an Acknowledgement Purchase Order message), and a Two-Way External Service Endpoint is created. The application is deployed to the Windows Azure BizTalk Service you provisioned. When the XML Request-Reply Bridge is deployed, an HTTPS Endpoint URL is automatically created.

·  A WCF web service hosted on Windows Azure.

In this design, the following occurs:

1.  A partner sends a purchase order (PO) in OAGIS format to the HTTPS Endpoint URL of the Bridge.

2.  The XML Request-Reply Bridge (ReceiveOrders) receives the purchase order message (ProcessPurchaseOrder).

3.  In the Request-side of the Bridge, the Validate stage uses a custom .NET component to validate the mandatory fields:

If validation succeeds, the message is routed to the Success Service Bus Topic. Using the SB-Messaging adapter, the on-premise BizTalk Server subscribes to the Success Service Bus Topic. In on-premise BizTalk, the message is converted into an SAP IDOC message and then sent to the on-premise SAP system.

If validation fails, the message is routed to the Failure Service Bus Topic. Using the SB-Messaging adapter, the on-premise BizTalk Server subscribes to the Failure Service Bus Topic. In on-premise BizTalk, the failed message is stored in an on-premise file location for archiving.

4.  In the Request-side of the Bridge, an XSLT Transform (ProcessPO_AckPO_Oagis) executes. This Transform creates a positive or negative Acknowledgement based on message validation that occurs within the XSLT code:

The output of the Transform is the Acknowledgement message (AckPO), which is then routed to a Two-Way External Service Endpoint (EchoServiceEndPoint):

This Two-Way External Service Endpoint uses a WCF web service with a custom .NET component to route the same AckPO message back to the Reply-side of the Bridge:

5.  The same AckPO is then returned as a response to the partner.

Explanation

The Request-side of the Bridge contains the complete flow of the message. Meaning, the Request-side receives the PO, validates the PO, sends the PO to the Service Bus Topic and is then processed by on-premise BizTalk.

The Reply-side of the Bridge is used to send the acknowledgement back to the partner. The WCF Web Service uses Echo Service binding to return the same acknowledgement PO message back to the Reply-side of the Bridge. As a result, the same acknowledgement PO is sent to the partner.

Proposed Solution: Asynchronous EDI 850 over AS2 Purchase Orders

“Fire and Forget It”

This solution consists of the following components:

·  A BizTalk Service provisioned on Windows Azure.

·  Using the BizTalk Services Portal, an AS2 Agreement is created between Microsoft and the partner. When the Agreement is created, an HTTPS Inbound URL is also created.

·  In Visual Studio, a BizTalk Service application that consists of a XML One-Way Bridge, a Transform (converts the incoming message to an OAGIS PO), and a One-Way Relay Endpoint is created. The application is deployed to the Windows Azure BizTalk Service you provisioned.

·  Using the BizTalk Services Portal, an X12 Agreement is created between Microsoft and the partner. This X12 agreement receives messages from the AS2 Agreement and sends the messages to the XML One-Way Bridge.

In this design, the following occurs:

1.  A partner sends the EDI 850 message to the HTTPS Inbound URL of the AS2 agreement.

2.  The AS2 agreement receives the 850 message, validates the partner’s AS2 settings, and sends an asynchronous MDN (a functional acknowledgement for receipt of the 850 message) to the partner.

3.  The X12 agreement receives the 850 message from the AS2 agreement. In the X12 agreement settings, a Transform converts the 850 message to the Purchase Order OAGIS standard. Then, the X12 agreement routes the message to the XML One-Way Bridge.

4.  The XML One-Way Bridge (EDI850ReceiveOrders) receives the message (ProcessPurchaseOrder).

5.  In the XML One-Way Bridge, the Validate stage uses a custom .NET component to validate the mandatory fields:

If validation succeeds, a positive 997 acknowledgement message is sent to the partner. Simultaneously, the message is routed to the Success Service Bus Topic. Using the SB-Messaging adapter, the on-premise BizTalk Server subscribes to the Success Service Bus Topic. In on-premise BizTalk, the message is converted into an SAP IDOC message and then sent to the on-premise SAP system.

If validation fails, a negative 997 acknowledgement message is sent to the partner. Simultaneously, the message is routed to the Failure Service Bus Topic. Using the SB-Messaging adapter, the on-premise BizTalk Server subscribes to the Failure Service Bus Topic. In on-premise BizTalk, the failed message is stored in an on-premise file location for archiving.

6.  In the XML One-Way Bridge, an XSLT Transform (PurchaseOrder_To_EDI997) executes. This Transform creates a positive or negative 997 Acknowledgement based on message validation that occurs within the XSLT code:

The output of the Transform is the 997 acknowledgement message, which is then routed to a One-Way Relay Endpoint (Response997To…):

The One-Way Relay Endpoint sends the 997 acknowledgement message to a Service Bus Topic:

Using the SB-Messaging adapter, the on-premise BizTalk Server subscribes to the 997 Service Bus Topic. In on-premise BizTalk, the 997 message is stored in an on-premise file location for archiving.

Explanation

The message is sent and received using an EDI agreement; specifically using the AS2 protocol and the functional acknowledgements within AS2. The Bridge contains the complete flow of the message. Meaning, the Bridge receives the PO, validates the PO, sends the PO to the Service Bus Topic and is then processed by on-premise BizTalk. The Bridge then creates a 997 acknowledgement message to send to the partner.

Result

The following key performance indicators (KPI) are used to determine success or failure:

·  Number of Orders Processed at Maximum throughput

·  Number of Orders stored in the Windows Azure Service Bus Topic

·  Percent of platform availability

·  Number of Days to scale out for peak sales

·  Cost to expand each BizTalk hub for peak sales

Synchronous OAGIS XML Purchase Orders

When testing OAGIS XML messages with one Premium BizTalk unit, around 200,000 messages are processed per hour, which is around 60 transactions per second. The Windows Azure BizTalk Services Premium edition can scale up to 8 BizTalk Units. This scale capacity equates to around 1.6 million transactions per hour.

The following table lists the KPI results:

Key Performance Indictor (KPI) / Baseline:
Current On-Premise application / Target / Hybrid Solution using WABS:
One Premium BizTalk Unit / Hybrid Solution using WABS:
8 Premium BizTalk Units
Number of Orders Processed at Maximum throughput for one Premium BizTalk unit
x=25,000 transactions per hour / 1x / 4x / 7x
192,000 messages per hour / 60x
1,500,000 messages per hour
Number of Orders stored in the Windows Azure Service Bus Topic
x= 5 hours / 1x / 3x / 32,256,000 orders for 7 days: 5KB message size
If on-premise is offline, can store 7 days of orders. / 258,048,000 orders for 7 days: 5KB message size
If on-premise is offline, can store 7 days of orders.
% platform availability / 99.9% / 99.9% / 99.9% / 99.9%
Number of days to scale hub for Peak Sales; including installing and configuring on-premise BizTalk and provisioning a BizTalk Service.
1 Hub = 6 Servers / 80 days / <45 days / <3 days / <15 minutes
Cost to expand each BizTalk hub to a 6-Server hub during Peak Sales. / $35,750 / $35,000 / <$8,000
Note Cost listed is WABS Preview Pricing. / $64,000 per month
Note Cost listed is WABS Preview Pricing.

Asynchronous EDI 850 flow over AS2 Purchase Orders

When testing EDI AS2 messages with one Premium BizTalk unit, around 60,000 messages are processed per hour, which is around 20 transactions per second. The Windows Azure BizTalk Services Premium edition can scale up to 8 BizTalk Units. This scale capacity equates to around 500,000 transactions per hour.

The following table lists the KPI results:

Key Performance Indictor (KPI) / Baseline:
Current On-Premise application / Target / Hybrid Solution using WABS:
One Premium BizTalk Unit / Hybrid Solution using WABS:
8 Premium BizTalk Units
Number of Orders Processed at Maximum throughput for one Premium BizTalk unit
x=25,000 transactions per hour / 1x / 4x / 2x
60,000 messages per hour / 20x
500,000 messages per hour
Number of Orders stored in the Windows Azure Service Bus Topic
x= 5 hours / 1x / 3x / 10,000,000 orders for 7 days: 200KB message size
If on-premise is offline, can store 7 days of orders. / 80,000,000 orders for 7 days: 200KB message size
If on-premise is offline, can store 7 days of orders.
% platform availability / 99.9% / 99.9% / 99.9% / 99.9%
Number of days to scale hub for Peak Sales; including installing and configuring on-premise BizTalk and provisioning a BizTalk Service.
1 Hub = 6 Servers / 80 days / <45 days / <3 days / <15 minutes
Cost to expand each BizTalk hub to a 6-Server hub during Peak Sales. / $35,750 / $35,000 / $8,000
Note Cost listed is WABS Preview Pricing. / $64,000 per month
Note Cost listed is WABS Preview Pricing.

Lessons Learned

Using a hybrid solution with Windows Azure BizTalk Services and BizTalk Server 2013 provided the following benefits:

Improved Delivery Time / With IaaS and PaaS BizTalk Solutions, MSCIS deploys new solutions to DEV, SIT, and UAT teams more quickly than the time required to build and configure physical servers.
Scale / As per the requirement, the Windows Azure BizTalk Service can scale out or scale down.
Using the BizTalk Services scale option, MSCIS can scale solutions to multiple BizTalk Units for peak volumes and can scale back for normal volumes. This elastic scale feature helps MSCIS plan for the complexity of new product launch cycles.
Cost / In Windows Azure BizTalk Services, you pay for what you use.
There is opportunity for MSCIS to realize huge cost savings with a hybrid POC solution to auto grow in the cloud. This reduces the demand for physical servers, hosting charges, rack space, and power consumption. It also reduces support costs incurred by System Center Operations Manager (SCOM) automated IR alerts.
Availability / Windows Azure BizTalk Services provides 99.9% availability; which is a common industry practice.
Fault Tolerance / If the on-premise server or the downstream system (like SQL or SAP) is down, there is zero impact to business because orders are queued in the Service Bus Topic.
Trading Partner Management / Windows Azure BizTalk Services provides the BizTalk Services portal for EDI Partner agreements. Using the BizTalk Services portal, it is easy to onboard EDI and AS2 partners; which reduces the onboarding lead times. The BizTalk Services portal provides consistent behavior for AS2 certificate management; which could help offload certificate management from on-premise to the cloud.
Simplified View and Management of Complex Environments / The Windows Azure Management Portal provides an in-depth view on the utilization of the deployed resources. This information helps make the right decisions about procurement, right-sizing, scaling according to usage, and performance.

Online References

Service Bus Queues, Topics, and Subscriptions (http://msdn.microsoft.com/en-us/library/windowsazure/hh367516.aspx)