BizTalk Cloud Migration – Part 1
I recently had an interesting request from one of our customers. As part of their new cloud adoption strategy, the organization was planning to migrate their on-premises systems to Azure, including refactoring their current BizTalk integration layer into Azure workloads. The target was to eventually leverage cloud-native solutions.
This blog post will be the first part of two, in this first part, I will describe the proposed high-level approach for migrating the on-premises integration solution to reliably fulfill the organization’s cloud adoption strategy.
I will follow that with a second part where I will dive deep into the design details of the Azure Integration Services and I will highlight on the key design decisions.
Organization Existing Landscape
BizTalk Server comprised the integration layer for real-time, stateless, synchronous integrations with disparate on-premises and cloud-hosted downstream systems, in a hub and spoke style.
The downstream systems comprised of the organization’s bespoke systems, and Commercial Off-The-Shelf (COTS) systems. Moreover, these systems spanned across cloud and on-premises platforms.
BizTalk primarily handled protocol mediation, business process automation, message transformation & translation among the integrated systems, as well as the standardization of the published service interfaces for the client applications.
Why the need to Migrate?
At first, I worked closely with the customer to understand the motives behind the migration to make sure that they were migrating for the right reasons. The main reasons were the following:
- Cost savings: The consumption-based pricing model was a great fit for the organization’s solution and traffic
- Faster Time to Market: Increasing competition required IT to keep up with the business with a faster cadence
- Offload local data centers management: It was becoming an increasing hassle to manage on-premises data centers with different applications, operating systems, and the underlying infrastructure
- Cloud rapid elasticity: The organization’s traffic throughput was somewhat hard to predict, on-demand scalability was important
- Higher Availability: Designed correctly, Azure provided a higher availability, which meant lower service disruptions
- Azure Cloud Offerings: The wide range of cloud offerings has the potential for providing a competitive advantage to the organization
- Organization Team Readiness: The organization’s IT team have already been training and working with Azure workloads
Non-functional requirements
The customer stressed on the following quality attributes: integration solution latency overhead to be kept to a minimum, the incoming requests throughput is expected to fluctuate at any time, and security must be addressed with the proper authentication scheme, transport-level, and the necessary protection against network and application level exploits.
The Migration Stages
In order to achieve the cloud-native and minimize the impact of breaking changes for the existing upstream and downstream systems, I recommended the following (4) stages to reach the target cloud-native architecture. Needless to say at each stage, comprehensive end-to-end testing will be conducted.
- Stage 1: Cloud Integration Foundation
- Stage 2: Re-Host BizTalk Published Services
- Stage 3: Completed Hybrid Model
- Stage 4: Cloud Native Model
Let’s start by going through each stage, and have look more closely to understand their purpose.
Stage 1: Cloud Integration Foundation
In this stage, I was laying the foundation for the cloud-hosted integration architecture with a Hybrid integration model. Newly developed APIs will leverage Azure Integration Services, while existing APIs will still be provided through BizTalk Server.
In this stage, client applications will be required to consume two separate service endpoints, one the for cloud-hosted integrations and the other for the on-premises BizTalk published integrations. Moreover, Azure Integration Services could initially leverage any existing BizTalk published integrations, if needed.
Stage 2: Re-Host BizTalk Published Services
The second stage, will take the hybrid model a step further. Additional new integration points will continue to be developed directly using Azure Integration Services, while existing integration points previously published through BizTalk Server will be wrapped and re-published through Azure Integration Services.
During this stage, existing client applications will gradually on-board to the new Azure integrations services endpoints, and by the end of this stage, all client applications should have switched to the new Azure integrations services endpoint and have already stopped using the on-premises integrations service endpoint.
The idea was that I wanted to make sure that the APIs are designed top-down; starting with the client-facing business services contracts while materializing the implementation that serve these contracts, not the other way round.
In other words, this will make sure that the underlying implementation does not seep out and influence these service contracts and that the APIs design is actually driven by the service consumer business needs.
Further, the underlying BizTalk published services will still be maintained during this stage, to be leveraged by Azure Integration Services to temporarily fulfill the incoming requests.
Stage 3: Completed Hybrid Model
The third stage, a step closer towards the target architecture, finalizes the hybrid integration model. Azure Integration Services will be refactored to directly integrate with the downstream services, instead of leveraging BizTalk published services.
The great part is that this integration layer refactoring will be invisible to the client applications, which are shielded by the service contracts from stage (2). At the end of this stage, BizTalk Server will be retired.
Stage 4: Cloud Native Model
This is the target stage that completely fulfills the organization’s cloud-adoption strategy, in which all backend systems become hosted on the cloud, Azure Integration Services will integrate these cloud-hosted systems in a decoupled manner using its wide range of connectors, while promoting service re-usability across the organization applications.
Slowly but Surely
The intermediate stages are introduced to gradually and reliably conduct the cloud migration for the BizTalk solution. This approach carefully isolates the change scope impact for limited set of systems one step at a time, and avoids introducing major changes to all systems at once.
Let’s inspect AIS solution closely…
In the second part, I will expand on the design details of the Azure Integration Services block, and the rationale behind some of the key design decisions.