How to build a Highly Available BizTalk and SQL HA Multi-Server Environment on Azure Resource Manager
In this article we will talk about how to go about building a BizTalk and SQL HA Multi-Server Environment on Azure Resource Manager. We will talk about all considerations that need to be taken, and create the best possible configuration your money can buy.
We will assume that you’ve already made the decision to deploy BizTalk and SQL HA Multi-Server Environment on Azure.
Licensing for a Highly Available BizTalk and SQL HA Multi-Server Environment on Azure
If you have your basics covered regarding licensing for on premise deployments you probably know that with Microsoft, all non-production licenses can be covered with an MSDN subscription; that is, one subscription per user. Most of my customers ask me if MSDN Professional Edition will cover it, and Professional with Dev Essentials or Enterprise Edition will be sufficient. If you look at the below table you notice that Enterprise Edition has it all, however, you can also get there with Professional with Dev Essentials, because Dev Essentials is free. The only gotcha is Office (actually Excel), but you probably have a license somewhere … otherwise you can probably just buy a license of Excel instead of getting a full MSDN Enterprise Edition. Keep in mind that MSDN is not just for downloading software; it also provides a lot more, which we’re not going to cover here.
Here’s an extract of Microsoft’s MSDN license coverage for the products we’ll need to build a Highly Available BizTalk and SQL HA Multi-Server Environment on Azure:
| Product (Updated May 25, 2017) | Visual Studio Enterprise with MSDN | Visual Studio Professional with MSDN | MSDN OS | MSDN Platforms | Visual Studio Dev Essentials |
| BizTalk Server 2016 Developer Edition | ✓ | ✓ | ✓ | ||
| Office Professional Plus 2016 | ✓ | ||||
| SQL Server 2016 Developer Edition | ✓ | ✓ | ✓ | ✓ | |
| Visual Studio Professional 2015 | ✓ | ✓ | |||
| Windows Server 2016 | ✓ | ✓ | ✓ | ✓ |
Here’s where you can download the full list
Environments
Typically, an organization that needs HA should have at least 4 environments:
- Development: These are individual developer servers where the code is written.
- Integration: This environment is where all the code is gathered, compiled and tested together. Also, commonly where the first set of binding files are created for deployment.
- Test/QA: This is the environment that looks closest to the Production environment. It is where the development team tests everything before it is released to production. It is built closest to production so it will know how to behave. This is also the environment where users test the solutions before it is released to production. Sometimes they are separate environments so both teams can work simultaneously.
- Production: This is the environment where final code is deployed and executed.
My typical deployment consists of the DEV server as single server with all software installed on the same image. I’ve seen a lot of people splitting the BizTalk and SQL role into two servers, which is completely unnecessary. The single image, even with 4 GB of RAM will work just fine for a single developer.
If my development team consists of several developers, I personally prefer to have less servers, and allow more sessions opened for multiple developers to access at the same time. For this I’ve written an article on how to enable multiple logins (Windows Server limits by default to a maximum of 2 logins at once), check it out here: Enable Multiple Login Sessions in Windows Server. The thought behind this is that having multiple servers consumes more RAM and CPU anyway, why not merge them, and build a stronger server (assuming you’re building it with some kind of virtualization technology). This helps so you don’t have to maintain multiple servers.
In this environment I don’t care too much about my bindings, ports names, and Hosts.
The same goes with the INT environment. Since the goal is to simply gather all the code from all of the developers and test them as a single integrated unit, this is the place to do it. The reason I keep it to a single server is because I am trying to save $$, while unit testing the solution. This is the environment where we try to make things as close to Production as possible, this way when we create the binding files, we can test them when deploying to Test or QA. Ideally you should try to have all the Hosts match the configuration of the Production environment.
The Test/QA environment has to look as close to Production as possible, at least from a topology and configuration perspective, however, it doesn’t have to have the same resource capability. This is the environment where the least amount of resources are required, even less than DEV and INT, because you won’t need Visual Studio, or Office, and you won’t be compiling and debugging.
Now, you might be asking, what about Stress environment, or UAT? Those are derivations of the ones we’ve mentioned before. If you follow the same train of thought you will know whether you really need them, and if you do, the topology you should use to configure them.
The production environment is the one that obviously we’re mostly concerned about. Only because this is where the actual data is going to run, and if any failures occur, this is where we will notice it the most. Anything we do to this environment needs to first be tested in the other environments, including deployment packages.
The Topology
The topology that we’re mostly concerned about, and the main topic of this article is Production. We will talk about what’s minimally needed to build a cost effective BizTalk HA environment.
Having built several environments one learns several tricks, however, since Azure constantly changes, I recommend double checking some of the limitations that I will point out in this article when you’re getting ready to plan/build.
We already know that to have HA we need two of each, in case one of them goes down, whether for maintenance, or failure. BizTalk makes use of the following main resources to function:
- Hardware (CPU, NIC, HDD, RAM, etc.)
- Windows Server Operating System
- SQL Server
- Enterprise Single Sign-On
- BizTalk Server
Of course, there are more, but these are the main that we really need to be aware.
So, for the purpose of this exercise we will build an environment with 2 SQL Server nodes, and 2 BizTalk Server nodes. Here’s the diagram describing our topology:
The diagram above describes a Clustered configuration for SQL Server, and not an AlwaysOn Availability group, why? I wrote an article about it a while ago here: BizTalk 2016 with SQL Server 2016 AlwaysOn. This is the part where you might want to see if further enhancements have been made after I published that article.
The rest is your conventional BizTalk Group (it’s similar to how a SharePoint farm works if you’re more familiar with SharePoint deployments).
Planning your SQL Server configuration
In Azure, licensing works a bit differently than in conventional on-premise licensing. Contrary to on premise licensing, in Azure you pay for servers that are Allocated (this is an actual Azure VM term). Since we will have two allocated SQL nodes running at all times, we might as well make use of them, instead of making one node passive. Therefore, we will have two SQL instances, one running on each node at all times. One will host the MessageBox database, and the other everything else.
What you will need to build the cluster
To build a SQL Cluster you will need to plan it out, especially in Azure. For example, Microsoft has made it a requirement to use Premium storage when hosting SQL Server on Azure. So, let’s go over these parameters and plan it out correctly.
Disk Configuration
The reason we need to decide on disk first is because based on what we end up with here it will determine the VM size we’ll end up selecting. VM sizes have a limitation on number of Disks you can attach to them. Traditionally in the past, planning for SQL Server involved in making many disks available to allocate dedicated IO to each disk and map them to specific tasks and services. In Azure however, it is a little different. In normal circumstances one would try to divide as many disks and allocate them to different SQL resources as possible to gain some extra bandwidth, but in Azure, Premium storage works different. Microsoft provides this table for Premium storage in their Azure Pricing page (as of 11/3/2017):
| DISK TYPES | P10 | P15 | P20 | P30 | P40 | P50 | P60 |
| Disk Size | 128 GB | 256 GB | 512 GB | 1 TB | 2 TB | 4 TB | 8 TB |
| Price per month | $19.71 | $38.02 | $73.22 | $135.17 | $259.05 | $495.57 | $946.08 |
| IOPs per disk | 500 | 1,100 | 2,300 | 5,000 | 7,500 | 7,500 | 7,500 |
| Throughput per disk | 100 MB/second | 125 MB/second | 150 MB/second | 200 MB/second | 250 MB/second | 250 MB/second | 250 MB/second |
As you can see, as the disk size goes up, so does the performance, therefore the concept of having many small disks to allocate and dedicate IO and bandwidth to a particular disk is not the same as conventional concepts. BTW, check to see if this table has been updated since I posted this article.
As we look at the table above we should also look at this table of entry level VMs that support premium storage (this is where you would start at a minimum). I like the DSv2 series because I have more data disks per VM, which gives me better flexibility. You should review this link to appropriately select your own VM size: Azure VM Sizes:
| Size | vCPU | Memory: GiB | Temp storage (SSD) GiB | Max data disks | Max cached and temp storage throughput: IOPS / MBps (cache size in GiB) | Max uncached disk throughput: IOPS / MBps | Max NICs / Expected network performance (Mbps) |
| Standard_DS1_v2 | 1 | 3.5 | 7 | 2 | 4,000 / 32 (43) | 3,200 / 48 | 2 / 750 |
| Standard_DS2_v2 | 2 | 7 | 14 | 4 | 8,000 / 64 (86) | 6,400 / 96 | 2 / 1500 |
| Standard_DS3_v2 | 4 | 14 | 28 | 8 | 16,000 / 128 (172) | 12,800 / 192 | Apr-00 |
| Standard_DS4_v2 | 8 | 28 | 56 | 16 | 32,000 / 256 (344) | 25,600 / 384 | Aug-00 |
| Standard_DS5_v2 | 16 | 56 | 112 | 32 | 64,000 / 512 (688) | 51,200 / 768 | 8 / 6000 – 12000 † |
Of course, make sure you get an updated table from the link I provided above.
The third factor is to know what storage you need and how much of it. In a conventional installation you generally require independent disks for the following, probably broken down even further:
| Purpose | Minimum Size | Description |
| DTC-INST1 | 1 GB | MSDTC dedicated disk for SQL Instance 1 |
| DTC-INST2 | 1 GB | MSDTC dedicated disk for SQL Instance 2 |
| DATA-INST1 | 20 GB | Data Disk for SQL Instance 1, MessageBox DB |
| LOG-INST1 | 20 GB | Log Disk for SQL Instance 1, MessageBox DB |
| DATA-INST2 | 129 GB | Data Disk for SQL Instance 1, all other DBs |
| LOG-INST2 | 129 GB | Log Disk for SQL Instance 1, all other DBs |
| BACKUP | 129 GB | Backup for all DBs |
| SHARED | 129 GB | Disk for archiving and other disk needs |
| TEMP-INST1 | 5 GB | Temp DB for SQL Instance 1 |
| TEMP-INST2 | 5 GB | Temp DB for SQL Instance 2 |
I pick the following sizes based on my experience and based on my client’s needs. These numbers should definitely be adjusted by you. Notice though that you can always increase the size of the disks but not decrease them. To decrease them you would have to delete and recreate, which is more work than you think, so try to avoid doing that.
How to achieve Clustered Disk?
We all know that to build a cluster you need clustered (or shared) disks, and in Azure there isn’t such Disk type. So instead we use mirroring technology that exposes the disks as clustered or shared disks.
Currently there are two options available in the market, both of them supported by Microsoft.
- SIOS DataKeeper
- Microsoft S2D
Both of these synchronize the data on the data disk and present the synchronized storage as a storage pool. I’ve used SIOS DataKeeper, which generally has worked fine. I haven’t tested Microsoft’s S2D yet, but they claim that it works just as fine.
This means that the disks listed above need to be direct attached types. Each node with its own set of disks, and be mapped and synced with its corresponding server/disk. This also tells us the number of disks we will need to allocate per server, which will consequently tell us which VM size.
Disk Performance
Finally, it is worth mentioning that we can take advantage of disk striping to gain more IOPS per disk when needed, when the size of the disk is less than 129 GB. Anything less than 129 GB will have an IOPS of 500. So, for example, if we want better IOPS for the Data Disk for the MessageBox DB, we could split it into 2 Disks of 10 GB, bringing the dedicated IOPS to 1,000 (500 per disk and striping them). As soon as we assign 129 GB, the IOPS gets bumped to 2,300.
Keep under consideration that maybe merging all the disks into – let’s say – one single drive can potentially increase the IO bandwidth to more than what you would get if you allocated them separately, but remember that with this method you cannot manage traffic priority. So, for example, it is never a good idea to combine Backup disks with Data or Log disks, because you don’t want Backup bandwidth to challenge your data or log traffic.
Consequently, you need to do a little planning and come up with what you think it is a good combination of settings that best meet your needs, as this example cannot cover everyone’s.
In my case I need a minimum of 10 data disks, which tells me that I’d better select DS3_v2 as my VM size. If I feel like this is too expensive for my budget, then I pick a smaller size, and merge the disks strategically.
So, now that we know what VM size we’re going with, we will plan out all the disks per node in the following fashion:
| Name | Size | Storage Account | Id | IOPS |
| DTC-INST1 | 1 GB | A | 1 | 500 |
| DTC-INST2 | 1 GB | B | 2 | 500 |
| DATA-INST1-1 | 10 GB | A | 3 | 500 |
| DATA-INST1-2 | 10 GB | A | 4 | 500 |
| DATA-INST2-1 | 129 GB | B | 5 | 2300 |
| LOG-INST1-1 | 10 GB | A | 6 | 500 |
| LOG-INST1-2 | 10 GB | A | 7 | 500 |
| LOG-INST2-1 | 129 GB | B | 8 | 2300 |
| ARCHIVE-INST1 | 128 GB | A | 9 | 500 |
| BACKUP-INST2 | 128 GB | B | 10 | 500 |
| TEMP-INST1-1 | 3GB | A | 11 | 500 |
| TEMP-INST1-2 | 3GB | A | 12 | 500 |
| TEMP-INST2-1 | 3GB | B | 13 | 500 |
| TEMP-INST2-2 | 3GB | B | 14 | 500 |
Additionally, I have decided to split the disks into two different Storage Accounts. Microsoft recommends to not allocate more than 20 Disks per Storage Account, so to be extra cautious I have 2, covering my 14 (7 per storage account) disks. As you can see, I have decided to take advantage of almost all of my 16 disk available spaces, to improve IO on my IO intensive Disks. Notice that I have selected 128 GB on Archive and Backup disks and not 129 GB, because I don’t want to pay higher price on Disks when the lower IO will work just fine, however, if I really need more space, it would be a different story.
Maximum IOPS per VM
Each VM type has a maximum IOPS capacity. Since our disks will be configured without caching (this is the recommended method by Microsoft and SIOS), our max IOPS of our VM size is 12,800. If we add up all of our disk’s IOPS consumption we end up with 10,600 IOPS used. This means that if all disks are bursting at the same time, we would almost maxing out the IOPS, but obviously that’s not how it works. It never happens that all disks read/write at their peak at the same time, maybe half. So, I think we’ve done well with our design so far.
NIC Allocation
When you first create your VM you should specify the number of NICs you need allocated. This can only be done at VM creation time, and it cannot be changed at a later time. I would recommend allocating a second NIC per VM, and use it for Disk synchronization, this way you let normal traffic through your default NIC. SIOS allows you to specify which NIC (by specifying the IP address), and maybe different subnet to use for syncing the data, I am not sure about Microsoft’s S2D technology.
Availability Sets
In Azure you can’t have floating IPs assigned to an imaginary DNS to be allocated to a Cluster. Instead you need to assign the IP to an Internal Load Balancer (ILB). To create an ILB and bind it to our Cluster we need to make these two nodes part of the same Availability Sets, and this can only be done at VM creation time, and once created it cannot be changed.
BizTalk Servers
The BizTalk Servers are much easier to create and configure. You don’t need the VMs to be the same size, nor clustered, instead all you need is a VM and attach it to the BizTalk group. Each time you feel like your BizTalk Servers are the bottleneck you can either increase the size of your VMs, or keep adding more VMs to the environment. I personally like to increase size because I have to maintain less servers.
For my Production environment for a Highly Available BizTalk and SQL HA Multi-Server Environment on Azure, I have decided to go with DS1_v2 because it gives me enough to run my start-transactions, and I have room to grow. I am trying to achieve HA with as little cost as possible, and since I have two servers, the load will be balanced between the two servers.
In my last several deployments I’ve taken a little extra time to cluster my BizTalk Servers as well, only because there are some adapters that don’t like it when there are two servers, and both try to do the same thing at the same time, and they overwrite each other, or do the same action twice (FTP adapter, SQL adapter and others). For this we cluster a Host, and make one of the Host Instances active and the other one passive. So, consider configuring your BizTalk servers as clustered. I don’t bother with giving it a dedicated IP, because it is simple to make one of the nodes passive, and nobody will use the IP to call into the cluster. Instead, you can create an ILB for that if you wish.
Thoughts and Recommendations for a Highly Available BizTalk and SQL HA Multi-Server Environment on Azure
At this point you know which VM size to pick, you know what to plan for each configuration, you know about how much it will cost you to host everything. Now it is time to build and test BizTalk and SQL HA Multi-Server Environment on Azure.
I always like to document what I do, whether my customer asks me to do it or not, only because I will know what I’ve done, I am organized, and I have something to go to next time I build something like it. Maybe the documentation is not as detailed as what someone else would want it to be, but at least I make sure I understand it, and maybe I can create a detailed document off of it if I am asked at a later time.
Consider scripting the work you do as much as you can. For example, the VM creation should definitely be scripted, because in most cases, when you create your VM you will notice that you forgot something and you need to delete and start again. Having scripted the VM creation will save you a ton of time.
Grouping the pair of servers into Resource Groups is always a good idea. This will help you manage the resources related to them, and it is easier to delete them if you need to, without affecting other servers. Additionally, Azure billing lets you know costs per Resource Group.
When selecting the VM, for production you need to select the SQL/BizTalk type VM if you’re not bringing your own license. For SQL Server you will notice that it will come preinstalled and configured. So, you need to uninstall SQL Server on each node, and then reinstall/configure as a cluster. This method is recommended by Microsoft.
High-Level Build Steps for creating a BizTalk and SQL HA Multi-Server Environment on Azure:
This is my to-do list when I build my environment:
- Create VMs (2 SQL + 2 BizTalk).
- Create Cloud Storage for Witness.
- Create Service Accounts and Groups
- Adjust server name, IPs, add to domain, turn off firewall, configure updates on all 4 servers.
- Add Roles and Features on all 4 servers.
- Cluster the SQL nodes.
- Configure Disk mirroring (SIOS or S2D).
- Install SQL Server (both instances).
- Make SQL ports static.
- Add DTC to each SQL Instance, bind DTC to its SQL instance respectively.
- Configure DTC security on all 4 servers.
- Configure ILB Ports for SQL, DTC, probe, etc.
- Configure DTC port.
- Install and Cluster SSO.
- Install BizTalk Servers and Configure them.
- Configure jobs.
- Configure monitoring and notifications.
We encourage you to leave your comments below and check our other posts.
You can also contact us for a free consultation.