In this post, I will show you step by step how to add in a new Azure Site to the ACI Cloud Fabric. In a previous post we had already shown step by step on how to add a AWS site to the physical ACI Fabric. We will continue from there and now add in another site which will be on the Azure cloud.
Keep in mind though that you don’t need to have a physical fabric to configure ACI on Cloud Sites. You could just add Azure or AWS or a combination of AWS and Azure to your Cloud ACI fabric. Once installed you can configure and manage your tenants from a single pane of glass and consistent policy across the entire ACI Anywhere Fabric, whether it be onPrem or Cloud or any combination of them.
In the diagram below we show that the intention is to add in the Azure Site now to the ACI Anywhere Fabric.

Updates:
- 6/1/2020: Azure cAPIC release 5.0.1k is now out in Azure Marketplace. Please install this release.
- The next maintenance release (5.0.2 JMR-1 which is due any time shortly (as of 6/17/2020) will have many more features. This MR1 release is due shortly.
- Support for Azure Multi-Node service Graphs
- Support for Inter-VNET/VPC services
- Support for Azure NLB (Network Load Balancer ) automation with service chaining
- Support for static IP for Load Balancer
- 32 Character VRF Support on CSR (which runs on IOS-XE)
- Support for VNET peering for Azure cloud (currently VNG deployment could take up to 45 minutes (depending on the region). With VNET peering the connectivity should be immediate)
- Support for comma-separated filters for rule creation in contract
The steps that we need to add in the Azure Site in principle is very much like AWS.
Let’s summarize the steps needed to add a Azure site to the ACI Fabric:
- Check / Verify that your Azure account has the required Limits / Quotas
- Register the necessary Resource Providers in Azure
- Create Key Pairs for the capic ssh login (please make sure you copy the key correctly, no newline character “\n” is allowed in the key)
- Subscribe and Register to vCSRs on Azure Market Place
- Subscribe and Register for cAPIC on Azure Market Place
- Run the ARM (Azure Resource Manager) Template and put in the information as required and complete the run
- Verify that the cAPIC has spun up
- Configure the IAM (Identity and Access Conrol Manaager) for cAPIC
- Browse to cAPIC and do the initial configuration
- Browse to MSO and add in the new Azure Site
- Download the configuration files for physical Site vCSR routers that you will need to configure for matching IPsec tunnels. Use those configs to configure the on Premise CSRs.
- Verify on the CSRs that tunnel has come up, OSPF neighbors are up and bgp l2vpn evpn peers are up
- Verify from cAPIC that everything is looking good
Let’s now Start on the Azure Side
Step 1: Check / Verify that your Azure account has the required Limits / Quotas. Please refer to this link.


Step 2: Register the necessary Resource Providers in Azure
Please click here for reference to this item: necessary Resource Providers in Azure
Step 3: Create Key Pairs for the capic ssh login.
On your local mac terminal or Linux box, use “ssh-keygen -f azure-capic-key”. If you are using a Windows Machine, you can do this with putty. Please refer to this link.
Quick Note: on the key. When you generate the key, the last part of the key will have the username@hostname in the public key. When pasting in the public key for cAPIC later, make sure not to include that last part (as shown in the diagram below).
(please make sure you copy the key correctly, no newline character “\n” is allowed in the key)

Step 4: Subscribe and Register to vCSRs on Azure Market Place



Note: The version of CSR that you need is dependent on the version of cAPIC you deploy. For Azure, Version 5.0 of cAPIC still needs CSR release 16.12.x (unlike AWS where cAPIC requires you to subscribe to CSR version 17.1). 5.0.2 (JMR-1 will require CSR release 17.1) When the cAPIC tries to spin up the CSRs, if you have subscribed to the wrong version of CSR, the CSRs will not spin up. On cAPIC, you will see a fault saying that you have not acknowledged the subscription. To be on the safe side you might just want to subscribe to Version 16.12 and 17.1. You won’t be charged just for subscribing. cAPIC will spin up the correct version of CSR that goes with that APIC release.
Also, it’s important to note that you need to choose the release that says “Bring Your Own License”
For required version of CSR, please see this link.


Step 5: Subscribe and Register for cAPIC on Azure Market Place



Step 6: Run the ARM (Azure Resource Manager) Template and put in the information as required and complete the run




Step 7: Verify that the cAPIC has spun up


Step 8: Configure the IAM (Identity and Access Control Manaager) for cAPIC

Step 9: Browse to cAPIC and do the initial configuration







Note: The items in the below screenshot ( CSR IP and External Subnet Pool) have moved to MSO from release 5.2. These options in 5.2 cAPIC will not be there.


You could at this time ssh to the Azure Installed CSRs (spun up by cAPIC) and look around

Step 10: Browse to MSO and add in the new Azure Site






Step 11: Download the configuration files for physical Site vCSR routers that you will need to configure for matching IPsec tunnels. Use those configs to configure the on Premise CSRs.

Step 12: Verify on the CSRs that tunnel has come up, OSPF neighbors are up and bgp l2vpn evpn peers are up

Step 13: Verify from cAPIC that everything is looking good


Your Azure ACI Cloud Fabric is now up and ready for you to start using. Please see the blog post on how to add Trusted Tenant Azure subscriptions if needed.
References:
- Cisco Cloud ACI on AWS White Paper
- Cisco Cloud ACI on Microsoft Azure White Paper
- Internet Service for Cisco Cloud APIC Workloads: (how to create untrusted user)
- Cisco Cloud APIC for AWS Installation Guide, Release 5.0(x)
- Shared On-Premises L3Out for Cisco Cloud APIC Workloads
- Cloud APIC Install Guide-Azure
- Cisco Cloud on Azure White Paper
- Cloud APIC Install / Upgrade & Configure Guide
Configuration Verifications and Screenshots captured and annotated by Goran Saradizic and Soumitra Mukherji