Cloud ACI 5.2: ACI/Azure Tenant vNET Peering across Azure Acitive Directories

From Cloud APIC 5.2 you can configure Tenant vNET peerings across Azure Active Directories. This will be very useful for B2B connectivity. Prior to this tenant vNET peerings for Azure using cAPIC was only possible across subscriptions in the same Azure Active Directory.

To Follow along this Proof Of Concept Lab in your own Azure subscriptions, you will need the following:

Pre-Requisites: 

  • You will need 2 Azure Active Directories With 1 Subscription in each.
    • AD1/Subscription1:  Running cApic 5.2, Infra Tenant
    • AD2/Subscription1:  Will be used for User Tenant
Figure 1: Recap of Azure Object Mapping to ACI Objects

VNET peering across Azure Active Directories

  • Prior to 5.2 release, VNET peering (cAPIC managed) could not be established between two VNETs if they belonged to two different subscriptions corresponding to different Azure Active Directories.
    • Azure Active Directory (Azure AD) is Microsoft’s enterprise cloud-based identity and access management (IAM) solution
    • An Azure subscription has a trust relationship with Azure AD. A subscription trusts Azure AD to authenticate users, services, and devices.
  • In 5.2 release, VNET peering across Active Directories is supported. Thus, the spoke VNETs can be deployed in subscriptions in different Azure ADs.
Figure 2: Recap: Azure vNET Peering for Tenant vNETs to Infra vNETs in ACI Fabric

Recap: Azure Active Directory

  • Azure Active Directory (Azure AD) is Microsoft’s enterprise cloud-based identity and access management (IAM) solution
  • An Azure subscription has a trust relationship with Azure AD. A subscription trusts Azure AD to authenticate users, services, and devices.
  • Multiple subscriptions can trust the same Azure AD directory.
  • Each subscription can only trust a single directory. One or more Azure subscriptions can establish a trust relationship with an instance of Azure Active Directory (Azure AD) in order to authenticate and authorize security principals and devices against Azure services.
Figure 3: Example of peering acoss Azure Active Directory

Considerations:

  • What this feature allows:
    • Allow the Infra VNETs and Spoke VNETs to be in different Azure ADs
    • Spokes can be spread across multiple different subscriptions across different Azure ADs
  • What this feature doesn’t allow:
    • Infra VNETs in a given site cannot be deployed or split across multiple different subscriptions. All the Infra VNETs in a site need to belong to the same subscription which means same Azure AD.

Managed Identity
Existing Infra Tenant and User Tenant’s Account Config:

  • Prior to 5.2 release, only ”Managed Identity” was supported for Infra Tenant’s Authorization or Credentials. For user tenants, both “Managed Identity” and “Service Principal” were supported.
  • To support VNET Peering across Azure ADs, both the Infra and User Tenants must use the Account only in Service-Principal mode.  In Release 5.2 support for Service Principal is now an option for Infra Tenant. This makes VNET peering possible when Infra Subscription and Tenant Subscriptions are in different Azure ADs.
    • VNET Peering across the Azure ADs can only be achieved by using Service Principal mode of Authorization on Azure. This cannot be achieved using the Managed Identity.
    • This limitation comes from the Azure SDK where this can be done only using the service Principal mode of credentials.
  • If Peering between VNETs across multiple Azure ADs is NOT required, then they can continue to use the existing Managed Identity for both users and Infra Tenants.

Note: Managed Identity and Service Principal
Managed Identity

  • Managed identities provide an identity for applications to use when connecting to resources that support Azure Active Directory (Azure AD) authentication. Applications may use the managed identity to obtain Azure AD tokens. For example, an application may use a managed identity to access resources like Azure Key Vault where developers can store credentials in a secure manner or to access storage accounts.
  • Here are some of the benefits of using Managed identities:
    • You don’t need to manage credentials. Credentials are not even accessible to you.
    • You can use managed identities to authenticate to any resource that supports Azure Active Directory authentication including your own applications.
    • Managed identities can be used without any additional cost.
  • https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/overview

In Case you are wondering why we were able to create ACI Tenants in different subscription from Infra Tenant Subscription using VNG (VPN Gateway) but for VNET Peering there are extra Service Principal configurations involved, then the below will hopefully clarify that.

  • Prior to release 5.2. when we needed to build the User Tenant in a different Azure AD from the Infra Tenant Azure AD, we needed to create a Service Principal (unmanaged identity) in the Tenant Subscription. This allowed us to establish VPN Gateway connectivity between Infra VNET and Tenant VNET.  The cAPIC was given the Service Principal credentials of the Tenant Account and the cAPIC was able to configure items in the Tenant Subscription in different Azure AD.
  • In 5.2, if you wanted to use VNET Peering instead of VPN Gateway between Infra Tenant Subscription and User Tenant Subscription, then each of these subscriptions would need roles to establish the VNET Peering to the other Subscription. For that reason, we would need to now create Service Principals in both Infra Tenant AD and User Tenant AD. Then a trust relationship has to be built so that the Service Principal on each AD is allowed to build the VNET Peering from its AD/subscription VNET to the other AD/subscription VNET.

What We Will Do In This Lab:

In this guided step by step example, we intend to do the following:

  • ACI Infra Tenant is present in Azure AD1/Subscription1 (that’s where cAPIC resides)
  • We intend to build a User Tenant in Azure AD2/Subscription1 which will be part of the cAPiC Fabric
  • We also want to have VNET Peering between User Tenant and Infra Tenant instead of using VNG

What we need to do to make the above work:

  1. Create Service Principal “ACI-Infra-SP” in Azure AD1 and give the role of Contributor to that Service Profile for AD1/Subscription1
  2. Create Service Principal “ACI-Tenant-SP”  in Azure AD2 and give the role of Contributor to that Service Profile for AD2/Subscription1
  3. Give Consent for ”ACI-Infra-SP” to be able to be used for other ADs
  4. Give Consent for “ACI-Tenant-SP” to be able to be used for other ADs
  5. Give ACI-Infra-SP  ”Network Contributor” Role for AD2/Subscription1 (cross AD role)
  6. Give ACI-Tenant-SP  ”Network Contributor” Role for AD1/Subscription1 (cross AD role)
  7. Go to cAPIC and change Infra Tenant to use Service Principal instead of using Managed Identity
  8. Build a simple ACI Tenant (use Service Principal) in AD2/Subscription1 and use VNET Peering instead of VNG
  9. Verify
Figure 4: Steps to complete AC Tenant vNET Peering across Azure Active Directories

Go to Azure Active Directory  for Infra Tenant Subscription and click on App registrations and add New registration

Figure 5: Creating the Service Principal for Infra Tenant Subscription

As you continue on creating the Service Principal, please follow the below figure.  It is important that you configure the Redirect URL:   https://www.microsoft.com

Figure 6: Continue on creating Service Principal

On the overview of the SP,  look at the relevant information and jot it down to a text file

Figure 7: Copy the information from Service Principal Overview to a text file

Now,  click on “Certificates & Secrets” and create either Certificate or Client Secret.  In this example, I’ll keep it simple and create Client Secret

Figure 8: Creating the Client Secret

Name the Client Secret

Figure 9: Naming the Client Secret

Add the secret information and jot  down the info in the same text file

Figure 10: Jotting down the Secret Information to the text file

ACI-Infra-SP Service Principal  and Client Secret has been created. 

Please  now make the Service Principal  “ACI-Tenant-SP” on  AZ AD2.  I will not show it here, just repeat the same process.

Once Done, we need to give each SP (in each AD)  the roles of Contributor for its subscription.  Let’s start giving Contributor Role for the Infra Tenant Subscription.

Figure 11: Add a role for the Infra Subscription

Select Contributor Role, Assign to Service Principal, in the Select box, type in ACI.  You will get a match on ACI-Infra-SP.  Click on that and add it in.

Figure 12: Adding Contributor Role to the Serrvice Principal in the Infra Subscription

Note that we also need to add in the Service Principal of the other AZ Directory “ACI-Tenant-SP” here as a Network Contributor.  However, it’s not showing up at this time !  

We’ll deal with that a little bit later

Now, go ahead and repeat the same process for Contributor role assignment process of ACI-Tenant-SP  for AD2/Subscription1.  I will not show it here.

Now, it’s time to work on giving cross account Service Principal Roles.  For this we will need to finish off the Microsoft Consent Process.   To do this you will need to create a URI and browse to it.  The URI format is shown below.  However, let’s do this with a simple bash script to make it much easier and error free.

Reference: https://docs.microsoft.com/en-us/azure/virtual-machines/windows/share-images-across-tenants

Figure 13: Format of URI to obtain Consent for cross directory roles

Copy and Paste the below on your MAC Terminal or Ubuntu Terminal Shell.  Make sure to choose everything below:

cat > getURI.sh <<EOF
#!/bin/bash
# Modify the Infra Dir ID   and Infra APP Id below
INFRADIR_ID="c10d8d5a-afd8-49af-ba49-5ca83470cd06"
INFRAAPP_ID="5d6b7cdb-2400-48b2-9fae-1041e73e2e45"

# Modify the Tenant Dir ID   and Tenant APP Id below
TENANTDIR_ID="01f0541f-adbe-45d4-b69a-309545208f6d"
TENANTAPP_ID="845371a4-291d-4e7a-9d6f-1a00297c9dd4"
echo
echo
echo  "use the below uri for ACI-Infra-SP consent"
echo "https://login.microsoftonline.com/$TENANTDIR_ID/oauth2/authorize?client_id=$INFRAAPP_ID&response_type=code&redirect_uri=https%3A%2F%2Fwww.microsoft.com%2F"
echo
echo
echo  "use the below uri for ACI-Tenant-SP consent"
echo "https://login.microsoftonline.com/$INFRADIR_ID/oauth2/authorize?client_id=$TENANTAPP_ID&response_type=code&redirect_uri=https%3A%2F%2Fwww.microsoft.com%2F"
EOF

Next, vi the file “getURI.sh”  and populate the information of Infra Tenant and User Tenant you collected earlier.  Example of my setup is shown in the figure below.

Figure 14: Populating the information in the shell script

Now, make the shell script executable and run it to obtain the 2 URIs needed

Figure 15: chmod 775; and running the script to get the 2 needed URIs
Figure 16: Browsig to the 1st URI (for Infra subscription)

Browse the the first URI from a browser

Figure 17: Accepting the consent
Figure 18: Web Browser will now redirect to http://www.microsoft.com

Repeat the steps for consent using the 2nd URI from script output.  I will not show it here.

Now go back to IAM of each subscription (from Azure UI) and add in the cross Role for the Service Principals as shown below. This was not possible until we finished the consent.

Figure 19: Adding Cross Subscription Service Principal “Network Contributor” Role

The Final Roles should show as below:

Figure 20: Viewing Final Roles for both Infra and Tenant Subscriptions
Figure 21: Step 7: Time to go to step for changing Infra Tenant to use Service Principal instead of Managed Identity
Figure 22: changing Infra Tenant to use Service Principal instead of Managed Identity
Figure 23: Populating the Information from the text file for Infra Subscription

You are all done to enable building ACI Tenants using vNET Peering (across Azure Active Directories) instead of VNG

Figure 24: Start building the Tenant using vNET peering instead of VNG (across Azure Active Directories)
Figure 25: Creating ACI Tenant across Azure Active Directory (from cAPIC – can use MSO also)
Figure 26: Creating the VRF, Could Context Profile and using VNET peering
Figure 27: Moving on to the Verification Step
Figure 28: Go to Virtual Network and double click on overlay-1
Figure 29: Verifying that cAPIC shows VNET Peering up in Infra VRF (overlay-1)
Figure 30: Verifying that VNET Peering is up from Tenant VRF
Figure 31: Verifying VNET Peering from Azure UI – on Infra Subscription (overlay-1 VNET)
Figure 32: Verifying from Azure UI that VNET peering is up – from Tenant VNET

Conclusion:  We have verified that from cAIC release 5.2, Tenant vNET peerings across Azure Active Directories is now possible. This will be very useful for B2B connectivity. Prior to this tenant vNET peerings for Azure using cAPIC was only possible across subscriptions in the same Azure Active Directory.


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.