Site icon

Cloud ACI 5.2: AWS Enhancements in TGW with TGW Connect Attachments

Soumitra Mukherji

Cloud ACI 5.2 for AWS can now use TGW Connect Attachment to enhance the previous ACI/AWS integration.  You can read more about what we had pre cAPIC 5.2 by using only TGW VPC attachment at a previous writeup.

Before we start discussing and showing how TGW Connect Attachment benefits this integration, let’s quickly discuss what TGW Connect Attachment is.  If you visit this AWS documentation you will see that it says “AWS Transit Gateway Connect offers a native way to integrate SD-WAN appliances with AWS Transit Gateway.”

Figure 1: AWS Transit Gateway Connect documentation

Further if you visit this AWS documentation, you will see there are several kind of TGW attachements.  In this writeup we will talk about the TGW Connect Attachment and how it benefits AWS/ACI Fabric integration.

Figure 2: AWS TGW Attachment types

In essence, the TGW connect attachment allows the TGW to connect directly to 3rd party network appliances.  From cAPIC 5.2 we have taken advantage of this AWS feature to bring up BGP peers from the Infra TGW to the Infra CSRs.  Before we start discussing about these enhancements in cAPIC 5.2, let’s recap what we had pre cAPIC 5.2 with TGW.

Figure 3: Pre cAPIC 5.2 TGW usage

The figure above is a representation of what we had pre cAPIC 5.2 release:

Notice the following:

In cAPIC 5.2 we have utilized the capability of TGW connect attachment to:

The figure below is a representation of ACI / AWS fabric using TGW Connect Attachment.

Figure 4: representation of ACI / AWS fabric using TGW Connect Attachment

Note: AWS has a limitation of having only up to 4 Connect Peers per Connect attachment.  The Implication of that is that with TGW we can support 4 CSRs in Infra per region

cAPIC 5.2 install for AWS is pretty similar to previous installs.  There are few differences (just like Azure cAPIC 5.2 install).

The below figures 5.x related to setup shows you the important parts of initial cAPIC setup with 5.2:

Figure 5a: cAPIC 5.2 initial setup DNS/NTP
Figure 5b: cAPIC 5.2 initial setup Region Management, enable TGW and choose regions and where CSRs go
Figure 5c: cAPIC 5.2 initial setup Region Management – subnet pools for cloud routers
Figure 5d: cAPIC 5.2 initial setup Region Management – HUB (TGW) Regions where CSRs are
Figure 5e: cAPIC 5.2 initial setup Region Management – Add CIDR for HUB subnet (GRE subnet)
Figure 5f: cAPIC 5.2 initial setup Region Management – completed CIDR allocation for HUB Subnet (GRE subnet)
Figure 5g: cAPIC 5.2 initial setup Region Management – BGP AS for HUB and CSRs
Figure 5g: cAPIC 5.2 initial setup, number of CSRs per region, Public IP:yes/no, credentials and license info

Once the cAPIC setup is completed and if you needed to add this site to MSO, please follow the appropriate part of this writeup which shows details on that. 

Once setup is completed, go to the AWS UI to verify your TGW that got created. You will notice that in 5.2 you only have one TGW instead of 2 which was the case pre Cloud ACI 5.2

Figure 6: cAPIC 5.2 creates only 1 TGW

If from AWS UI you look at TGW Attachments, you will notice that the TGW connect Gateway Attachment has 2 connect peers.  One to each CSR.   You will also notice that there is a BGP peer going from TGW to each CSR

Figure 7: viewing TGW Connect Gateway Attachment, Connect Peers and BGP Peers.

If you ssh to the AWS CSRs, you can  look at the interface and cross reference the Peer BGP address matches up to Tunnel 1 on CSRs

Figure 8: Matching up the BGP peer addresses on CSRs

Also, notice that the tunnel1 source is Gig2 of CSR and the tunnel destination is the AWS TGW GRE IP

Figure 9: tunnel1 source is Gig2 of CSR and the tunnel destination is the AWS TGW GRE IP

You will also notice that BGP peering information from CSRs match up between Azure UI and CSRs

Figure 10: validating the BGP Peers from CSR to TGW

On cAPIC you can also verify that you only have 1 TGW as shown in the figure below

Figure 11: Confirming that there is only 1 TGW from cAPIC

Conclusion:

  1. cAPIC now supports AWS TGW Connect Attachment Type in Infra VPC.  The CSRs are considered as the NVA (Network Virtualization Appliance) and BGP peering is built from CSRs to the TGW for control plane (over GRE tunnels).  This gives you the benefit not having to deploy 2 TGWs per region.  TGW is a highly available AWS resource and now you only need to use 1 TGW per region thus cutting down on your AWS resource consumption.
  2. If Tenant Region is not the infra region and is not a hub region (meaning no CSRs were deployed in that region), then from that Tenant Region connectivity to other DC sites (on Premise or Cloud) can still take place over the high speed TGW Inter Region Peering and then through the Infra CSRs.  Pror to 5.2, you would either need to bring up CSRs in that non Infra HUB region or have to configure VGW tunnels between that Tenant VPC and the Infra VPC.
Exit mobile version