Prerequisites
While there are no prerequisites, if you need a refresher on Fabric Access Policy Configurations inside of ACI (i.e., AAEP, Switch Profiles, Interface Profiles, Vlan Pools, Policy Groups), check out this post, Configuring Fabric Access Policies.
vPC Overview
A virtual port channel (vPC) allows links that are physically connected to two different ACI Leaf nodes to appear as a single port channel to a third device (i.e., network switch, server, any other networking device that supports link aggregation technology).
vPCs consist of two ACI leaf switches designated as vPC peer switches. Of the vPC peers, one is primary and one is secondary. The system formed by the switches is referred to as a vPC domain.
The following behavior is specific to the ACI vPC implementation:
- No dedicated peer-link between the vPC peers; Instead the fabric itself serves as the MCT.
- Peer Reachability protocol – ZMQ is utilized in lieu of CFS
- ZMQ is an open-source high-performance messaging library that uses TCP as transport
- This library is packaged as libzmq on the switch and linked into each application that needs to communicate with vPC peer
- Peer-reachability is not handled via a physical peer-link; Instead, routing triggers are used to detect peer reachability
- The vPC Manager registers with URIB for peer route notifications.
- When ISIS discovers a route to the peer, URIB notifies vPC manager, in turn attempts to open ZMQ socket with the peer
- When the peer route is withdrawn by ISIS, the vPC manager is again notified by URIB, and it brings the MCT link down
Defining vPC switch Pairs inside of ACI
Note – This section is not where you define your vPC to end-host connectivity; This section is where you group your Leaf switches into vPC peer groups.
Best Practice – Keep it simple. Group your Leaf switch peer groups like this:
- Leaf201_202
- Leaf203_204
- Leaf205_206
For more information on ACI Naming Best Practices, check out the ACI Naming Best Practices article.
You can find the configuration window under Fabric > Access Policies > Switch Policies > Policies > Virtual Port Channel default.
- Name the vPC peer-grouping name. I choose something simple, like “Leaf201_202”. This tells me which two Fabric nodes are vPC peers.
- Provide the vPC peer ID (logical peer ID). For this setting, I use the first node ID number for the ID (i.e., 201, assuming the peer group is Leaf201_202).
As you can see below, the vPC switch peer group name that we entered is shown below (Leaf201_202) as well as the Logical Pair ID that we entered (201). The Virtual IP is an auto-generated IP address from the system TEP pool, and represents the virtual shared (Anycast) TEP of the vPC switch pair, Leaf201_202 (i.e., packets destined to vPC-connected endpoints off of Leaf201_202 will use this Anycast VTEP to send the packets)
vPC Policy Design Options
The section below will give a few visual examples of how to define vPC configuration Policy in ACI. For all examples, we will be modifying configurations under Fabric > Access Policies.
Option 1 – VPC with SAME Leaf interfaces across two leafs with Combined Profiles
In the example below, we have defined the following:
- A Combined Switch Profile called Leaf201_202_SwProf (Node 201 and Node 202)
- A Combined Interface Profile called Leaf201_202_IntProf (Node 201 and Node 202)
- An Access Port Selector called Eth1_1 (under the Leaf201_202 Interface Profile) is pointing towards a vPC Interface policy group.
- The vPC Interface Policy group is pointing towards an AAEP called Customer_AEP.
- The AEP (Customer_AEP) has an association with the Customer_PhysDom
- The Customer_PhysDom has an association with a Vlan Pool called Customer_Static_VLPool.
So what does this do? On ports Eth1/1 on both switches Leaf201 and Leaf202, we will configure those ports to be apart of a vPC. This vPC interface will have access to Vlans 1201-1299. Depending on the Interface Policy Group, we could enable LACP Active and other interface specific policy configurations.
When would I use this configuration approach?
- Customers who dedicate Leaf switches as Compute Leafs – If you had dedicated pairs of compute leafs, with nothing but vPC-connected servers, for example, this would be a solid use case for using combined-switch/interface profiles under your fabric access policies (for those switches). You could pre-configure your Switch, Interface, Access Port-Selector, and vPC Interface Policy Groups in such a way that allowed you to plug in 48 Chassis-type servers, with minimal effort.
Option 2 – VPC with SAME Leaf interfaces across two leafs with Individual Profiles
In the example below, we have defined the following:
- Individual Switch Profiles called Leaf201_SwProf and Leaf202_SwProf (Node 201 and Node 202)
- Individual Interface Profiles called Leaf201_IntProf and Leaf202_IntProf (Node 201 and Node 202)
- Access Port Selectors called Eth1_1 (under the Leaf201 and Leaf202 Interface Profiles) is pointing towards the same vPC Interface policy group.
- The vPC Interface Policy group is pointing towards an AAEP called Customer_AEP.
- The AEP (Customer_AEP) has an association with the Customer_PhysDom
- The Customer_PhysDom has an association with a Vlan Pool called Customer_Static_VLPool.
So what does this do? On ports Eth1/1 on both switches Leaf201 and Leaf202, we will configure those ports to be apart of a vPC. This vPC interface will have access to Vlans 1201-1299. Depending on the Interface Policy Group, we could enable LACP Active and other interface specific policy configurations.
When would I use this configuration approach?
- When you have Leafs that support mixed workloads (i.e., compute, services, APICs, etc). In this case, having Individual Interface Profiles allows for the most amount of flexibility, while allowing you to keep your Fabric > Access Policies configuration as clean and manageable as possible. This is the approach I see the most from customers.
Option 3 – VPC with DIFFERENT Leaf interfaces across two leafs with Individual Profiles
In the example below, we have defined the following:
- Individual Switch Profiles called Leaf201_SwProf and Leaf202_SwProf (Node 201 and Node 202)
- Individual Interface Profiles called Leaf201_IntProf and Leaf202_IntProf (Node 201 and Node 202)
- Access Port Selector called Eth1_1 (under the Leaf201 Interface Profile) is pointing towards the same vPC Interface policy group.
- Access Port Selector called Eth1_2 (under the Leaf202 Interface Profile) is pointing towards the same vPC Interface policy group.
- The vPC Interface Policy group is pointing towards an AAEP called Customer_AEP.
- The AEP (Customer_AEP) has an association with the Customer_PhysDom
- The Customer_PhysDom has an association with a Vlan Pool called Customer_Static_VLPool.
So what does this do? On ports Eth1/1 on Leaf201 and Eth 1/2 on Leaf202, we will configure those ports to be apart of a vPC. This vPC interface will have access to Vlans 1201-1299. Depending on the Interface Policy Group, we could enable LACP Active and other interface specific policy configurations.
When would I use this configuration approach?
- If you want to make you life difficult, and enjoy and having to constantly look back at the GUI (or other documentation source) to determine where you plugged in your server, this is for you.
- Sometimes in a lab environment, you cannot use interfaces that match up; this is where I frequently see this.
- Do not do this in production.. Just don’t.
