At some point, you will need to configure Layer-3 routing from your ACI Fabric to External world. In this article, we will explore using EIGRP as the routing protocol to connect to our ACI Fabric.
Assumptions
- Routed Interfaces will be used from External Devices to ACI Border LEAFs
- VRF is operating in Enforced Mode (meaning, we are enforcing Contracts between EPGs, which is the default operation).
- Preferred Group Membership will not be used
- Note – There is a caveat of using Preferred Group Membership with L3outs; Your L3EPG Subnet will have to be defined as 0.0.0.0/1 and 128.0.0.0/1. See the link here for more details.
- External Routers will be configured with an MTU of 9000
Prerequisites for this design
- Configure ACI Fabric BGP route reflectors
- Configure Fabric Access Policies (ACI Leaf Interfaces that connect your External Routers will need to be configured)
- Tenant Configuration is complete (Application Profile, EPG, BD, Contracts / Filters, and VRF already exist)
Caveats for this design
- Border Leaf switches will connect to only (1) External device each (i.e., we did not connect Leaf201 to both N7K1 and N7K2). Reference DDTS CSCuy16355.
- If you have a requirement to connect a border leaf to more than one external device (i.e., classic-V L3 topology), please make note of the considerations for this design which can be found in the Transit Routing section of the “Cisco APIC and Transit Routing Document” on CCO.
HW/SW requirements
- Minimum Software of APIC 1.1 is required for EIGRP
- No hardware requirements
Border Leaf Switches will connect to one External Device each, using routed interfaces.
In order to achieve the above configuration, we will do the following:
- Create EIGRP Protocol Policies – This will define the protocol policies which are using for our interface on the Border LEAFs; parameters, such as hello timers, split horizon, etc).
- Create External Routed Network (L3OUT) – Here, we will select our routing protocol (in this case, EIGRP), the VRF (routing table) to attach the L3out, and the External Domain.
- Define Node Profiles – We are essentially identifying which LEAF switches will be used as Border LEAFs; We will select the node(leaf), and configure a loopback address. (Note – we will do this twice; one Node profile for Leaf201, and one for Leaf202).
- Define Interface Profiles – We will select the Switch Interface, and determine how to configure it (SVI, Routed Sub-interface, or Routed). This is where you will define your IP address, MTU for the interface, etc).
- Define the External EPG (L3ExtInstP – or L3EPG) – This is an External EPG; The External EPG is an external destination that we are trying to reach from within the fabric. While I will use 0.0.0.0/0 (which will define any address as a destination in the external network), it is possible to narrow this scope to networks that exist behind the L3out.
- Associate External EPG to appropriate Contract – We will provide and consume Contracts here (i.e., internal EPGs will consume L3OUT services, etc).
- Associate BDs to the L3out AND ensure our BD Subnets are configured to advertise externally.
- Configure External Router
Configuration Steps:
1. Create EIGRP Protocol Policies – Tenants > Networking > Protocol Policies > EIGRP Interface > Create EIGRP Interface Policy
- Name the policy
- Use the default configurations and hit submit.
2. Define your L3out – Tenant > Networking > External Routed Networks > Create Routed Outside
- Name your L3out
- Select your VRF
- Select your External Routed Domain (while we will not use a Vlan associated with the External Routed Domain, it is required by ACI or a fault will be generated).
- Select EIGRP as your Routing Protocol and define your Autonomous System #
3. Define our Node Profile (Repeat this for Leaf202)
The next step is to configure our node (Border Leaf201). To do this, click the plus sign under ‘Nodes and Interfaces Protocol Profiles’.
- Name the Node Profile (i.e., Leaf201_NodeProf)
- Select your Node
- Configure a Router-ID
- Select the “Use Router ID as a Loopback Address”
4. Define our Interface Profiles (Repeat this for Leaf202)
- After your create the Node Profile, click the “+” button next to the EIGRP Interface Profiles to continue.
- Name your Interface Profile (i.e., Leaf201_IfProf)
- Ensure “Config Protocol Policies” is selected, and click Next.
- Under the EIGRP Profile, select the EIGRP Interface Policy that you configured in Step 1, then Click next.
- Select the Interface type, in this case, Routed.
- Select your Interface (i.e., Node-201, Eth1/3)
- Define you IP address for your Routed Interface
- Note the MTU; Inherit means that we will inherit the default system MTU, in this case 9000.
5. Define the External EPG (L3ExtInstP – or L3EPG) – Tenant > Networking > External Routed Networks > L3outName > Networks > Create External Network
Note – External EPGs are a mapping to the external L3OUT using IP prefix and mask. More than one External EPGs may be configured, depending if different policies need to be applied to these external EPGs.
Contracts will be needed to allow communication to occur between internal EPGs in the VRF and the External EPGs configured by the L3OUT. Without contract, all connectivity from outside is blocked and external routes will not be learnt.
When creating an external EPG, the “Subnet” field defines the external subnets/network which are allowed to be advertised to the ACI fabric from outside. Multiple entries are allowed.
- Name your External EPG (L3EPG)
- Click the “+” button to define your L3EPG Subnet
- Define your Subnet
- Note – This configuration will not allow transit routing to occur; this means that routes that originate from Nexus7K1 will not be routed through ACI and be advertised to Nexus7K2. The knobs that enable transit routing to occur are the “Export Route Control Subnet” and “Aggregate Export”. For more information about transit routing, checkout this document on CCO that discussed Transit Route Control.
6. Associate External EPG to appropriate Contract – Tenant > Networking > External Routed Networks > L3outName > Networks > L3EPG
- In the upper right hand corner of your External EPG, select Contracts
- Select the contract you wish to provide (assumption is that you have previously configured a contract).
- You can either 1. Consume this contract with a VZany contract (this is what we will do) or 2. Consume this contract on a per-EPG basis.
7. Associate BDs to the L3out – Tenant > Networking > BD
- Ensure the Subnet defined on the BD is set to a scope of “advertise externally”
- Associate the BD to the L3out we just configured.
We have completed the L3out Configuration for ACI, lets move on to the External Router Configuration:
8. Configure External Router
Note – Ensure your MTU matches! Failure to match your MTU will result in EIGRP neighbor sessions that do not fully form.
system jumbomtu 9000
!
router eigrp 50
autonomous-system 50
!
interface Ethernet1/9
description L3 connection to Leaf201 e1/3
mtu 9000
ip address 192.168.201.2/30
ip router eigrp 50
no shutdown
Verification
From the Nexus – N7K1:
LabCore01# show ip route
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
100.1.1.0/24, ubest/mbest: 1/0 . <<< BD Subnet
*via 192.168.201.1, Eth1/9, [170/51456], 01:59:14, eigrp-50, external
111.111.111.111/32, ubest/mbest: 2/0, attached
*via 111.111.111.111, Lo1, [0/0], 3d13h, local
*via 111.111.111.111, Lo1, [0/0], 3d13h, direct
192.168.1.1/32, ubest/mbest: 2/0, attached
*via 192.168.1.1, Lo0, [0/0], 3d14h, local
*via 192.168.1.1, Lo0, [0/0], 3d14h, direct
192.168.50.0/24, ubest/mbest: 1/0, attached
*via 192.168.50.251, Vlan50, [0/0], 12:52:08, direct
192.168.50.251/32, ubest/mbest: 1/0, attached
*via 192.168.50.251, Vlan50, [0/0], 12:52:08, local
192.168.201.0/30, ubest/mbest: 1/0, attached
*via 192.168.201.2, Eth1/9, [0/0], 01:59:17, direct
192.168.201.2/32, ubest/mbest: 1/0, attached
*via 192.168.201.2, Eth1/9, [0/0], 01:59:17, local
201.1.1.1/32, ubest/mbest: 1/0 . << Leaf201 RouterID/Loopback
*via 192.168.201.1, Eth1/9, [90/128576], 01:59:14, eigrp-50, internal
From the Leaf201:
Note – I had to specify the VRF for our Tenant. The usage is “Tenant_Name:VRF_Name”
Leaf201# show ip route vrf Coast:coast_vrf
IP Route Table for VRF "Coast:coast_vrf"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
100.1.1.0/24, ubest/mbest: 1/0, attached, direct, pervasive
*via 10.0.48.66%overlay-1, [1/0], 3d14h, static
100.1.1.1/32, ubest/mbest: 1/0, attached, pervasive
*via 100.1.1.1, vlan8, [1/0], 01w08d, local, local
111.111.111.111/32, ubest/mbest: 1/0
*via 192.168.201.2, eth1/3, [90/128576], 02:01:12, eigrp-default, internal
192.168.1.1/32, ubest/mbest: 1/0
*via 192.168.201.2, eth1/3, [90/128576], 02:01:12, eigrp-default, internal
192.168.1.2/32, ubest/mbest: 1/0
*via 10.0.0.93%overlay-1, [200/128576], 02:00:38, bgp-65001, internal, tag 65001
192.168.50.0/24, ubest/mbest: 1/0
*via 192.168.201.2, eth1/3, [90/3072], 02:01:12, eigrp-default, internal
192.168.201.0/30, ubest/mbest: 1/0, attached, direct
*via 192.168.201.1, eth1/3, [1/0], 02:01:19, direct
192.168.201.1/32, ubest/mbest: 1/0, attached
*via 192.168.201.1, eth1/3, [1/0], 02:01:19, local, local
192.168.202.0/30, ubest/mbest: 1/0
*via 10.0.0.93%overlay-1, [200/0], 02:00:40, bgp-65001, internal, tag 65001
201.1.1.1/32, ubest/mbest: 2/0, attached, direct
*via 201.1.1.1, lo10, [1/0], 02:01:19, local, local
*via 201.1.1.1, lo10, [1/0], 02:01:19, direct
202.1.1.1/32, ubest/mbest: 1/0
*via 10.0.0.93%overlay-1, [1/0], 02:00:42, bgp-65001, internal, tag 65001
Sample Configuration
For a sample XML configuration of this L3out configuration, use the following link below.
Customer_L3Out_EIGRP_Routed.xml
