In ACI the external Routing Peer to the router is done through border leaves with a object called L3Out.   L3Out has an object in it called the L3Out InstP also known as the External EPG.    In the External EPG you can put in Prefixes and choose the scope based on your intended goal.   Under the cover choosing these options defines security and route maps.   Often times people don’t quite understand what these options do and land up doing the wrong thing.   The intention of the writeup is to clarify these options and what they do.   We will also take a short example to see how to choose the right options for a Shared L3Out Scenario

The Figure below shows the Scope Options available.

Export Route Control Subnet along with Aggregate Export

Checking -Export Route Control Subnet enables a route map that allows external prefixes to get advertised out of your L3Out.   Note the word “external prefixes”.  This means that prefixes learnt from other L3Outs.  In other words this is Transit Routing.

Hint:  How I remember this. Keyword to remember here is “export”  meaning egress. Export means export what I learn from another L3Out to the outside world, i.e. transit routing.

In addition you will notice in the figure above,  that I’ve also checked the “Aggregate Export” check box.    This Aggregate Export checkbox is only available to click if:

a) your prefix is 0.0.0.0/0. and 

b) you have checked the “Export Route Control Subnet” checkbox.    When yu check this “Aggregate Export” checkbox, what it means is that it will advertise all the external prefixes it learns from another L3Out, out of your L3Out.   

Aggregate Export is not route summarization as you may be tempted to think.   Also, note from condition (a) above, that this option requires you to have 0.0.0.0/0 for the prefix.  In other words, this option is not available for other prefixes.   If you don’t choose this option, meaning you have 0.0.0.0/0 and you check “Export Route Control Subnet” but not “Aggregate Export”, then it. will do an exact match on 0.0.0.0/0 meaning if you have learnt the default route from the other L3Out,  it will advertise this out of your L3Out.   Let’s take another example. Let’s say you put 3.3.3.0/24 in that prefix.  Notice from the figure below that “Aggregate Export” is not available now.   So, what this means is that if you have learnt 3.3.3.0/24 from the other L3Out, it will advertise 3.3.3.0/24 out of your L3Out.

Note:   You also need to enable the checkbox for  “External Subnets for the external EPGs”  for the access lists.  Otherwise the transit Prefixes will be advertised but the access-control will prevent communications to work.

 

Import Route Control Subnet

This checkbox is greyed out by default.   This feature can be enabled by checking the main L3Out and checking the option “Route Control Enforcement: Import” to allow this feature.  The “Import Route Control Subnet” feature configures an ingress Route Map to give you control over what prefixes you want to import from your L3Out into the Fabric.   This feature is not commonly used.   If you have 0.0.0.0/0 as the prefix and enable this feature and check the checkbox, then you could also use the “Aggregate Import” feature with it, which has a similar concept to the above feature but on the ingress side.

 

External Subnets for the External EPG

This is by default turned on for any new prefix that you put in the External EPG.  Ofcourse you can turn it off when desired as you will see in the Shared Use Case example I’ll show you after I finish this discussion.   This knob is very simple.  Think of it as an access list for that external destination.  In the example below, I put prefix 4.4.4.0/24 with “External Subnets for the External EPG”.  What this means is that I’m allowing endpoints from my fabric (based on the contracts ofcourse), to external subnet 4.4.4.0/24.   If I had put 0.0.0.0/0 it would mean allow every external prefix reachability based on the contract.

Shared Route Control Subnet,  Shared Security Import Subnet and Aggregate Shared Routes

I like to put these three knobs in the same bucket, because they all have the word Shared in it.  Ofcourse you don’t have to use them together.  You will have to decide which combinations to use based on your use case.   I’ll elaborate this with an example after this last discussion of the meaning of these knobs.

 

Whenever you see the word “Shared”,  it should trigger in your mind that this is a functionality for Shared L3Out.  In other words, you want Route Leaking to another VRF.

Hint: How I remember this. Shared means on the way in, — share what I learnt from outside to another vrf inside the fabric

Shared Route Control Subnet:   This means that prefixes I learn from my routing peer (the external router),  will be advertised to that other VRF.  Ofcouse you will need contracts / consumed contract interfaces based on the VRF, but I will not discuss that here, because that is pretty well understood by folks.   In the example above what I am saying is that 4.4.4.0/24 prefix will be advertised to the other VRF in my ACI Fabric who is sharing this L3Out.

Shared Security Import Subnet:   This knob is a very simple one.  Think of it as an inbound access list for the other VRF for the prefix I use.   In the figure above, it would mean that the access list allows 4.4.4.0/24 to source from the outside to EPGs in that VRF where the relevant contracts are applied. 

Aggregate Shared Routes:  By now, you know what this feature does based on the other Aggregates described above.  Again, this is not route summarization.  The only thing different with this particular knob is that it works with prefixes other than 0.0.0.0/0.   In the figure above, I used 4.4.4.0/24 with Aggregate Shared Routes.   This means that if I was learning prefixes 4.4.4.0/25 and 4.4.4.128/25 from the outside world then both of these prefixes will be advertised to the other VRF  ( who is sharing this L3Out) in my fabric.

Example of Shared L3 Out:

Enogh talk about all this.  Let’s look at an example to see how these knobs can be used.

Let’s look at the figure below:

Explanation:

• I have a endpoint  10.3.1.10/24 hanging off Leaf1.  This endpoint belongs to a EPG in the User BD/VRF space.
• I have a L3 Out from Leaf2 in Common Tenant, Default VRF that peers with an external router.  I have 2 hosts outside somewhere which are 33.33.33.33/24 and 44.44.44.44/24.
• The User VRF does not have it’s own L3 Out. It wants to use that Shared L3 Out from Common Tenant

 

Basic Setup:

I won’t go into too much details here, because this part is well understood, but here’s the summary.

• I have a any/any Global Scope contract defined in Common Tenant.
• External EPG (InstP) provides the contract (incidentally, I prefer to make the External EPG the consumer generally the consumer, but thats beyond the scope of this discussion)
• User EPG-A Consumes the contract.   The BD subnet is marked with “Advertise Externally” and “Shared”.  There is no need to consume contract-interface here, because the contract is defined as global scope in Common Tenant and is thus available in this tenant.
• There is no need to put any subnets in the EPG level in this case, because the EPG is the consumer.

 

Intent:

I want the endpoint 10.3.1.10/24 to be able to reach 33.33.33.33/24 but not 44.44.44.44/24

Implementation:

The Strategy I have decided to take here is:

a). Allow the advertisements for the otuside prefixes to get advertised to the User VRF.

b). Implement the security Controls to only allow for my endpoint to have access to 33.33.33.0/24.   44.44.44.0/24 should not be able to be reached from the endpoint in the fabric.

Implementation of (a)

As you can see, from the above figure,  what I’ve done is put the prefix 0.0.0.0/0 and checked the boxes for “Shared Route Control Subnet”. and “Aggregate Shared Routes”

Now I should expect that on the User VRF,  I should see both prefixes show up.   However I should not be able to reach any of those prefixes from my endpoint, because I have not configured any sort of allow access-list.

Let’s verify:

You can see from the above figure , that both 33.33.33.0/24 and 44.44.44.0/24 now showup in the user VRF

Let’s verify that you can’t actually reach them from the endpoint

33.33.33.33 is not reachable

44.44.44.44 is not reachable

 

 

O.K.  So far all good !

Now,  let’s do our Security Configuration for those shared prefixes.   To recap,  we want only 33.33.33.0/24 subnets reachable

Just 2 clicks as you can see below:

• We introduce the prefix 33.33.33.0/24
• External Subnets for the External EPG – for the egress access list to that prefix
• Shared Security Import Subnet – for the ingress access list to the user VRF for that prefix
• Recall the route advertisements were already taken care of by the 0.0.0.0/0 prefix from the previous step, so we don’t need to do anything more

Let’s Verify now:

33.33.33.33 is now reachable from our Fabric Endpoint

44.44.44.44 is not reachable from our Fabric Endpoint

We’ve Accomplished Our Goal !!!

 

References:

L3Out Subnet Scope