ACI / APIC Software Guidance

DISCLAIMER – This post is not meant to take the place of a software recommendation process.  Customers should still perform their own due diligence before selecting a software version for their own ACI Fabric. This post is meant to summarize the available ACI Software features, by release, and be used as a quick reference guide. I’ll offer up my favorite version as well, but as all good consultants know, when asked “what version should I run”, the answer is always “It Depends”. 😉

It depends. As someone who has been in a consulting role for a long time, this is my go-to answer. Although I do use this statement whenever I am in doubt of an answer, I just as often use It depends even if I feel I know the answer. This is especially true when it comes to providing Software Guidance for ACI. While I have a preferred “go-to” software version that I start with, a lot depends on the customer, the features they are running in their environment, the hardware, and where they are going in the near future. Take a look at the versions below. I’ve tried to keep it as straight forward as possible, including where Hardware and Software features were first supported. I’ve also included a few versions never to run, based on my own experience.

• It is a Long-lived Supported APIC/ACI Code Release. Cisco supports direct upgrades from long-lived release to long-lived release, which means you don’t have to step upgrade to get to the previous (or next) long-lived release! If you want to verify what the upgrade path looks like, check out the ACI Upgrade Tool on CCO to verify.
• Lots of features!
  • MultiPod Support
  • MultiSite Support
    • Lots of MultiSite enhancements!
  • MultiPod + MultiSite
  • Lots of ACIAnywhere enhancements (AWS + Azure)
  • Remote Leaf Support
  • Cisco AVE
  • Flood in encap (critical for ACI-Centric deployments)
  • Vmware Vsphere 7.0 VMM is supported
  • Application Services Engine Support

APIC 1.0

In addition, here are the end-of-life announcements for all 1.x APIC Software.

1.3 (Bronx)

• New Hardware Supported
  • N9K-C93180YC-EX (Leaf)
  • N9K-X9732C-EX (Spine LC)
  • N9K-C9504-FM-E (Spine Fabric Module)
  • N9K-C9508-FM-E (Spine Fabric Module)

APIC – 2.0

2.0(2) (Congo)

• New Software Features
  • VMM – VMware VCenter 6.0 is supported
  • Contract Permit Logging
  • MultiPod – MultiPod support introduced
  • Copy Services
  • EPG deployment via AAEP
  • L3 multicast support (requires at least -EX based Leaf)
  • Policy-based redirect
  • Syslog in NXOS Style CLI Format
  • Proxy ARP
  • Per-EPG MCP
• New Hardware Supported
  • N9K-C93108TC-EX

2.1 (Crystal)

2.2 (Danube)

2.3 (Drava)

3.1 (Euphrates)

• Do not use 3.1(1i) due to CSCvh29461 – DSCP-cos translation policy may break MultiPod BGP. This issue is resolved in 2.2(4f) and later in the 2.2(4) train, and in 3.1(2m) and later in the 3.x train.
• Other bugs to be aware of:
  • Endpoint Learning Bug – CSCvi11291 – XR learn on BL even with “Disabled remote EP learn” for BGP packets (tcp port 179). This bug is first fixed in 3.2(1). This issue is most commonly seen when you have external security port-scanners (or other devices generating tcp-179 packets) that are sent to endpoints on the ACI fabric. When this occurs, Remote (XR) Stale entries can pop up. This issue is resolved in 2.2(4m) and 3.2(1).
• New Software Features
  • Monitor Active GUI Sessions
  • BFD support for Spine switches
  • Cisco AVE (Next-Gen AVS)
  • L4-7 Cloud Orchestrator Mode
  • Flooding is limited to Encapsulation (Flood-in-Encap)
  • Downlink support for Uplink ports on EX-based and FX-based Leaf switches.
  • OpenShift Container support
  • Remote Leaf Switches
  • MultiSite – N9K-C9364C Spine Switch is now available for MultiSite
• New Hardware Supported
  • Beginning with ACI 3.1(2) – N9K-C9336C-FX2

3.2 (Fraser)

• Guidance – While Latest 4.2(x) is preferred, 3.2 is also a current long-lived release. If you don’t require any of the features in 4.x train, please use the gold star release for 3.2.x

• If needed, do not use 3.2(1l) due to CSCvj65274 – Switch crash possible during upgrade to 3.2(1). Switches with Call Home Inventory Policies enabled and applied to switches may encounter a switch crash with the eventmgr service.
• Do not use 3.2(5d) due to CSCvo83991 – SNMP traps not being sent from the TOR. 
• All versions of ACI code up until 3.2(6i) are vulnerable to CSCvo80686 – Cisco Nexus 9000 Series Fabric Switches ACI Mode Default SSH Key Vulnerability.
• Other bugs/enhancements to be aware of:
  • CSCvm12554 – Contract Preferred group l3out prefix not deployed on ingress VPC; this bug was re-introduced to 3.2 affects all available versions of 3.2 up until 3.2(4d) where it has been fixed again.
  • Endpoint Learning bug – CSCvi11291 – Remote Learn on Border Leaf even with Disabled Remote EP learn with pkt with src/dst of 179. This issue is most commonly seen when you have external security port-scanners (or other devices generating tcp-179 packets) that are sent to endpoints on the ACI fabric. When this occurs, Remote (XR) Stale entries can pop up. This bug is first fixed in 3.2(1).
  • Endpoint Learning Enhancement – CSCvj17665 – EP announce support for stale IP XR EPs – This enhancement improves endpoint learning functionality by allowing a new EP Announce delete message to be sent to all leafs within the site on the expiration of Bounce IP XR Entries. This enhancement is available beginning with 3.2(2).
  • CSCvj90443 – Preconfigured VPC can lead to duplicate VIP/TEP IP assignment – This issue is resolved in 3.2(2o) and later
• 3.2 is a long-lived code train starting with 3.2(2); see the link here.

4.2 (Indus)

Warning – Do NOT use 4.2(1i). Instead, use the latest gold star version of 4.2(x).

• 4.2(x) – Long Lived Release and General Recommendation for customers (unless new features warrant newer version). Please use the gold star version in the 4.2.x train.
  
• New Software Features
  • Floating L3out
  • Viptela SD-WAN integration
  • Backup (N+M) PBR
  • Cluster validation tool
    • ifav40-ifc1# acidiag cluster
  • PIMv6 support
  • ACI CNI SNAT
  • Docker Enterprise Edition Integration
  • Pre-download of Image for faster upgrade
  • CDP/LLDP support for management interfaces
  • 4.2(3) – Separating CRC errors and FCS errors
  • Multi-Pod Convergence improvements for L2, L3 unicast, and BUM cases
  • L3out Enhancements
    • BGP Session Shutdown and soft reset
    • IPv6 Multicast address family for BGP L3out
    • etc
  • Cloud Features
    • ACI Anywhere – Azure Extension
    • Ability to have cloud only deployments (AWS and Azure)
    • Support for CSR version 16.12
  • Multi-Site Enhancements
    • MSO Inter-version validation
    • Intersite L3out
    • AVE support (requires MSO 2.2(1))
    • Migration of EPG/BD
    • DHCP Relay for stretched BDs
    • 4.2(4) – vzAny contract support for Multisite
  • Remote Leaf Enhancements
    • 4.2(4) – Remote leaf support with 10Mbps link in IPN
    • 4.2(4) – Dot1q tunnel support on Remote Leaf
  • 4.2(4) – Allow custom names for ACI created port-groups (DVS, Hyper-V)
• Scale Enhancements
  • 64 Remote Leaf nodes for RL Direct
  • 256K support of contract policy rules on FX TORs
  • SPAN/ERSPAN SCALE: 32 session scale & port scale increase to 63
  • MultiPod
    • 4.2(4) – Support for 500 leafs per Pod.
  • MultiSite
    • 4.2(4) – Increase preferred group scale to support 500 EPGs in a multisite deployment
    • 4.2(4) – Scale increase of up to 10 templates per schema from 5 templates per schema
    • 4.2(4) – Support for 500 leafs per site
• New Hardware Supported
  • 4.2(2) – Spine: 16x400G ports spine : (N9K-C9316D-GX)
  • 4.2(2) – Leaf: 28×40/100G+8x400G ToR :  (N9K-C93600CD-GX)
  • 4.2(2) – RL POD Redundancy support
  • 4.2(3) – N9K-C9364C-GX support as ACI leaf
  • 4.2(3) – Application Services Engine
  • 4.2(3i) – N9K-C9364C-GX software support added

APIC 5.0

5.0 (Jordan)

• New Software Features
  • Cloud Features
    • AWS Transit Gateway Support
    • AWS TGW Peering
    • Stats Filter for AWS
    • Cloud APIC UI topology view support
    • Common Driver for AWS and Azure
    • Cloud + CSR Upgrade Reconciliation
    • Cloud APIC Stateless cPE
    • AWS – 17.1 CSR Upgrade
  • Service Provider Features
    • SR-MPLS Features
      • SR-MPLS Handoff Support on ACI Border Leaf
      • QoS Marking for North-South and East-West traffic flow
      • SR-MPLS Statistics
    • Legacy SP PBR gaps
      • MAC rewrite
      • L3Out service EPG
    • PBR : L4-L7 devices(Deployed Device) in different BDs for load-balancing L1/L2 PBR
    • ACI Peering with vRouters (Floating L3out) a.k.a BGP next-hop unchanged propagation
  • Endpoint Security Group support
  • Usability & Day 2 Operations
    • During upgrade –> keep visibility on what’s happening on the APIC that is upgrading
    • MSO UI/UX enhancements for cAPIC
    • SNMP/Syslog apply policy in Day0 wizzard
    • Pre-download image to all/selected leafs
    • Trigger MPOD parallel switch upgrade across PODs
    • Upgrade Checker
    • ACI App Center RBAC
    • Two Factor Authentication with DUO for APIC, cAPIC
  • Openstack/Containers Features
    • Support for additional platforms
      • Openstack Ironic on Redhat OSP13
      • Kubernetes on DockerEE Release 3
      • Redhat Openshift 4.3 on AWS with ACI-CNI
      • Redhat Openshift 4.3 on OSP13 with ACI-CNI
    • Infrastructure improvements
      • Infra-VLAN hardening for Opensource VMMs
      • Restricted admin account for orchestrators like Openstack
      • Kubernetes support on mixed baremetal and VM topologies
      • Installer improvements (POD BD in common tenant,  validations, etc)
    • Data plane improvements
      • Add active/active VIP support on Neutron L2
      • Support SVI connectivity for OVS/opflex ports in Openstack
      • Improve scale per leaf pair  to support 120 Opensource OpFlex hosts
      • Support VMware teaming with enhanced LAG for Kubernetes nested in ESX VMM
• Scale Enhancements
  • VMM Integration: Support to Increase the # of DataCenters in a vCenter Server from 4 to 15
  • Cisco N9K-C9336C-FX2 switch, you can now apply a breakout configuration on ports 1 through 34, which can give up to 136 (34*4) server or downlink ports.
• New Hardware Supported
  • Cisco Nexus 9508-FM-G Fabric Module support