The ability to upgrade an entire datacenter fabric from one location is one of the most amazing features that ACI brings to the table, and it is a game changer. This post will briefly describe the nuts and bolts of the process.

Pre-work

1. Create a backup of your ACI Fabric prior to upgrading. This should be a no-brainer, but is often overlooked. I always recommend generating a backup of your ACI Fabric configuration both BEFORE and AFTER an ACI Fabric upgrade. The rule of thumb is to always have a backup based on your current running version of code. Need help with creating a backup? Just go to the Creating a backup for your APIC Cluster post.
2. Always make sure that all of APICs are in a Fully Fit state prior to any upgrades. Never kick-off an upgrade of your fabric unless all APICs are in a Fully Fit state!
3. Resolve faults prior to your upgrade. This is often overlooked, but can lead to serious issues. It is always recommended to have as many faults resolved prior to upgrading the fabric.
4. Review the Cisco CCO APIC Upgrade/Downgrade Guide. This guide is comprehensive and covers which versions you can upgrade/downgrade to and from, guidelines, and suggested maintenance group configuration.
5. Determine which version of code if best for you and your ACI Fabric. If you have questions on what version of code you should target, check out post on ACI / APIC Software Guidance for a quick reference cheat guide.
6. Check out Recommended APIC Code page on CCO; great place to find out the long-lived ACI releases and supported CIMC images.
7. Check Release Notes for your intended version. It’s always good to check the release notes for any bugs, limitations, or version specific notes BEFORE you pull the upgrade trigger.
   • APIC 2.2(4) Release Notes
   • APIC 3.2(5) Release Notes
   • APIC 4.0(1) Release Notes
   • APIC 4.1(1) Release Notes
8. Download the code from CCO. You’ll need two files; the ISO file for your APIC Cluster, and the BIN file for your switches (one BIN file will work for both Spines and Leafs).
9. Check your CIMC code and ensure you are running supported code on your CIMC. If you need help with upgrading your CIMCs, check out the Upgrading your CIMC for your APIC article here.

ACI Upgrade notables

• In an ACI Multi-Pod environment, it is recommended to have separate maintenance groups per Pod
  • Switches in different pods can only be upgraded serially, regardless if they are in the same maintenance group. The switches cannot be upgraded in parallel.
• For your BGP-RRs (your Spines) in each Pod, make sure you always have one BGP-RR available during the upgrade process. In other words, don’t place both all of your BGP-RR spines in the same maintenance group. This is especially true when performing an ACI Fabric Upgrade for Multi-Pod deployments.
  • Note – This is referring to the “Route Reflector Nodes” under the BGP RR configuration. 

How long will the upgrade take?

While I can’t tell you exactly how long the upgrade will take, there are some guidelines that can be provided based on experience. The following times are approximate.

• CIMC = 45-60 minutes per CIMC/APIC << (may not be required)
• APICs = 30-45 minutes for the APIC cluster
• Switch Maintenance Groups = 15-30 minutes per Maintenance Group

So, if you have an APIC cluster of (3) APICs, and four maintenance groups in a single pod ACI Fabric, you’ll be looking anywhere from 1hr 30 minutes to 2hrs 45 minutes.

Upgrading your Fabric

Now that the pre-work is complete, lets move onto the actual upgrade!

Copy the ISO and BIN files to the APIC

There are several ways to get files from CCO onto your APIC Fabric.

1. SCP the files to the APIC
2. Upload the files via the GUI

While you can upload from the GUI, I prefer to use SCP to transfer the files to the APIC. I personally like to see the byte counts offered via SCP as opposed to the ominous transfer bar in the GUI 🙂

Option 1 – SCP the files to your APIC. 

• You will notice a couple of things about the scp command. “admin” is sample username we are using for the APIC.
• The IP address field is the IP of one of your APICs from your APIC Cluster (you only need to upload the files to one APIC).
• Don’t forget the colon “:” between the IP/ServerName and the directory
• New to SCP? Check out this helpful link.

1. Copy ISO file for APICs – scp filename.iso admin@10.18.188.101:/tmp/.
2. Copy BIN file for Switches – scp filename.bin admin@10.18.188.101:/tmp/. 

Add the files to the APIC Firmware Repository

1. Login to APIC and add firmware to firmware repo
   1. SSH to the APIC
   2. apic1# bash
   3. apic1# cd /tmp
   4. apic1# firmware add filename.ISO << go get a coffee. This will take 3-5 mins
   5. apic1# firmware add filename.BIN << stir your coffee. This will take 1-2 mins

Option 2 – Upload the files via the GUI

1. Upload ISO file for APICs
2. Upload BIN file for Switches

Verify the Firmware is present in the APIC Firmware Repository

1. Verify Firmware is in Firmware Repository on the APIC
   1. Admin > Firmware > Firmware Repository

Upgrade the Fabric!

1. Upgrade the APIC Controllers
   1. Admin > Firmware > Controller Firmware
   2. Click Action button > Upgrade Controllers
      1. Set target firmware version and Select Upgrade Now Option
      2. Click Submit
2. Upgrade your ACI Fabric Switches
   1. Set proper Firmware Group
      1. Admin > Firmware >  Fabric Node Firmware > Firmware Groups >
      2. Use existing firmware group or create new
      3. Firmware Groups are used to determine which software version our switches will use. It is normal for all switches to use the same firmware version.
   2. Go to Maintenance Groups and upgrade one maintenance group at a time.
      1. Admin > Firmware > Fabric Node Firmware > Maintenance Groups
      2. Maintenance groups allow an ACI Admin to divide their switches based on even/odds, function-type (compute leafs, service leafs, border leafs), etc, and upgrade just those switches at a given time. Check out the examples from the CCO Upgrade/Downgrade link of how you can create your maintenance groups.
         1. Two-group method
            1. Divide your switches into two groups: a red group and a blue group. Put one half of the spine switches in the red group, and the other half in the blue group. Also, put one half of the leaf switches in the red group and the other half in the blue group.
            2. Upgrade the red group.
            3. After the red group upgrade is complete, confirm that the fabric is healthy.
            4. Upgrade the blue group.
         2. Four-group method
            1. Divide your switches into four groups: a red spine switches group, a blue spine switches group, a red leaf switches group, and a blue leaf switches group. Put one half of the spine switches in the red spine switches group, and the other half of the spine switches in the blue spine switches group. Then, place half the leaf switches in the red leaf switches group, and the other half in the blue leaf switches group
            2. Upgrade the red leaf switches group.
            3. After the red leaf switches group upgrade is complete, confirm that the fabric is healthy.
            4. Upgrade the blue leaf switches group.
            5. After the blue leaf switches group upgrade is complete, confirm that the fabric is healthy.
            6. Upgrade the red spine switches group.
            7. After the red spine switches group upgrade is complete, confirm that the fabric is healthy.
            8. Upgrade the blue spine switches group.