Logging ACL/Contract Permits and Denies with ACI

Posted on by Jody

Screen Shot 2018-08-11 at 1.10.26 PM

Did you know that you can enable logging for permitted and denied traffic that flows through your ACI Fabric? While this feature is not meant as a replacement for Tetration or Netflow, this can be a great tool for troubleshooting and examining traffic that is flowing through applications that reside in your ACI Fabric.

In the example below, we are examining packets that are being logged against a “permit-any filter that exists between two specific EPGs in my Tenant.

Screen Shot 2018-08-11 at 12.58.25 PM.png
Tenant > UserTenant > Operational > Packets > L3 Permit

This data can be reviewed in the GUI (Tenant > UserTenant > Operational > Packets > L3 Permit), from the the CLI on Leaf switches (show logging ip access-list internal packet-log permit|deny), or by sending the Logging messages as SYSLOG events to a SYSLOG servers.

Prerequisites

  • If you want to send ACL/Contract logging entries as SYSLOG events, you must properly configure SYSLOG for your ACI Fabric. If you need an example of how to do this, check out the Configuring SYSLOG for ACI post!

Notables

  • Up to Cisco APIC, Release 3.2(1), the ACL permit and deny logs did not identify the EPGs associated with the contracts being logged. In release 3.2(1) the source EPG and destination EPG are added to the output of ACI permit and deny logs.

  • Logging for Contract Permits/Denies is enabled by Sending “Events” from the Fabric default Monitoring policies (Fabric > Fabric > Policies > Monitoring > default)
  • When sending Logging for Contract Permits/Denies to an External SYSLOG Server, the “default” facility must be configured for at least the “notification” severity.

Caveats

  • Contract Logging output for uSeg EPGs is not supported.
  • Contract Logging output or for EPGs used in shared services (including shared L3Outs) is not supported.
  • Using the log directive on filters in management contracts is not supported.
  • Contract permit and deny logging is available on -EX and later switches (-EX, -FX, -FX2). Contract permit and deny logging is not available on First Generation ACI switches.
  • Contract logging is limited to 500 packets per seconds (these packets are punted to the CPU on the Leaf switches)

Enabling the Contract Permit and Deny Logging option in the GUI

To enable Contract logging, just follow the steps below:

Tenant > Contracts (or Tenant > Security > Contracts)

  1. Create a Contract
    1. Add a Name for your contract
    1. Select the scope (default is VRF)
    1. Add a Subject for your contract
Screen Shot 2018-08-11 at 2.13.33 PM
Creating a Contract
  1. Specify a Subject
      1. Add a name for the Subject
      1. Add a filter
        1. In the directives drop-down list, select “Log”
        1. From the Action drop-down list, select either “Permit” or “Deny”
Screen Shot 2018-08-11 at 2.14.13 PM.png
Enabling Logging
  1. Consume the Contract you created between EPGs in your Tenant!

Examining Log messages

From the GUI

Tenant > UserTenant > Operational > Packets > L3 Permit|Deny

Screen Shot 2018-08-11 at 12.58.25 PM
Tenant > UserTenant > Operational > Packets > L3 Permit

From the Leaf CLI

Leaf202# show logging ip access-list internal packet-log permit
[ Sat Aug 11 13:47:15 2018 236946 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 443, DPort: 45076, Src Intf: port-channel1, Proto: 6, PktLen: 60

[ Sat Aug 11 13:47:15 2018 236483 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac: 0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 45076, DPort: 443, Src Intf: port-channel1, Proto: 6, PktLen: 74

[ Thu Aug 9 14:31:00 2018 891334 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 443, DPort: 45074, Src Intf: port-channel1, Proto: 6, PktLen: 60

[ Thu Aug 9 14:31:00 2018 890879 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac: 0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 45074, DPort: 443, Src Intf: port-channel1, Proto: 6, PktLen: 74

[ Thu Aug 9 14:30:51 2018 756855 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 80, DPort: 40672, Src Intf: port-channel1, Proto: 6, PktLen: 60

[ Thu Aug 9 14:30:51 2018 756374 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac: 0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 40672, DPort: 80, Src Intf: port-channel1, Proto: 6, PktLen: 74

[ Thu Aug 9 14:30:40 2018 246443 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 443, DPort: 45070, Src Intf: port-channel1, Proto: 6, PktLen: 60

[ Thu Aug 9 14:30:40 2018 246139 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac: 0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 45070, DPort: 443, Src Intf: port-channel1, Proto: 6, PktLen: 74

[ Thu Aug 9 14:30:38 2018 805988 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 80, DPort: 40668, Src Intf: port-channel1, Proto: 6, PktLen: 60

[ Thu Aug 9 14:30:38 2018 805696 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac: 0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 40668, DPort: 80, Src Intf: port-channel1, Proto: 6, PktLen: 74

From a Centralized SYSLOG server

[root@c6_CoastDhcp ~]# tail -f /var/log/messages

Aug 6 13:14:17 Aug 06 15:23:32.482 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac:0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 51162, DPort: 22, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.482 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac:0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 51162, DPort: 22, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.484 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 22, DPort: 51162, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.484 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 22, DPort: 51162, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.501 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 22, DPort: 51162, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.502 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 22, DPort: 51162, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.503 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac:0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 51162, DPort: 22, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.504 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac:0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 51162, DPort: 22, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.982 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac:0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 51162, DPort: 22, Src Intf: port-channel1, Proto: 6, PktLen: 102
Aug 6 13:14:17 Aug 06 15:23:32.983 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac:0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 51162, DPort: 22, Src Intf: port-channel1, Proto: 6, PktLen: 102
Aug 6 13:14:17 Aug 06 15:23:32.986 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 22, DPort: 51162, Src Intf: port-channel1, Proto: 6, PktLen: 66

6 thoughts on “Logging ACL/Contract Permits and Denies with ACI”

  1. Great article, thanks… Do you have any links to Cisco documentations that covers contract logging is limited to 500 packets per seconds? I’m unable to find this in my searching.

    Reply

    1. Mike – thanks! This info is not available externally on Cisco’s website to my knowledge. However, this is the current rate limiting for this feature.

      Reply

  2. Hi Jody, great article !! It seems that “deny” log is by default enabled. Is the “log” directive just for enabling permit ? I don’t see permit logging at all.

    Additionally even on Gen-1 switches, I am able to see deny logs.

    Reply

    1. Hey Peter! Thanks for reading. The default directive is “none”, at least in my testing with 4.0. You would have to enable the “log” directive for both deny and permits. Also, the logging functionality is only available for Gen-2 and later switches (EX, FX, FX2).

      Reply

  3. Works like a charm.. Thank you. Now I see the ACL permit/deny on the log server AND in the GUI. How can I limit the logging on GUI Events for ACL permits/deny, etc? I now have 22K pages of event logs that are mostly permit/deny logs.

    Reply

  4. thanks Jody, great one. I wonder where the logs are stored, or will it be archived somewhere?

    Reply

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.