Did you know that you can enable logging for permitted and denied traffic that flows through your ACI Fabric? While this feature is not meant as a replacement for Tetration or Netflow, this can be a great tool for troubleshooting and examining traffic that is flowing through applications that reside in your ACI Fabric.
In the example below, we are examining packets that are being logged against a “permit-any filter that exists between two specific EPGs in my Tenant.
This data can be reviewed in the GUI (Tenant > UserTenant > Operational > Packets > L3 Permit), from the the CLI on Leaf switches (show logging ip access-list internal packet-log permit|deny), or by sending the Logging messages as SYSLOG events to a SYSLOG servers.
Prerequisites
- If you want to send ACL/Contract logging entries as SYSLOG events, you must properly configure SYSLOG for your ACI Fabric. If you need an example of how to do this, check out the Configuring SYSLOG for ACI post!
Notables
-
Up to Cisco APIC, Release 3.2(1), the ACL permit and deny logs did not identify the EPGs associated with the contracts being logged. In release 3.2(1) the source EPG and destination EPG are added to the output of ACI permit and deny logs.
- Logging for Contract Permits/Denies is enabled by Sending “Events” from the Fabric default Monitoring policies (Fabric > Fabric > Policies > Monitoring > default)
- When sending Logging for Contract Permits/Denies to an External SYSLOG Server, the “default” facility must be configured for at least the “notification” severity.
Caveats
- Contract Logging output for uSeg EPGs is not supported.
- Contract Logging output or for EPGs used in shared services (including shared L3Outs) is not supported.
- Using the log directive on filters in management contracts is not supported.
- Contract permit and deny logging is available on -EX and later switches (-EX, -FX, -FX2). Contract permit and deny logging is not available on First Generation ACI switches.
- Contract logging is limited to 500 packets per seconds (these packets are punted to the CPU on the Leaf switches)
Enabling the Contract Permit and Deny Logging option in the GUI
To enable Contract logging, just follow the steps below:
Tenant > Contracts (or Tenant > Security > Contracts)
- Create a Contract
-
- Add a Name for your contract
-
- Select the scope (default is VRF)
-
- Add a Subject for your contract
- Specify a Subject
-
-
- Add a name for the Subject
-
-
-
- Add a filter
-
-
-
-
- In the directives drop-down list, select “Log”
-
-
-
-
-
- From the Action drop-down list, select either “Permit” or “Deny”
-
-
- Consume the Contract you created between EPGs in your Tenant!
Examining Log messages
From the GUI
Tenant > UserTenant > Operational > Packets > L3 Permit|Deny
From the Leaf CLI
Leaf202# show logging ip access-list internal packet-log permit
[ Sat Aug 11 13:47:15 2018 236946 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 443, DPort: 45076, Src Intf: port-channel1, Proto: 6, PktLen: 60
[ Sat Aug 11 13:47:15 2018 236483 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac: 0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 45076, DPort: 443, Src Intf: port-channel1, Proto: 6, PktLen: 74
[ Thu Aug 9 14:31:00 2018 891334 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 443, DPort: 45074, Src Intf: port-channel1, Proto: 6, PktLen: 60
[ Thu Aug 9 14:31:00 2018 890879 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac: 0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 45074, DPort: 443, Src Intf: port-channel1, Proto: 6, PktLen: 74
[ Thu Aug 9 14:30:51 2018 756855 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 80, DPort: 40672, Src Intf: port-channel1, Proto: 6, PktLen: 60
[ Thu Aug 9 14:30:51 2018 756374 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac: 0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 40672, DPort: 80, Src Intf: port-channel1, Proto: 6, PktLen: 74
[ Thu Aug 9 14:30:40 2018 246443 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 443, DPort: 45070, Src Intf: port-channel1, Proto: 6, PktLen: 60
[ Thu Aug 9 14:30:40 2018 246139 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac: 0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 45070, DPort: 443, Src Intf: port-channel1, Proto: 6, PktLen: 74
[ Thu Aug 9 14:30:38 2018 805988 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 80, DPort: 40668, Src Intf: port-channel1, Proto: 6, PktLen: 60
[ Thu Aug 9 14:30:38 2018 805696 usecs]: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac: 0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 40668, DPort: 80, Src Intf: port-channel1, Proto: 6, PktLen: 74
From a Centralized SYSLOG server
[root@c6_CoastDhcp ~]# tail -f /var/log/messages
Aug 6 13:14:17 Aug 06 15:23:32.482 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac:0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 51162, DPort: 22, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.482 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac:0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 51162, DPort: 22, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.484 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 22, DPort: 51162, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.484 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 22, DPort: 51162, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.501 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 22, DPort: 51162, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.502 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 22, DPort: 51162, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.503 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac:0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 51162, DPort: 22, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.504 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac:0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 51162, DPort: 22, Src Intf: port-channel1, Proto: 6, PktLen: 66
Aug 6 13:14:17 Aug 06 15:23:32.982 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac:0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 51162, DPort: 22, Src Intf: port-channel1, Proto: 6, PktLen: 102
Aug 6 13:14:17 Aug 06 15:23:32.983 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: BD_EXT_VLAN, Vlan-Id: 29, SMac:0x00defb798d43, DMac:0x0022bdf819ff, SIP: 2.2.2.107, DIP: 101.1.1.101, SPort: 51162, DPort: 22, Src Intf: port-channel1, Proto: 6, PktLen: 102
Aug 6 13:14:17 Aug 06 15:23:32.986 Leaf202 %LOG_-6-SYSTEM_MSG [E4204936][transition][info][sys] %ACLLOG-5-ACLLOG_PKTLOG_PERMIT: CName: COAST:COAST_vrf(VXLAN: 2097152), VlanType: FD_VLAN, Vlan-Id: 21, SMac: 0x005056938878, DMac:0x0022bdf819ff, SIP: 101.1.1.101, DIP: 2.2.2.107, SPort: 22, DPort: 51162, Src Intf: port-channel1, Proto: 6, PktLen: 66