ACI contains a plethora of information. Using SYSLOG to get that information from all of the devices in your fabric to a centralized SYSLOG server is still a good way of aggregating logging data, alerts, and audit information. In this post, we’ll review what data is available to be forwarded to your external SYSLOG server, how to configure ACI to send data to your external SYSLOG server, helpful troubleshooting commands, and caveats.

Reviewing Syslog

Before we dive in, lets review SYSLOG. SYSLOG specifies eight different severity levels for the message types that come in. All the way from SEV-0 messages indicating basically that your world is about to end, down to SEV-7 messages, which are very detailed debugging level messages. By default ACI will send SEV4 (Warning) and above SYSLOG messages; meaning that we will send all SEV0, SEV1, SEV2, SEV3, and SEV4 level events to the external SYSLOG server. What is sent from ACI is completely configurable, giving you the option to send as much (or as little) information about your ACI Fabric to your centralized SYSLOG system.

ACI Message Categories for SYSLOG Events

When it comes to information that we can send to SYSLOG from ACI, there are four main categories of information:

1. Faults – Faults that are generated by the system fall into one of several categories:
   1. Generic System issues
   2. Equipment is inoperable or has a functional issue
   3. Configuration related faults (system cannot push the config)
   4. Environmental issues (power, thermal, voltage)
   5. Network (Link down, etc)
2. Events – Holds records of system related events (i.e., link state transitions, Logged Contract hits)
3. Audit Logs – Records user-initiated events (i.e., logins, configuration changes)
4. Session Logs – Records session events (i.e., REST-client authentication updates for API sessions)

By default, the only selected category is Faults, but you can select which categories of messages you wish to export to SYSLOG events.

Configuring SYSLOG in ACI

Below are the high-level steps required to successfully configuring your ACI Fabric to send messages to an external SYSLOG server.

1. Ensure that your Out-of-Band -or- Inband Contracts permit UDP Port 514.
2. Create a SYSLOG Destination
3. Create ACI Fabric Monitoring Sources
   1. Select and Configure the appropriate Fabric-level Fabric SYSLOG events (Fabric > Fabric)
      1. Fabric > Fabric > Policies > Monitoring > Common Policy
      2. Fabric > Fabric > Policies > Monitoring > default
      3. (optional) Fabric > Fabric > Policies > Monitoring > Common Policy > Syslog Message Policies > Policy for system syslog messages
   2. Select and Configure the appropriate Access-level SYSLOG events (Fabric > Access)
   3. Select and Configure the appropriate Tenant-level SYSLOG events (Tenants > common > Policies > Monitoring > default

Step 1 – Configuring your Management Tenant Contracts to permit SYSLOG

If you have not yet configured Out-of-band Management for your ACI Fabric, do that first. If you need help configuring this, you can check out this article, Configuring Out-of-band access for your fabric.

NOTE – If you are using a permit any contract, then you can skip this step

When it comes to SYSLOG data and the Out-of-Band Management contracts, you technically do not need to configure an explicit filter to make SYSLOG work from an OOB Contracts perspective, however, it is considered a best practice to do so.

If you already have OOB or Inband Contracts defined inside of Tenant Mgmt, you will need to add the appropriate filters to your contracts for SYSLOG (UDP-514).

1. Add UDP-514 to the existing Out-of-Band Contract (if you are not using default/common or a permit any contract)
   1. Tenants > Tenant mgmt > Security Policies
   2. Expand Out-of-Band Contracts
      1. Edit existing OOB Contract
      2. Select OOB Subject
      3. Select OOB Filter
      4. Review filter and add (if necessary) UDP-514
   3. If you also have Inband Mgmt Connectivity configured, also verify the INB contract filter to permit UDP-514.
2. Verify you are providing Out-of-Band Contract in Tenant mgmt
   1. Tenant > Tenant mgmt > Node Management EPGs > Out-of-Band EPG default
   2. Under the “Provided Out-of-Band Contracts” in the policy window, provide the appropriate contract (this could be a the default/common contract, or a specific contract you have created and modified).
3. Verify you are consuming Out-of-Band Contract in Tenant mgmt
   1. Tenant > Tenant mgmt > External Management Network Instance Profiles > YourInstanceProfile
   2. Consume the same contract which you provided in the previous step
   3. Enter the subnets which are allowed to have access to the APIC (0.0.0.0/0 will permit all)

Step 2 – Creating a SYSLOG Destination

You can create an external SYSLOG destination by going to the Syslog folder, in the Admin > External Data Collectors > Monitoring Destinations > Syslog > YourSyslogServer section.

1. On the Syslog folder, right click and select “Create Syslog Monitoring Destination Group. On this screen, you will name the policy. You’ll also have an opportunity to change the defaults for Local File logging levels as well as Console Destination logging levels.

2. Next, you’ll define your remote (external) syslog destination.
   1. Provide the FQDN hostname or the IP address of your SYSLOG server
   2. Make sure the Admin state is set to enabled
   3. Define which severity level for your SYSLOG messages that you want sent to your remote destination (by default, this is set to “warnings”, however, in the example below, we have changed it to “information”).
   4. Verify your Management EPG.

Step 3 – Configuring ACI SYSLOG Sources

ACI separates the available monitoring sources into three categories:

• Fabric Monitoring Sources: Fabric ports, chassis, fans, linecards
• Access: Access ports, VMM-related alerts
• Tenant: VRF, BD, and EPG-related events, application profiles, etc.

If you want to receive SYSLOG events for all of the items above, you will need to configure ACI to send data from these sources independently.

Step 3a – Configuring the appropriate Fabric-level SYSLOG Sources from the GUI

1. Fabric > Fabric > Policies > Monitoring > Common Policy > Callhome/Smart Callhome/SNMP/Syslog/TACACs
   uni/fabric/moncommon
   1. Create Syslog Source
   2. Provide a name for this source (i.e., FabricCommonSyslog)
   3. Select the Minimum Syslog Severity Level (default is warning; we have changed this to information)
   4. Select the categories of messages to source (default is faults; we have selected all categories)
   5. Select the Destination Syslog Server (this is the server we previously defined)

2. Fabric > Fabric > Policies > Monitoring > default > Callhome/Smart Callhome/SNMP/Syslog/TACACs
   uni/fabric/monfab-default
   1. Create Syslog Source
   2. Provide a name for this source (i.e., FabricDefaultSyslog)
   3. Select the Minimum Syslog Severity Level (default is warning; we have changed this to information)
   4. Select the categories of messages to source (default is faults; we have selected all categories)
   5. Select the Destination Syslog Server (this is the server we previously defined)

Enabling the sending of ACL/Contract Log entries as SYSLOG events

Next, we will change the setting for “default” facility filter in the SYSLOG SYSTEM MESSAGEs to “informational. The main reason for changing the this setting is that this will allow ACI to send Contract Permit/Deny log messages as SYSLOG events to your SYSLOG server. If you learn more about Contract Logging with ACI, check out the Logging ACL/Contract Permits and Denies with ACI post!

3. Fabric > Fabric > Policies > Monitoring > Common Policy > Syslog Message Policies > Policy for system syslog messages
   uni/fabric/moncommon/sysmsgp
   1. Select the “default” syslog facility filter and change the severity level to “information”.

Step 3b – Configuring the appropriate Access-level SYSLOG Sources from the GUI

Fabric > Access > Policies > Monitoring > default > Callhome/Smart Callhome/SNMP/Syslog/TACACs
uni/infra/monifra-default

1. Create Syslog Source
2. Provide a name for this source (i.e., FabricAccessSyslog)
3. Select the Minimum Syslog Severity Level (default is warning; we have changed this to information)
4. Select the categories of messages to source (default is faults; we have selected all categories)
5. Select the Destination Syslog Server (this is the server we previously defined)

Step 3c – Configuring the appropriate Tenant-level SYSLOG Sources

The final source to configure is the messages that are generated from objects in the Tenant space. There are a couple of ways to configure this; you can either configure the SYSLOG source from the Common Tenant, and then select that default configuration from each of your used-defined Tenants, or you can create a separate SYSLOG source in each, respective tenant. We will be defining the Tenant-level SYSLOG source from the Common Tenant.

Create your Tenant Source Policy

Tenant > common > Policies > Monitoring > Callhome/Smart Callhome/SNMP/Syslog/TACACs
uni/tn-common/monepg-default

1. Create Syslog Source
2. Provide a name for this source (i.e., TNCommonSyslog)
3. Select the Minimum Syslog Severity Level (default is warning; we have changed this to information)
4. Select the categories of messages to source (default is faults; we have selected all categories)
5. Select the Destination Syslog Server (this is the server we previously defined)

 

Apply the Policy inside of your User-Defined Tenant

Tenant > UserTenant > Policy > Monitoring Policy

From the Tenant Policy screen, we’ll select the Monitoring Policy we defined in Tenant common. If you defined the source policy in your User-based Tenant, you could select that here as well.

Caveats

1. When using Out-of-band management for your ACI Fabric, it does not require that you explicitly enable UDP Port 514 (SYSLOG). However, it is still considered a best practice to configure the specific filter for UDP Port 514 (SYSLOG) and add it to your Out-of-band contract (if you have a permit-any contract, you are good).
2. When using Inband management for your ACI Fabric, the Inband management EPG DOES require the specific UDP Port 514 (Syslog) to be enabled.
3. If you want to enable the logging of Contract permit/deny events and send those to your SYSLOG server, you will have to change Facility filter for the default facility to informational.

Verification

Verify your SYSLOG configuration by using the “logit” command from the APIC

We can generate SYSLOG messages of differing severity levels from the APIC in order to test that messages are making to your SYSLOG server successfully. This is a great way to ensure (for example) that you have the appropriate severity level enabled from the ACI Fabric.

From your APIC
coast-apic1# logit severity 1 dest-grp SyslogServer server 10.18.188.110 “THIS IS A TEST“

From your SYSLOG server
[root@c6_CoastDhcp ~]# tail -f /var/log/messages
Aug 11 09:19:10 Aug 11 11:28:32.743 coast-apic2 %LOG_-6-SYSTEM_MSG [login,session][info][subj-[uni/userext/user-admin]/sess-8590507914] From-10.99.71.20-client-type-ssh-Success
Aug 11 09:19:17 Aug 11 11:28:39.572 coast-apic1 %LOG_-1-SYSTEM_MSG [E4210472][transition][info][sys] sent user message to syslog group:SyslogServer:THIS IS A TEST
Aug 11 09:19:19 Aug 11 11:28:42.351 coast-apic2 %LOG_-6-SYSTEM_MSG [refresh,session][info][subj-[uni/userext/user-admin]/sess-8590507915] From-10.18.188.189-client-type-REST-Success

 

Other Great Info

Check out the Technote by Tomas de Leon. Very detailed information and great troubleshooting information! – Technote: SYSLOG in the ACI Fabric

Need a quick and simple Syslog Server on Centos6? Check out this article!