Excluding log data by using exclusion rules in the web UI

In an IBM Log Analysis instance, you can configure exclusion rules through the logging web UI to stop logs from counting against your data usage quota and from being stored for search.

As of 28 March 2024 the IBM Log Analysis and IBM Cloud Activity Tracker services are deprecated and will no longer be supported as of 30 March 2025. Customers will need to migrate to IBM Cloud Logs, which replaces these two services, prior to 30 March 2025. For information about IBM Cloud Logs, see the IBM Cloud Logs documentation.

Complete the following steps to define an exclusion rule:

Verify that each exclusion rule that you add behaves as expected. Improper configured exclusion rules can result in storing data not intended for storage.

Launch the IBM Log Analysis web UI.
Click the Settings icon . Then select Usage > Exclusion Rules.
Select Add Rule. The Create Rule section opens.
Enter a name for the rule in the section What is this rule for?.
Enter the exclusion criteria. You can select 1 or more sources, 1 or more apps, enter a query, or a combination of sources, apps and query.

For example, to exclude all the lines from a specific source, select that source and leave the apps and query fields blank.

You might want to exclude logs from an app, then leave the sources and query fields blank, and enter an app.

You might want to exclude all the logs that are coming from a specific source and app. You must choose the source and app, and leave blank the query field.

You can enter a query to define the exclusion rule, or to refine the exclusion rule when you specify a source, an app, or both.
Select **Preserve these lines for live-tail and alerting ** to show through the live tail the log lines that are excluded. Notice that you can still use these log lines to set up an alert.
Click Save.
After you configure an exclusion rule, verify that the exclusion rule behaves as you expect.

Check the query in a custom view by entering the search criteria in the search bar of the Everything view, and validating that the data that is displayed is the data that you want excluded.

Sample 1: Exclude syslog data for a worker while keeping entries that report errors only

You will configure the rule so that you are able to see all log data through views and be able to define alerts on all the data.

Complete the following steps to define the exclusion rule:

Prereq: You must have a cluster configured to forward logs to a logging instance. Learn more.

Launch the IBM Log Analysis web UI.
Click the Settings icon . Then select Usage > Exclusion Rules.
Select Add Rule. The Create Rule section opens.
Enter a name for the rule in the section What is this rule for?. For example, enter Worker X no syslog data.
Enter the exclusion criteria. You can select 1 or more sources, 1 or more apps, enter a query, or a combination of sources, apps and query.

Click the Sources field. The list of options is displayed. Choose a worker. You can choose more than 1 source by clicking the field again and choosing a different source.

Click the Apps field. The list of options is displayed. Choose syslog.

In the Query section, enter -level:error to exclude all lines except the ones that report an error.
Select **Preserve these lines for live-tail and alerting ** to show through the live tail the log lines that are excluded. Notice that you can still use these log lines to set up an alert.
Click Save.

Check the query in a custom view by entering the search criteria in the search bar of the Everything view, and validating that the data that is displayed is the data that you want excluded.

Sample 2: Exclude kube-system data from the cluster while keeping entries that report errors only

You will configure the rule so that you are not able to see excluded log data through views.

Complete the following steps to define the exclusion rule:

Prereq: You must have a cluster configured to forward logs to a logging instance. Learn more.

Launch the IBM Log Analysis web UI.
Click the Settings icon . Then select Usage > Exclusion Rules.
Select Add Rule. The Create Rule section opens.
Enter a name for the rule in the section What is this rule for?. For example, enter Exclude log records from the namespace kube-system except error ones.
Enter the exclusion criteria. You can select 1 or more sources, 1 or more apps, enter a query, or a combination of sources, apps and query.

In the Query section, enter Namespace:kube-system -level:error to exclude all lines except the ones that report an error.
Leave unchecked the option Preserve these lines for live-tail and alerting.
Click Save.

Check the query in a custom view by entering the search criteria in the search bar of the Everything view, and validating that the data that is displayed is the data that you want excluded.