Exporting events programmatically using the V1 API

From an IBM Cloud Activity Tracker instance, you can export events programmatically by using the V1 Export REST API.

As of 28 March 2024 the IBM Log Analysis and IBM Cloud Activity Tracker services are deprecated and will no longer be supported as of 30 March 2025. Customers will need to migrate to IBM Cloud Logs, which replaces these two services, prior to 30 March 2025. For information about IBM Cloud Logs, see the IBM Cloud Logs documentation.

The V1 Export REST API does not support pagination. If you need pagination support, you will need to use the V2 export REST API.

Consider the following information when you export events:

You can export a set of event entries. To define the set of data that you want to export, you can apply filter and searches. You can also specify the time range.
When you export events programmatically, you can choose to send an email or to write events into your terminal.
The compressed events file that contains the data that you want to export is available for a maximum of 12 hours.
When you export events, you have a limit of lines that you can export in a request. You can specify to export older lines or newer lines in case you reach the limit in the time range that you specify for the export. The maximum number of lines you can export for a 7-day plan is 10,000 lines. The maximum number of lines you can export for all other plans is 20,000 lines.

Prerequisites

You must have a paid service plan for the IBM Cloud Activity Tracker service. Learn more.
You must have a user ID that has permissions to launch the web UI, view or manage service keys, and view events. Learn more.
You must check that the auditing instance has the export feature enabled.

Export API

Use ENDPOINT/v1/export?QUERY_PARAMETERS" -u SERVICE_KEY: to export events.

ENDPOINT represents the entry point to the service. Each region has a different URL. To export events from an auditing instance, see Endpoints.

QUERY_PARAMETERS are parameters that define the filtering criteria that is applied to the export request.

SERVICE_KEY is an API key that you must use to validate your credentials with the auditing instance.

The : after SERVICE_KEY is required.

Query parameters

You can define query parameters to refine the events that you want to export.

The following table lists the query parameters that you can set:

Query parameters
Query parameter	Type	Status	Description
`from`	`int32`	Required	Start time. Set as UNIX timestamp in seconds or milliseconds.
`to`	`int32`	Required	End time. Set as UNIX timestamp in seconds or milliseconds.
`size`	`string`	Optional	Number of event lines to include in the export.
`hosts`	`string`	Optional	Comma-separated list of services. To get the value that you can set for a specific service, you can check the label `Source` in an event that is generated by the service.
`apps`	`string`	Optional	Comma-separated list of service instances represented as CRNs. To get the value that you can set for a specific service, you can check the label `App` in an event that is generated by the service.
`levels`	`string`	Optional	Comma-separated list of severities. Use this field to set the severity of the events that you want to export.
`query`	`string`	Optional	Search query. You can test your query in a view in the UI. Then, when you copy the query to use for the export, replace each blank space in the query with `%20`.
`prefer`	`string`	Optional	Defines the log lines that you want to export. Valid values are `head`, first log lines, and `tail`, last log lines. If not specified, defaults to tail.
`email`	`string`	Optional	Specifies the email with the downloadable link of your export. By default, the event lines are streamed.
`emailSubject`	`string`	Optional	Use to set the subject of the email. Use `%20` to represent a space. For example, a sample value is `Export%20logs`.

When you include a query or a subject to an email, use %20 to represent a space.

For example, you can define a set of parameters to include information:

ENDPOINT/v1/export?to=START_TIME&from=END_TIME&hosts=SERVICE_LIST&levels=SEVERITY_LIST&size=N&query=(SEARCH_QUERY)" -u $TOKEN:

Exporting events

Complete the following steps to export events programmatically:

Step 1. Get a service key

Get a service key.

Step 2. Identify the data to pass through the export parameters

When you use the export API, you can define 1 or more parameters to refine the set of events that you export. For example, consider the following parameters:

levels: This parameter is used to filter events based on the criticality of a request.
query: This parameter is used to define the search query that is used to filter events.
hosts: This parameter is used to list the services from which you want to export data.

To verify that the query that you enter returns the set of events that you are looking for, define the search query through the UI.

Consider the following information when you define the search query through the UI

Select the sources.

Sources represent IBM Cloud services.

The values that you identify will be required to set the hosts parameter.

If you set the host field in the query, this parameter is not required.
Select the levels.

In Activity Tracker, level maps to the severity event field. Severity defines the level of threat an action may have on the IBM Cloud.

Valid severity values are critical, warning, and normal.

If you set the field severity in the query, this parameter is not required.
Define the search query.

To get more information on how to search, see Select the set of events to display through a view by applying a search query.

Refine the query until you can only see the events that you want to export.

Step 3. Map the data to the query parameters

To define the parameters that you need for the export request, complete the following steps:

Map your sources to the hosts parameter. The hosts parameter is a comma-separated list of services.
Map the severity to the levels parameter. The levels parameter is a comma-separated list of severity values.

Map the query to the query parameter.

Notice that when you copy the query from the UI, you must replace every space with %20, every [ with %5B, and every ] with %5D.

For example, if the query from the UI is:

action:iam-groups.group.create OR action:iam-groups.group.delete OR action:iam-groups.group.update OR action:iam-identity.account-profile.create OR action:iam-identity.account-profile.delete OR action:iam-identity.account-profile.update OR action:iam-am.policy.create OR action:iam-am.policy.delete OR action:iam-am.policy.update

You would specify the query as this in the API:

action:iam-groups.group.create%20OR%20action:iam-groups.group.delete%20OR%20action:iam-groups.group.update%20OR%20action:iam-identity.account-profile.create%20OR%20action:iam-identity.account-profile.delete%20OR%20action:iam-identity.account-profile.update%20OR%20action:iam-am.policy.create%20OR%20action:iam-am.policy.delete%20OR%20action:iam-am.policy.update

For this query from the UI:

"q": "action:[iam-groups.group.create.iam-groups.group.delete,iam-groups.group.update,iam-identity.account-profile.create,iam-identity.account-profile.delete,iam-identity.account-profile.update,iam-am.policy.create,iam-am.policy.delete,iam-am.policy.update]"

You would run the following command:

curl -XGET  'https://api.eu-de.logging.cloud.ibm.com/v1/export?from=1690365540000&to=1690451940000&hosts=iam-identity,iam-groups,iam-am&query=action:%5Biam-groups.group.create,iam-groups.group.delete,iam-groups.group.update,iam-identity.account-profile.create,iam-identity.account-profile.delete,iam-identity.account-profile.update,iam-am.policy.create,iam-am.policy.delete,iam-am.policy.update%5D' --header 'servicekey:     xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

Step 4. Export the events

Run the following cURL command to export events:

curl "ENDPOINT/v1/export?QUERY_PARAMETERS" -u SERVICE_KEY:

Where

ENDPOINT represents the entry point to the service. Each region has a different URL. Learn more.
QUERY_PARAMETERS are parameters that define the filtering criteria that is applied to the export request.
SERVICE_KEY is the service key that you created in the previous step.

Samples

To query events for a service in us-south, you can run the following command:

curl "https://api.us-south.logging.cloud.ibm.com/v1/export?to=1592337600&from=1592179200&hosts=kms&size=10&query=(target.id:crn:v1:bluemix:public:kms:us-south:a/xxxxxxx:xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx%20(action:kms.secrets.create%20%20OR%20action:kms.secrets.delete%20))" -u $TOKEN:

To write events into the terminal, you can run the following command:

curl "https://api.us-south.logging.cloud.ibm.com/v1/export?to=$(date +%s)000&from=$(($(date +%s)-86400))000" -u $TOKEN:

To send an email with the link to download the events that are specified on the export, you can run the following command:

curl "https://api.us-south.logging.cloud.ibm.com/v1/export?to=$(date +%s)000&from=$(($(date +%s)-86400))000&email=joe@ibm.com" -u $TOKEN:

To send an email with a custom subject, you can run the following command:

curl "https://api.us-south.logging.cloud.ibm.com/v1/export?to=$(date +%s)000&from=$(($(date +%s)-86400))000&email=joe@ibm.com&emailSubject=Export%20test" -u $TOKEN: