Template for tasks for migrating Log Analysis instances collecting logs in the account

For each instance

You can run the migration tool as follows:

Run the Migration Tool in a development or staging environment to test and validate the migration.

• When you migrate by using Terraform,

  1. Set the API key that is required to create resources.

     Set the API key that you must configure to create resources for migration.

     Run export IC_API_KEY=xxxxxx in the command line where you plan to run the Terraform CLI commands.

     If running in Windows, use set IC_API_KEY=xxxxxx instead.

  2. Check you have the Terraform CLI installed. For more information, see Installing the Terraform CLI.

• Run the migration tool

  ibmcloud logging migrate create-resources --scope instance --instance-crn xxx [--instance-name INSTANCENAME] [--instance-resource-group-id RESOURCEGROUPID] [--ecrn EVENT_NOTIFICATIONS_INSTANCE_CRN] [--api]|[-t -f]
  

  Add --api to migrate and create resources.

  Add -t -f to generate Terraform files and apply creation of resources. You are asked to confirm that you want to apply the scripts.

  You can add a new name for the instance that is created in Cloud Logs by adding the option --instance-name INSTANCENAME.

  You can change the resource group ID associated with the instance that is created in Cloud Logs by adding the option --instance-resource-group-id RESOURCEGROUPID.

  You can configure Event Notifications by adding destinations for your notification channels and topics and subscriptions to trigger alerts by adding the option --ecrn EVENT_NOTIFICATIONS_INSTANCE_CRN.

  This command will:

  • Migrate the instance, its resources such as views, dashboards, screens, alerts.

  • Check if archiving is enabled and migrate the archiving configuration. If it is, as part of the migration, 2 buckets are created to collect data and metrics.

  • Add IAM authroizations

    • Add an IAM authorization between Cloud Logs and the data bucket.

    • Add an IAM authorization between Cloud Logs and the metrics bucket.

    • Add an IAM authorization between Cloud Logs and the Event Notifications service.

    • Add an external integration in Cloud Logs to the Event Notifications instance.

  • Migrate notification channels (Slack, PagerDuty, WebHook) by creating the resources (topics, destinations, and subscriptions) to trigger alerts trough this channels in the IBM CLoud Events Notifications service.

  • IAM policies that apply to users, service IDs, trusted profiles, and access groups are migrated.

The Migration Tool only migrates configuration of selected resources.

Manual tasks

• Validate the queries migrated for views, and alerts.

• For dashboards, review the queries and design the layout of the widgets.

• If you have alerts and migrate by using Terraform, you must apply the Terraform files in /cl/accountID/manual-tf-files/event-notifications.

• If you migrate by using Terraform, you must apply the Terraform files in /cl/accountID/manual-tf-files/iam-policies/ to create IAM policies for the new Cloud Logs instance.

• IAM: For API keys (service ID / user ID), you need to recreate them and modify the applications that use it so they include permissions to work with the new services and resources.

• If you have parsing rules configured in the Log Analysis instance, you must manually recreate them in Cloud Logs. (In Cloud Logs, you must use Regex to parse the data.) For more information, see Extracting specific values as JSON keys.

• If you have log groups configured, you must manually migrate them to data access rules.

• If you have streaming configured, you must manually migrate the configuration. For more information, see Streaming data.

• Deploy/ Configure the Logging agent to collect and route logs to the Cloud Logs instance.

  • Deploy agent for Kubernetes clusters

  • Deploy agent for OpenShift clusters

  • Deploy agent for Linux servers

  • Deploy agent to collect and route rSyslog data

• Modify any runbooks for DevOps