Provisioning an IBM Cloud Logs instance by using Terraform
Terraform on IBM Cloud® enables predictable and consistent provisioning of IBM Cloud services so that you can rapidly build complex, multitier cloud environments that follow Infrastructure as Code (IaC) principles. Similar to using the IBM Cloud CLI or API and SDKs, you can automate the provisioning, update, and deletion of your IBM Cloud Logs instances by using HashiCorp Configuration Language (HCL).
Looking for a managed Terraform on IBM Cloud solution? Try out IBM Cloud® Schematics. With Schematics, you can use the Terraform scripting language that you are familiar with, but you don't need to worry about setting up and maintaining the Terraform command line and the IBM Cloud Provider plug-in. Schematics also provides pre-defined Terraform templates that you can install from the IBM Cloud catalog.
Before you begin, ensure that you have the required access to create and work with IBM Cloud Logs resources.
Step 1. Install the Terraform CLI
Complete the following steps to install the Terraform CLI:
-
Create a terraform folder on your local machine, and navigate to your terraform folder.
mkdir terraform && cd terraform
-
Download the Terraform version that you want. For example, you can download
terraform_1.12.2_darwin_amd64.zip
for a MacOS. -
Extract the Terraform zip file and copy the files to your terraform directory.
-
Set the environment PATH variable to your Terraform files.
export PATH=$PATH:<terraform-directory>/terraform
-
Verify that the installation is successful by using a terraform command.
./terraform
Step 2. Set up the directory
Create a folder and navigate into the folder. This folder is used to store all configuration files and variable definitions that are needed to create the IBM Cloud Logs instance.
For example, in your Terraform installation directory <terraform-directory>/terraform
, create the directory myproject
.
mkdir myproject && cd myproject
Step 3. Set up the IBM Cloud Provider plug-in
After the Terraform CLI installation is complete, you must set up the IBM Cloud Provider plug-in for Terraform so that you can start working with resources and services in IBM Cloud.
For a list of supported versions, see the IBM Cloud Provider plug-in releases.
Make sure that you use the latest IBM Cloud Provider plug-in release.
Create a versions.tf
file and specify the IBM Cloud Provider plug-in version that you want to use with the version
parameter.
terraform {
required_providers {
ibm = {
source = "IBM-Cloud/ibm"
version = "<provider version>"
}
}
}
For example:
terraform {
required_providers {
ibm = {
source = "IBM-Cloud/ibm"
version = ">=1.80.0"
}
}
}
Step 4. Configure the IBM Cloud Provider plug-in
After you complete the set up, you must configure the IBM Cloud Provider plug-in.
Before you can start working with Terraform on IBM Cloud, you must retrieve the credentials and parameters that are required for a Terraform resource or data source, and specify them in the provider
configuration. This configuration
is used by the IBM Cloud Provider plug-in to authenticate with the IBM Cloud platform and to view, create, update, or delete IBM Cloud resources and services.
The following table lists input parameters that you can set in the provider
block of your Terraform on IBM Cloud configuration file:
Input parameter | Required / optional | Description |
---|---|---|
ibmcloud_api_key |
Required | The IBM Cloud API key to authenticate with the IBM Cloud platform. For more information, about how to create an API key, see Creating an API key. You can
specify the API key in the provider block or retrieve the value from the IC_API_KEY or IBMCLOUD_API_KEY environment variables. If both environment variables are defined, IC_API_KEY takes precedence. |
ibmcloud_timeout |
Optional | The number of seconds that you want to wait until the IBM Cloud API is considered unavailable. The default value is 60 . You can specify the timeout in the provider block or retrieve the value from the IC_TIMEOUT or IBMCLOUD_TIMEOUT environment variables. If both variables are specified, IC_TIMEOUT takes precedence. |
region |
Optional | The IBM Cloud region where you want to create your resources. If this value is not specified, us-south is used by default. You can specify the region in the provider block or retrieve the value from the IBMCLOUD_REGION or IC_REGION environment variables. If both environment variables are specified, IC_REGION takes precedence. |
resource_group |
Optional | The ID of the resource group that you want to use for your IBM Cloud resources. To retrieve the ID, run ibmcloud resource groups . You can specify the resource group in the provider block or retrieve the value
from the IC_RESOURCE_GROUP or IBMCLOUD_RESOURCE_GROUP environment variables. If both environment variables are defined, IC_RESOURCE_GROUP takes precedence. |
For more information about how to use environment variables, see Using environment variables.
Configure input variables in the terraform.tfvars file
You can store information in a local terraform.tfvars
file and reference this data in your provider
block.
Do not commit the terraform.tfvars
into a public source repository. This file is meant to only be stored on your local machine.
In your project directory <terraform-directory>/terraform/myproject
, create a terraform.tfvars
file on your local machine and add the input parameters that are required for your resource or data source.
ibmcloud_api_key = "<ibmcloud_api_key>"
region = "region"
account_id = "<Account ID>"
rg_id = "d7c0e937c529461f90a19e1421f9746d"
plan = "standard"
cos_instance_crn = "crn:v1:bluemix:public:cloud-object-storage:global:a/<Account ID>:<COS instance ID>::"
cos_storage_class = "standard"
cos_bucket_data_name = "cloud-logs-tf-data"
cos_bucket_metrics_name = "cloud-logs-tf-metrics"
event_notifications_crn = "crn:v1:bluemix:public:event-notifications:eu-gb:a/<Account ID>:<Event Notifications instance ID>::"
event_notifications_instance_id = "<Event Notifications instance ID>"
event_notifications_region = "eu-gb"
Declare variables in the variables.tf file
In your project directory <terraform-directory>/terraform/myproject
, create a variables file that is named variables.tf
to include default values.
The following sample lists variables that you can use when you provision an IBM Cloud Logs instance:
variable "ibm_region" {
description = "Region to create resources. To see the list of valid regions, see https://cloud.ibm.com/docs/cloud-logs?topic=cloud-logs-regions."
type = string
default = "eu-gb"
}
variable "resource_group_name" {
type = string
description = "Resource group where resources are created"
default = "Default"
}
variable "rg_id" {
type = string
description = "Resource group ID where resources are created"
default = "b302120431c4456097f8970d80b93dfb"
}
variable "plan" {
type = string
description = "Service plan for Cloud Logs instances"
default = "standard"
}
variable "cos_instance_crn" {
type = string
description = "COS instance CRN where buckets are created"
default = "crn:v1:bluemix:public:cloud-object-storage:global:a/<Account ID>:<COS instance ID>::"
}
variable "cos_storage_class" {
type = string
description = "COS instance storage class"
default = "standard"
}
variable "account_id" {
type = string
description = "Account where resources are created"
default = "<Account ID>"
}
variable "cos_bucket_data_name" {
type = string
description = "Cloud Object Storage bucket data name"
default = "cloud-logs-tf-data"
}
variable "cos_bucket_metrics_name" {
type = string
description = "Cloud Object Storage bucket data name"
default = "cloud-logs-tf-metrics"
}
variable "event_notifications_crn" {
type = string
description = "Event Notifications CRN"
default = "crn:v1:bluemix:public:event-notifications:eu-gb:a/<Account ID>:<Event Notifications Instance ID>::"
}
variable "event_notifications_instance_id" {
type = string
description = "Event Notifications instance ID"
default = "<Event Notifications Instance ID>"
}
variable "event_notifications_region" {
type = string
description = "Event notifications region"
default = "eu-gb"
}
Reference variables in the provider.tf file
In your project directory <terraform-directory>/terraform/myproject
, create a provider.tf
file and use Terraform interpolation syntax to reference the variables from the terraform.tfvars
.
variable "ibmcloud_api_key" {}
variable "region" {}
provider "ibm" {
ibmcloud_api_key = var.ibmcloud_api_key
region = var.region
}
Step 5. Create the Terraform configuration files to provision an instance of IBM Cloud Logs
Next, create the following files:
-
data-bucket.tf
: Contains the Terraform resource definition to create a bucket to store log data.resource "ibm_cos_bucket" "data_bucket" { bucket_name = var.cos_bucket_data_name resource_instance_id = var.cos_instance_crn region_location = var.region storage_class = var.cos_storage_class }
-
metrics-bucket.tf
: Contains the Terraform resource definition to create a bucket to store metrics collected from log data.resource "ibm_cos_bucket" "metrics_bucket" { bucket_name = var.cos_bucket_metrics_name resource_instance_id = var.cos_instance_crn region_location = var.region storage_class = var.cos_storage_class }
-
main.tf
: Contains the Terraform resource definition to create the IBM Cloud Logs instance and attach the data bucket and the metrics bucket.data "ibm_resource_group" "group" { name = "marisa" } resource "ibm_resource_instance" "cloud_logs_instance" { name = "cloud-logs-via-tf" service = "logs" plan = var.plan location = var.region resource_group_id = data.ibm_resource_group.group.id parameters = { retention_period = "7" } } resource "null_resource" "update_instance_parameters" { triggers = { instance_id = ibm_resource_instance.cloud_logs_instance.id } provisioner "local-exec" { command = <<EOT ibmcloud login --apikey=$IC_API_KEY ibmcloud resource service-instance-update ${ibm_resource_instance.cloud_logs_instance.guid} -p '{"logs_bucket_crn": "${ibm_cos_bucket.data_bucket.crn}", "logs_bucket_endpoint": "${ibm_cos_bucket.data_bucket.s3_endpoint_direct}", "metrics_bucket_crn": "${ibm_cos_bucket.metrics_bucket.crn}","metrics_bucket_endpoint": "${ibm_cos_bucket.metrics_bucket.s3_endpoint_direct}"}' EOT } depends_on = [ibm_iam_authorization_policy.policy-cl-data-bucket,ibm_iam_authorization_policy.policy-cl-metrics-bucket] }
-
en_s2s.tf
: Contains the Terraform resource definition to create the service to service authorizations between the IBM Cloud Logs instance and the buckets.locals { cos_instance_id = split(":", ibm_cos_bucket.data_bucket.resource_instance_id)[7] } resource "ibm_iam_authorization_policy" "policy-cl-data-bucket" { source_service_name = "logs" source_resource_instance_id = ibm_resource_instance.cloud_logs_instance.guid roles = ["Writer"] resource_attributes { name = "serviceName" value = "cloud-object-storage" } resource_attributes { name = "serviceInstance" value = local.cos_instance_id operator = "stringEquals" } resource_attributes { name = "resourceType" value = "bucket" operator = "stringEquals" } resource_attributes { name = "resource" value = ibm_cos_bucket.data_bucket.bucket_name operator = "stringEquals" } resource_attributes { name = "accountId" value = var.account_id } } resource "ibm_iam_authorization_policy" "policy-cl-metrics-bucket" { source_service_name = "logs" source_resource_instance_id = ibm_resource_instance.cloud_logs_instance.guid roles = ["Writer"] resource_attributes { name = "serviceName" value = "cloud-object-storage" } resource_attributes { name = "serviceInstance" value = local.cos_instance_id operator = "stringEquals" } resource_attributes { name = "resourceType" value = "bucket" operator = "stringEquals" } resource_attributes { name = "resource" value = ibm_cos_bucket.metrics_bucket.bucket_name operator = "stringEquals" } resource_attributes { name = "accountId" value = var.account_id } }
-
event-notification-extension.tf
: Contains the Terraform resource definition to create an outbound integration between the IBM Cloud Logs instance and the IBM Cloud Event Notifications instance through which you send notifications to your destinations such as Slack or email.resource "ibm_logs_outgoing_webhook" "logs_outgoing_webhook_instance" { instance_id = ibm_resource_instance.cloud_logs_instance.guid region = ibm_resource_instance.cloud_logs_instance.location name = "Event-notification-cloud-logs-instance" type = "ibm_event_notifications" ibm_event_notifications { event_notifications_instance_id = var.event_notifications_instance_id region_id = var.event_notifications_region } depends_on = [ibm_iam_authorization_policy.policy-event-notifications] }
-
s2s.tf
: Contains the Terraform resource definition to create the authorization between the IBM Cloud Logs instance and the IBM Cloud Event Notifications instance.resource "ibm_iam_authorization_policy" "policy-event-notifications" { source_service_name = "logs" source_resource_instance_id = ibm_resource_instance.cloud_logs_instance.guid roles = ["Reader","Event Source Manager","Viewer"] description = "" target_service_name = "event-notifications" }
For additional information on how to use Terraform for IBM Cloud resources, see ibm_resource_instance
Step 6. Provision resources
Complete the following steps:
-
Initialize the Terraform CLI.
../terraform init
You should see the following message:
Terraform has been successfully initialized!
. -
Create a Terraform execution plan. The Terraform execution plan summarizes all the actions that need to be run to create the IBM Cloud Logs instance, the resource key, and IAM access policy in your account.
../terraform plan
-
Create the resources.
../terraform apply
To delete resources, run
./terraform destroy
.
What's next?
Verify that the resources are created. Launch the Observability UI and check the instance has been created.