Template to deploy the Logging agent using a Helm chart

Template versions

The following table lists the Helm chart version supported to deploy the Logging agent:

For information on recommended and supported Fluent Bit plug-ins see Fluent Bit support

Template to deploy the Logging agent using a Helm chart on an Openshift cluster

metadata:
  name: "logs-agent"
image:
  version: "1.3.0"  # required

clusterName: ""     # Enter the name of your cluster. This information is used to improve the metadata and help with your filtering.

env:
  # ingestionHost is a required field. For example:
  # ingestionHost: "<logs instance>.ingress.us-east.logs.cloud.ibm.com"
  ingestionHost: "" # required

  # If you are using private CSE proxy, then use port number "3443"
  # If you are using private VPE Gateway, then use port number "443"
  # If you are using the public endpoint, then use port number "443"
  ingestionPort: "" # required

  iamMode: "TrustedProfile"
  # trustedProfileID - trusted profile id - required for iam trusted profile mode
  trustedProfileID: "" # required if iamMode is set to TrustedProfile

scc:
  # true enables creation of Security Context Constraints in Openshift clusters
  create: true

defaultMetadata:
  # Configure to override the default subsystemName and applicationName that are used in the environment.
  subsystemName: ""  # The default value is set to the namespace that generated the log
  applicationName: ""  # The default value is set to the container name that generated the log

resources:
  # Configure this section to change the resources that are assigned to the agent container.
  limits:
    cpu: 500m
    ephemeral_storage: 10Gi
    memory: 3Gi
  requests:
    cpu: 100m
    ephemeral_storage: 2Gi
    memory: 1Gi

# Configure these parameters to include, exclude or restrict the set of logs that are processed by the agent
# By default, the agent will collect the logs from `/var/log/containers/*.log`.
# Each field is set as comma separated list, for example “/var/log/abc/*.log,/var/log/xyz/*.log”
additionalLogSourcePaths: "" # adds locations to the default set of logs that will be processed.
excludeLogSourcePaths: "" # ignores logs in the specified locations.
selectedLogSourcePaths: ""  # overrides the default path `/var/log/containers/*.log` and ignores the `additionalLogSourcePaths` configurations

# Configure this parameter to control the IAM endpoint used by the agent to exchange the tokens.
# The default value is `Production`.
# Valid values are :
# Set `Production` to use the iam.cloud.ibm.com default endpoint
# Set ProductionPrivate to use the private.iam.cloud.ibm.com endpoint
iamEnvironment: "Production"

# Configure this parameter to change the setting for the Kubernetes filter to include the annotations from Kubernetes with the log records.
# The default value for this setting is `false`.
includeAnnotations: true

# Configure this parameter to control the number of times the agent will retry sending data if an error occurs that is considered to be retryable.
# The default is `False`.
# For more information, see the [Fluentbit documentation about retries](https://docs.fluentbit.io/manual/administration/scheduling-and-retries) to understand the implications of setting this value.
retryLimit: False

# Configure additional tags as key/value pair tags that can be added as metadata to every log line.
additionalMetadata:
#  region: ca-tor
#  env: production

# Configure the level of logging
# Default value is info
# Valid values are: debug, info, or error
loggingLevel: info

Template to deploy the Logging agent using a Helm chart on a Kubernetes cluster

metadata:
  name: "logs-agent"
image:
  version: "1.3.0"  # required

clusterName: ""     # Enter the name of your cluster. This information is used to improve the metadata and help with your filtering.

env:
  # ingestionHost is a required field. For example:
  # ingestionHost: "<logs instance>.ingress.us-east.logs.cloud.ibm.com"
  ingestionHost: "" # required

  # If you are using private CSE proxy, then use port number "3443"
  # If you are using private VPE Gateway, then use port number "443"
  # If you are using the public endpoint, then use port number "443"
  ingestionPort: "" # required

  iamMode: "TrustedProfile"
  # trustedProfileID - trusted profile id - required for iam trusted profile mode
  trustedProfileID: "" # required if iamMode is set to TrustedProfile

scc:
  # true enables creation of Security Context Constraints in Openshift clusters
  # set to false for Kubernetes clusters
  create: false

defaultMetadata:
  # Configure to override the default subsystemName and applicationName that are used in the environment.
  subsystemName: ""  # The default value is set to the namespace that generated the log
  applicationName: ""  # The default value is set to the container name that generated the log

resources:
  # Configure this section to change the resources that are assigned to the agent container.
  limits:
    cpu: 500m
    ephemeral_storage: 10Gi
    memory: 3Gi
  requests:
    cpu: 100m
    ephemeral_storage: 2Gi
    memory: 1Gi

# Configure these parameters to include, exclude or restrict the set of logs that are processed by the agent
# By default, the agent will collect the logs from `/var/log/containers/*.log`.
# Each field is set as comma separated list, for example “/var/log/abc/*.log,/var/log/xyz/*.log”
additionalLogSourcePaths: "" # adds locations to the default set of logs that will be processed.
excludeLogSourcePaths: "" # ignores logs in the specified locations.
selectedLogSourcePaths: ""  # overrides the default path `/var/log/containers/*.log` and ignores the `additionalLogSourcePaths` configurations

# Configure this parameter to control the IAM endpoint used by the agent to exchange the tokens.
# The default value is `Production`.
# Valid values are :
# Set `Production` to use the iam.cloud.ibm.com default endpoint
# Set ProductionPrivate to use the private.iam.cloud.ibm.com endpoint
iamEnvironment: "Production"

# Configure this parameter to change the setting for the Kubernetes filter to include the annotations from Kubernetes with the log records.
# The default value for this setting is `false`.
includeAnnotations: true

# Configure this parameter to control the number of times the agent will retry sending data if an error occurs that is considered to be retryable.
# The default is `False`.
# For more information, see the [Fluentbit documentation about retries](https://docs.fluentbit.io/manual/administration/scheduling-and-retries) to understand the implications of setting this value.
retryLimit: False

# Configure additional tags as key/value pair tags that can be added as metadata to every log line.
additionalMetadata:
#  region: ca-tor
#  env: production

# Configure the level of logging
# Default value is info
# Valid values are: debug, info, or error
loggingLevel: info

Helm Chart Configuration Options

The following table contains a list of the parameters that you can configure in the logs-values.yaml file to adjust the Logging agent configurations:

env.iamMode

Configure this parameter to choose the authentication method to use by the agent when sending logs to an IBM Cloud Logs instance.

• You can choose an IAM APIKey or a Trusted Profile configuration.
• Valid values are: TrustedProfile or IAMAPIKey
• The default value is a Trusted Profile configuration.

The entry in the logs-values.yaml file looks as follows:

env:
  iamMode: IAMAPIKey

Consider the following information when setting this parameter:

• If env.iamMode: "TrustedProfile" is set, then the env.trustedProfileID variable must also be provided.

• If env.iamMode: "IAMAPIKey" is set, then the configuration expects a secret to be defined that contains an IAM Apikey with permissions.

  If the secret.iamAPIKey variable is provided on the helm command (ie. --set secret.iamAPIKey=<your iamAPIKey>), then the helm chart will create the Kubernetes secret.

  Alternatively, you can create the secret ahead of time with the command: (Make sure you are connected to your cluster.)

  kubectl create secret generic <helm install-name> -n ibm-observe --from-literal=IAM_API_KEY=<apikey>
  

defaultMetadata

This section allows the user to override the default subsystemName and applicationName that are used in the environment. By default, the values are not set and the output plugin will dynamically set the values to:

• subsystemName: the Kubernetes namespace that generated the log
• applicationName: the container name that generated the log

The entry in the logs-values.yaml file looks as follows:

defaultMetadata:
  subsystemName: ""
  applicationName: ""

resources

This section allows the user to change the resources that are assigned to the Logging agent container.

The entry in the logs-values.yaml file looks as follows and sets the following default values:

resources:
  limits:
    cpu: 500m
    ephemeral_storage: 10Gi
    memory: 3Gi
  requests:
    cpu: 100m
    ephemeral_storage: 2Gi
    memory: 1Gi

If you need to update any of the values, the entire configuration must be provided even if you don't update all of the values.

Log Source Paths configurations

By default the agent will collect the logs from /var/log/containers/*.log.

The following additional variables can be provided to include, exclude or restrict the set of logs to be processed:

• additionalLogSourcePaths adds locations to the default set of logs that will be processed.
• excludeLogSourcePaths ignores logs in the specified locations.
• selectedLogSourcePaths overrides the default path /var/log/containers/*.log and ignores the additionalLogSourcePaths configurations. Only the files that are set through this parameter are collected by the agent.

You can define multiple paths by using a comma separated list, for example “/var/log/abc/.log,/var/log/xyz/.log”.

The entry in the logs-values.yaml file looks as follows:

# comma separated list, for example “/var/log/abc/*.log,/var/log/xyz/*.log”
additionalLogSourcePaths: ""
excludeLogSourcePaths: ""
selectedLogSourcePaths: ""

iamEnvironment

This configuration controls the IAM endpoint used by the agent to exchange the tokens.

The default value is Production.

Valid values are :

• Set Production to use the iam.cloud.ibm.com default endpoint
• Set ProductionPrivate to use the private.iam.cloud.ibm.com endpoint

The entry in the logs-values.yaml file looks as follows:

iamEnvironment: "Production"

includeAnnotations

This configuration changes the setting for the Kubernetes filter to include the annotations from Kubernetes with the log records.

The default value for this setting is false.

The entry in the logs-values.yaml file looks as follows:

includeAnnotations: true

retryLimit

This configuration places a limit on the number of times the agent will retry sending data if an error occurs that is considered to be retryable.

The default is False.

For more information, see the Fluentbit documentation about retries to understand the implications of setting this value.

In some situations this setting could lead to log data being discarded by the agent due to the inability to send.

The entry in the logs-values.yaml file looks as follows:

retryLimit: 8

additionalMetadata

This is a list of key/value pairs that will be added under the meta object to permit additional tags.

The entry in the logs-values.yaml file looks as follows:

additionalMetadata:
  region: ca-tor
  env: production

The above example will result in the following additional fields added to each log line in IBM Cloud Logs:

{
  "meta": {
    "region": "ca-tor",
    "env": "production"
  }
}