Changes with Helm 1.6.0

New configuration format: The Helm chart now uses the YAML-based configuration format instead of the classic .conf style.

Backward compatibility: You can still use your existing logs-values.yaml file from version 1.5.x with version 1.6.0. The transition is designed to be non-disruptive.

Deprecated The classic configuration mode will be officially unsupported by Fluent Bit by the end of 2025. We recommend migrating to the YAML format as soon as possible to stay aligned with future updates. You can continue to use the previous versions of the Helm charts that were based on the classic configuration mode with newer versions of the Fluent Bit agent, but by the end of 2025 the classic configuration mode will be unsupported by Fluent Bit.

Template to deploy the Logging agent using a Helm chart on an Red Hat OpenShift on IBM Cloud cluster

metadata:
  name: "logs-agent"
image:
  version: "1.6.0"  # required

clusterName: "" # recommended to improve the metadata

env:
  # ingestionHost is a required field. For example:
  # ingestionHost: "<logs instance>.ingress.us-east.logs.cloud.ibm.com"
  ingestionHost: "" # required

  # If you are using CSE proxy, then use port number "3443"
  # If you are using private VPE Gateway, then use port number "443"
  # If you are using the public endpoint, then use port number "443"
  ingestionPort: "" # required

  iamMode: "TrustedProfile"
  # trustedProfileID - trusted profile id - required for iam trusted profile mode
  trustedProfileID: "" # required if iamMode is TrustedProfile

scc:
  # true here enables creation of Security Context Constraints in Openshift
  create: true

Template to deploy the Logging agent using a Helm chart on a IBM Cloud Kubernetes Service cluster

metadata:
  name: "logs-agent"
image:
  version: "1.6.0"  # required

clusterName: "" # recommended to improve the metadata

env:
  # ingestionHost is a required field. For example:
  # ingestionHost: "<logs instance>.ingress.us-east.logs.cloud.ibm.com"
  ingestionHost: "" # required

  # If you are using CSE proxy, then use port number "3443"
  # If you are using private VPE Gateway, then use port number "443"
  # If you are using the public endpoint, then use port number "443"
  ingestionPort: "" # required

  iamMode: "TrustedProfile"
  # trustedProfileID - trusted profile id - required for iam trusted profile mode
  trustedProfileID: "" # required if iamMode is TrustedProfile

New features with Helm 1.6.0

additionalLogSourcePaths

Deprecated

The additionalLogSourcePaths option will be removed in an upcoming Helm release. It was originally intended to be used for processing host files or application log files in non-container locations. These files were either inappropriately processed by the Kubernetes plug-in, or ignored with a warning - neither being efficient.

To improve parsing of the system files, a new configuration named systemLogs has been implemented to provide a better parsing experience for these host system files. See the new systemLogs configuration for details.

For non-system or non-container logs the additionalInputs section can provide a customized input section within the pipeline, giving full control over the input and processors for your files.

Helm Chart configuration options

The following table contains a list of the parameters that you can configure in the logs-values.yaml file to adjust the Logging agent configurations:

env.iamMode

Configure this parameter to choose the authentication method to use by the agent when sending logs to an IBM Cloud Logs instance.

• You can choose an IAM APIKey or a Trusted Profile configuration.
• Valid values are: TrustedProfile or IAMAPIKey
• The default value is a Trusted Profile configuration.

The entry in the logs-values.yaml file looks as follows:

env:
  iamMode: IAMAPIKey

Consider the following information when setting this parameter:

• If env.iamMode: "TrustedProfile" is set, then the env.trustedProfileID variable must also be provided.

• If env.iamMode: "IAMAPIKey" is set, then the configuration expects a secret to be defined that contains an IAM Apikey with permissions.

  If the secret.iamAPIKey variable is provided on the helm command (for example --set secret.iamAPIKey=<your iamAPIKey>), then the helm chart will create the Kubernetes secret.

  Alternatively, you can create the secret ahead of time with the command: (Make sure you are connected to your cluster.)

  kubectl create secret generic <helm install-name> -n ibm-observe --from-literal=IAM_API_KEY=<apikey>
  

defaultMetadata

This section allows the user to override the default subsystemName and applicationName that are used in the environment. By default, the values are not set and the output plug-in will dynamically set the values to:

• subsystemName: the Kubernetes namespace that generated the log
• applicationName: the container name that generated the log

The entry in the logs-values.yaml file looks as follows:

defaultMetadata:
  subsystemName: ""
  applicationName: ""

resources

This section allows the user to change the resources that are assigned to the Logging agent container.

The entry in the logs-values.yaml file looks as follows and sets the following default values:

resources:
  limits:
    cpu: 500m
    ephemeral_storage: 10Gi
    memory: 3Gi
  requests:
    cpu: 100m
    ephemeral_storage: 2Gi
    memory: 1Gi

If you need to update any of the values, the entire configuration must be provided even if you don't update all of the values.

Log source paths configurations

By default the agent will collect the logs from /var/log/containers/*.log.

The following additional variables can be provided to include, exclude or restrict the set of logs to be processed:

• additionalLogSourcePaths adds locations to the default set of logs that will be processed.
• excludeLogSourcePaths ignores logs in the specified locations.
• selectedLogSourcePaths overrides the default path /var/log/containers/*.log and ignores the additionalLogSourcePaths configurations. Only the files that are set through this parameter are collected by the agent.

You can define multiple paths by using a comma separated list, for example /var/log/abc/*.log,/var/log/xyz/*.log.

The entry in the logs-values.yaml file looks as follows:

# comma separated list, for example “/var/log/abc/*.log,/var/log/xyz/*.log”
additionalLogSourcePaths: ""
excludeLogSourcePaths: ""
selectedLogSourcePaths: ""

env.iamEnvironment

This configuration controls the IAM endpoint used by the agent to exchange the tokens.

The default value is Production.

Valid values are :

• Set Production to use the iam.cloud.ibm.com default endpoint
• Set PrivateProduction to use the private.iam.cloud.ibm.com endpoint
• Set Custom to use a custom IAM endpoint (for example private.eu-de.iam.cloud.ibm.com)

The entry in the logs-values.yaml file looks as follows:

env:
  iamEnvironment: "Production"

For the Custom iamEnvironment setting, the iamHost must also be provided.

env:
  iamEnvironment: "Custom"
  iamHost: "private.eu-de.iam.cloud.ibm.com"

includeAnnotations

This configuration changes the setting for the Kubernetes filter to include the annotations from Kubernetes with the log records.

The default value for this setting is false.

The entry in the logs-values.yaml file looks as follows:

includeAnnotations: true

retryLimit

This configuration places a limit on the number of times the agent will retry sending data if an error occurs that is considered to be retryable.

The default is False.

For more information, see the Fluentbit documentation about retries to understand the implications of setting this value.

In some situations this setting could lead to log data being discarded by the agent due to the inability to send.

The entry in the logs-values.yaml file looks as follows:

retryLimit: 8

additionalMetadata

This is a list of key/value pairs that will be added under the meta object to permit additional tags.

The entry in the logs-values.yaml file looks as follows:

additionalMetadata:
  region: ca-tor
  env: production

The previous example will result in the following additional fields added to each log line in IBM Cloud Logs:

{
  "meta": {
    "region": "ca-tor",
    "env": "production"
  }
}

keepParsedLogs

This is a boolean flag that determines whether to retain the original log field after it has been successfully processed by the Kubernetes plug-in.

The default is false.

Enabling this will result in duplicate log data, which may increase storage usage.

Example configuration:

keepParsedLogs: true

outputWorkers

This setting defines the number of worker threads used by the output plug-in to enable parallel processing of log data.

The default is 4.

In most environments, increasing the number of threads beyond 4 does not significantly impact performance. If your log volume is high and you’ve allocated more than 1 CPU, increasing this value to 8 can help improve output throughput.

For low-volume enviroments, reducing this value to 1 or 2 will limit the maximum log volume that can be sent to IBM Cloud Logs, which may be suitable for smaller workloads.

outputWorkers: 2

severityFieldName

This is an optional setting that allows you to override the default severity field sent to IBM Cloud Logs. Use this option if your log messages use a non-standard field name (for example, Log_Level, logSeverity, and so on) to indicate severity.

When configured:

• A new filter is added to the pipeline.
• This filter injects a severity field into each log record.
• The value of severity is populated from the custom field you specify.

severityFieldName: Log_Level

tolerations

In some Kubernetes environments, nodes may have taints that prevent the Fluent Bit agent from being scheduled. To ensure the agent can run on these nodes, you must define appropriate tolerations in your Helm chart configuration. Refer to the Kubernetes documentation about taints and tolerations for more information.

For example, if you have a taint on a node:

kubectl taint nodes 10.1.2.3 myNodeType=ingress:NoSchedule

You can allow the agent to be scheduled on that node by adding this configuration to your values.yaml file:

tolerations:
  - effect: NoSchedule
    key: myNodeType
    value: ingress
    operator: Equal

• These fields are directly added to the DaemonSet's tolerations section.
• This ensures the agent respects the taint rules and can be scheduled accordingly.

enableKubernetesFilter

By default the Helm chart enables the Kubernetes filter for processing the container logs. This provides capabilities such as:

• Enriching the log lines with metadata about the running container.
• Invoking another parser after the CRI parser to further process the log data.
• Using annotations to choose a secondary parser.

If you do not wish to use these features, then they can be disabled with this option.

enableKubernetesFilter: false

systemFiles

If you want to process files from a path other than /var/log/containers (for example, /var/log/syslog or /var/log/kubelet.log), then use the systemFiles setting to enable the processing for these files.

You can use the systemLogsParser option in order to consider a different parser - by default the kube_syslog parser is used which is appropriate for newer VPC-based systems. Some of the classic Kubernetes clusters may require the systemLogsParser to be set to kube_syslog_classic in order to match the timestamp format on those systems.

systemLogs:
  - /var/log/kube-proxy.log
  - /var/log/kubelet.log

additionalServiceOptions

This option can be used to add configuration options to the service section of the Fluent Bit configuration. This will add entries to the service section of the Fluent Bit yaml configuration.

additionalServiceOptions:
  storage.backlog.mem_limit: 5G

additionalKubeLogInputProcessors

This option allows the user to introduce additional processors for their container logs after the normal processing has completed. This may be useful for example, to introduce an additional parser within the processor pipeline. This will add entries to the pipeline.inputs.processors section of the Fluent Bit yaml configuration.

additionalKubeLogInputProcessors:
  - name: parser
    parser: json
    key_name: log
    reserve_data: true

additionalInputs

There might be some filters that should be applied to both containers and host logs - rather than including in the additionalKubeLogInputProcessors and additionalSystemLogsInputProcessors. This will add entries to the pipeline.filters section of the Fluent Bit yaml configuration.

additionalFilters:
  - name: modify
    match: '*'
    set:
      - deployment_env pre-production

additionalOutputs

In the event that the logs should be sent to another output location, the user can introduce additional output sections in order to send to a second IBM Cloud Logs instance. This will add entries to the pipeline.outputs section of the Fluent Bit yaml configuration.

additionalOutputs:
  - name: logger-icl-output-plugin
    id: logs-router-icl-output-plugin
    match: '*'
    # Connection
    target_host: "replace with host"
    target_port: 443
    target_path: /logs/v1/singles
    ...

additionalParsers

If you need to introduce a new parser, this option allows you to introduce your own parsers. If you need to introduce a new parser, this option allows you to introduce your own parsers. This will add entries to the [parsers](https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/yaml/parsers-section] section of the Fluent Bit yaml configuration.

additionalParsers:
  - name: json_raw
    format: json

additionalMultilineParsers

If you need to introduce a new multiline parser, this option allows you to introduce your own multiline parsers. This will add entries to the multiline_parsers section of the Fluent Bit yaml configuration.

additionalMultilineParsers:
  - name: multiline_sample
    type: regex
    flush_timeout: 1000
    rules:
      - state: start_state
        regex: '/^\[\d+\/\d+\/\d+,.*$/'
        next_state: cont
      - state: cont
        regex: '/^[^\[].*$/'
        next_state: cont
      - state: cont
        regex: '/^$/'
        next_state: cont