Deploying OpenShift Data Foundation on VPC clusters

Installing the OpenShift Data Foundation add-on from the console

To install ODF in your cluster, complete the following steps.

1. Before you enable the add-on, review the change log for the latest version information.

2. Review the parameter reference.

3. From the Red Hat OpenShift clusters console, select the cluster where you want to install the add-on.

4. On the cluster Overview page, on the OpenShift Data Foundation card, click Install. The Install ODF panel opens.

5. In the Install ODF panel, enter the configuration parameters that you want to use for your ODF deployment.

6. Select either Essentials or Advanced as your billing plan. For more information about billing type, see Feature support by billing type.

7. For VPC clusters, select Remote provisioning to dynamically provision volumes for ODF by using the Block Storage for VPC.

8. In the OSD storage class name field, enter the name of the Block Storage for VPC ODF storage class that you want to use to provision storage volumes. For multizone clusters, use a storage class with the VolumeBindingMode of WaitForFirstConsumer. See the Storage Class Reference for more information.

9. In the OSD pod size field, enter the size of the volume that you want to provision. Enter at least 512Gi.

10. In the Worker nodes field, enter the node names of the worker nodes where you want to deploy ODF. You must enter at least 3 worker node names. To find your node names, run the oc get nodes command in your cluster. Node names must be comma-separated with no spaces between names. For example: 10.240.0.24,10.240.0.26,10.240.0.25.Leave this field blank to deploy ODF on all worker nodes.

11. In the Number of OSD disks required field, enter the number of OSD disks (app storage) to provision on each worker node.

12. If you want to encrypt the OSD volumes (cluster wide encryption) used by the ODF system pods, select Enable cluster encryption.

13. If you want to enable encryption for the application volumes (app storage), select Enable volume encryption.

    1. In the Instance name field, enter a unique name for your Hyper Protect Crypto Services or Key Protect instance.
    2. In the Instance type field, enter the type of encryption instance.
    3. In the Instance ID field, enter your Hyper Protect Crypto Services or Key Protect instance ID. For example: d11a1a43-aa0a-40a3-aaa9-5aaa63147aaa.
    4. In the Secret name field, enter the name of the secret that you created using your Hyper Protect Crypto Services or Key Protect credentials. For example: ibm-hpcs-secret.
    5. In the Base URL field, enter the public endpoint of your Hyper Protect Crypto Services or Key Protect instance. For example: https://api.eu-gb.hs-crypto.cloud.ibm.com:8389.
    6. In the Token URL field, enter https://iam.cloud.ibm.com/identity/token.

14. After you enter the parameters that you want to use, click Install

15. Wait a few minutes for the add-on deployment to complete. When the deployment is complete, the add-on status is Normal - Addon Ready.

16. Verify your installation. Access your Red Hat OpenShift cluster.

17. Run the following command to verify the ODF pods are running.

    oc get pods -n openshift-storage
    

Next steps

Deploy an app that uses ODF.

Installing the add-on from the CLI

You can install the add-on by using the ibmcloud oc cluster addon enable command.

1. Review the VPC parameter reference. When you enable the add-on, you can override the default values by specifying the --param "key=value" option for each parameter that you want to override.

2. Access your Red Hat OpenShift cluster.

3. List the openshift-data-foundation add-on versions. Make a note of the default version and determine the version that you want to install.

   ibmcloud ks cluster addon versions
   

4. Before you enable the add-on, review the change log for the latest version information. Note that the add-on supports n+1 cluster versions. For example, you can deploy version 4.10.0 of the add-on to an OCP 4.9 or 4.11 cluster. If you have a cluster version other than the default, you must specify the --version option when you enable the add-on.

5. Review the add-on options.

   ibmcloud oc cluster addon options --addon openshift-data-foundation --version 4.12.0
   

   Example add-on options for version 4.12.0

   Add-on Options
   Option                Default Value
   clusterEncryption     false
   hpcsTokenUrl          <Please provide the KMS token URL>
   osdDevicePaths        <Please provide IDs of the disks to be used for OSD pods if using local disks or standard classic cluster>
   ocsUpgrade            false
   autoDiscoverDevices   false
   hpcsServiceName       <Please provide the KMS Service instance name>
   hpcsSecretName        <Please provide the KMS secret name>
   osdSize               250Gi
   osdStorageClassName   ibmc-vpc-block-metro-10iops-tier
   billingType           advanced
   hpcsInstanceId        <Please provide the KMS Service instance ID>
   hpcsBaseUrl           <Please provide the KMS Base (public) URL>
   odfDeploy             true
   numOfOsd              1
   workerNodes           all
   hpcsEncryption        false
   ignoreNoobaa          false
   

6. Enable the openshift-data-foundation add-on. If you want to override any of the default parameters, specify the --param "key=value" option for each parameter you want to override. If you don't want to create your storage cluster when you enable the add-on, you can enable the add-on first, then create your storage cluster later by creating a CRD.

   Example command to deploy add-on version 4.10 with the default storage cluster settings and encryption with Hyper Protect Crypto Services enabled.

   ibmcloud oc cluster addon enable openshift-data-foundation -c <cluster-name> --version 4.12.0 --param "odfDeploy=true" --param "hpcsTokenUrl=https://iam.cloud.ibm.com/identity/token" --param "hpcsEncryption=true" --param "hpcsBaseUrl=<hpcs-instance-public-endpoint>" --param "hpcsInstanceId=<hpcs-instance-id>" --param "hpcsServiceName=<hpcs-instance-name>" --param "hpcsSecretName=<hpcs-secret-name>"
   

   Example command for deploying the ODF add-on only.

   ibmcloud oc cluster addon enable openshift-data-foundation -c <cluster_name> --version <version> --param "odfDeploy=false"
   

   Example command for deploying the ODF and creating a storage cluster with the default configuration parameters.

   ibmcloud oc cluster addon enable openshift-data-foundation -c <cluster_name> --version <version> 
   

   Example command for deploying the ODF and creating a storage cluster while overriding the osdSize parameter.

   ibmcloud oc cluster addon enable openshift-data-foundation -c <cluster_name> --version <version> --param "osdSize=500Gi"
   

7. Verify the add-on is in a Ready state.

   oc get storagecluster -n openshift-storage
   

   Example output:

   NAME                 AGE   PHASE   EXTERNAL   CREATED AT             VERSION
   ocs-storagecluster   53m   Ready              2023-03-10T12:20:52Z   4.11.0
   

8. Verify that the ibm-ocs-operator-controller-manager-***** pod is running in the kube-system namespace.

   oc get pods -A | grep ibm-ocs-operator-controller-manager
   

9. If you enabled the add-on with odfDeploy set to false, follow the steps to create an ODF custom resource.

Installing the add-on from Terraform

• Install the Terraform CLI and the IBM Cloud Provider plug-in.
• Make sure you have an IBM Cloud API key.

1. Create a Terraform provider file. Save the file in your Terraform directory. For more information, see the Terraform IBM Cloud Provider documentation.

   Example Terraform provider file.

   terraform {
   required_providers {
       ibm = {
       source = "IBM-Cloud/ibm"
       version = "1.53.0"
       }
   }
   }
   
   provider "ibm" {
   region = "us-south"
   ibmcloud_api_key = "<api-key>"
   }
   

2. Create a Terraform configuration file for the ODF add-on. Save the file in your Terraform directory.

   Example configuration file.

   ibmcloud_api_key = "" # Enter your API Key
   cluster = "" # Enter the Cluster ID
   region = "us-south" # Enter the region
   
   # For add-on deployment
   odfVersion = "4.12.0"
   
   # For CRD Creation and Management
   autoDiscoverDevices = "false"
   billingType = "advanced"
   clusterEncryption = "false"
   hpcsBaseUrl = null
   hpcsEncryption = "false"
   hpcsInstanceId = null
   hpcsSecretName = null
   hpcsServiceName = null
   hpcsTokenUrl = null
   ignoreNoobaa = "false"
   numOfOsd = "1"
   ocsUpgrade = "false"
   osdDevicePaths = null
   osdSize = "250Gi"
   osdStorageClassName = "ibmc-vpc-block-metro-10iops-tier"
   workerNodes = null
   

3. In the CLI, navigate to your Terraform directory.

   cd <terraform_directory>
   

4. Run the commands to initialize and plan your Terraform actions. Review the plan output to make sure the correct actions are performed.

   terraform init
   

   terraform plan
   

5. Apply the Terraform files to create the cluster. Then, navigate to the IBM Cloud console to check that the cluster is provisioning.

   terraform apply
   

Creating your ODF custom resource

To create an ODF storage cluster in your VPC cluster by using dynamic provisioning for your storage volumes, you can create a custom resource to specify storage device details.

If you want to use an IBM Cloud Object Storage service instance as your default backing store, make sure that you created the service instance, and created the Kubernetes secret in your cluster. When you create the ODF CRD in your cluster, ODF looks for a secret named ibm-cloud-cos-creds to set up the default backing store that uses your Object Storage HMAC credentials.

1. Create a custom resource definition called OcsCluster. Save one of the following custom resource definition files on your local machine and edit it to include the name of the your storage class that you created earlier as the monStorageClassName and osdStorageClassName parameters. For more information about the OcsCluster parameters, see the parameter reference.

   Example custom resource definition for installing ODF on all worker nodes on a 4.8 cluster.

   apiVersion: ocs.ibm.io/v1
   kind: OcsCluster
   metadata:
     name: ocscluster-vpc # Kubernetes resource names can't contain capital letters or special characters. Enter a name for your resource that uses only lowercase letters, numbers, `-` or `.`
   spec:
     osdStorageClassName: <osdStorageClassName> # Specify an ODF storage class with a waitForFirstConsumer volume binding mode
     osdSize: <osdSize> # The OSD size is the total storage capacity of your OCS storage cluster. Use at least 250Gi OSDs for production workloads.
     numOfOsd: 1
     billingType: advanced
     ocsUpgrade: false
   

   Example custom resource definition for installing ODF only on specified worker nodes on a 4.8 cluster.

   apiVersion: ocs.ibm.io/v1
   kind: OcsCluster
   metadata:
     name: ocscluster-vpc # Kubernetes resource names can't contain capital letters or special characters. Enter a name for your resource that uses only lowercase letters, numbers, `-` or `.`
   spec:
     osdStorageClassName: <osdStorageClassName> # Specify an ODF storage class with a waitForFirstConsumer volume binding mode
     osdSize: <osdSize> # The OSD size is the total storage capacity of your OCS storage cluster. Use at least 250Gi OSDs for production workloads.
     numOfOsd: 1
     billingType: advanced
     ocsUpgrade: false
     workerNodes: # Specify the private IP addresses of the worker nodes where you want to install OCS.
       - <workerNodes> # To get a list worker nodes, run `oc get nodes`.
       - <workerNodes>
       - <workerNodes>
   

2. Save the file and create the OcsCluster custom resource to your cluster.

   oc create -f <ocs-cluster-filename>.yaml
   

3. Verify that your OcsCluster is running.

   oc describe ocscluster ocscluster-vpc
   

   Example output

   Name:         ocscluster-vpc
   Namespace:    
   Labels:       <none>
   Annotations:  <none>
   API Version:  ocs.ibm.io/v1
   Kind:         OcsCluster
   Metadata:
       Creation Timestamp:  2021-03-23T20:56:51Z
   Finalizers:
     finalizer.ocs.ibm.io
   Generation:  1
   Managed Fields:
     API Version:  ocs.ibm.io/v1
     Fields Type:  FieldsV1
     fieldsV1:
       f:spec:
         .:
         f:billingType:
         f:monSize:
         f:monStorageClassName:
         f:numOfOsd:
         f:ocsUpgrade:
         f:osdSize:
         f:osdStorageClassName:
     Manager:      oc
     Operation:    Update
     Time:         2021-03-23T20:56:51Z
     API Version:  ocs.ibm.io/v1
     Fields Type:  FieldsV1
     fieldsV1:
       f:metadata:
         f:finalizers:
           .:
           v:"finalizer.ocs.ibm.io":
       f:status:
         .:
         f:storageClusterStatus:
     Manager:         manager
     Operation:       Update
     Time:            2021-04-09T23:12:02Z
   Resource Version:  11372332
   Self Link:         /apis/ocs.ibm.io/v1/ocsclusters/ocscluster-vpc
   UID:               aa11a1a1-111f-aace-afac-1fa1afe1111a
   Spec:
     Billing Type:            hourly
     Mon Size:                20Gi
     Mon Storage Class Name:  ibmc-vpc-block-10iops-tier
     Num Of Osd:              1
     Ocs Upgrade:             false
     Osd Size:                250Gi
     Osd Storage Class Name:  ibmc-vpc-block-10iops-tier
   Status:
     Storage Cluster Status:  
   Events:                    <none>
   

4. Deploy an app that uses ODF.