Installing the IBM Cloud Object Storage cluster add-on

Understanding bucket creation and removal

• You can use an existing bucket by specifying the bucket name in your PVC.
• If you provide a bucket name and that bucket doesn't exist, then a bucket with that name is created.
• If you don't provide a bucket name, then a bucket with the naming convention temp-xxx is created.
• Buckets are deleted based on reclaim policy defined in your storage class.
  • If reclaimPolicy: Delete is set, the bucket is deleted when the PVC is deleted.
  • If reclaimPolicy: Retain is set, the bucket is retained even after the PVC is deleted.

Enabling the IBM Cloud Object Storage add-on

Before you begin: Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.

1. List the add-ons and find the version you want to install.

   ibmcloud ks cluster addon versions
   

   Example output

   OK
   Name                        Version            Supported Kubernetes Range   Supported OpenShift Range   Kubernetes Default                   OpenShift Default
   ibm-object-csi-driver       0.1 (default)      >=1.30.0                     >=4.15.0                    -                                    -
   

2. Install the add-on.

   ibmcloud ks cluster addon enable ibm-object-csi-driver --cluster CLUSTER [--version VERSION]
   

3. Verify the installation.

   ibmcloud ks cluster addon ls --cluster CLUSTER
   

   OK
   Name                    Version   Health State   Health Status
   ibm-object-csi-driver   0.1       normal         Addon Ready. For more info: http://ibm.biz/addon-state (H1500)
   

4. List the available storage classes.

   kubectl get sc | grep object
   

   ibm-object-storage-smart-rclone             cos.s3.csi.ibm.io   Delete          Immediate           false                  17h
   ibm-object-storage-smart-rclone-retain      cos.s3.csi.ibm.io   Retain          Immediate           false                  17h
   ibm-object-storage-smart-s3fs               cos.s3.csi.ibm.io   Delete          Immediate           false                  17h
   ibm-object-storage-smart-s3fs-retain        cos.s3.csi.ibm.io   Retain          Immediate           false                  17h
   ibm-object-storage-standard-rclone          cos.s3.csi.ibm.io   Delete          Immediate           false                  17h
   ibm-object-storage-standard-rclone-retain   cos.s3.csi.ibm.io   Retain          Immediate           false                  17h
   ibm-object-storage-standard-s3fs            cos.s3.csi.ibm.io   Delete          Immediate           false                  17h
   ibm-object-storage-standard-s3fs-retain     cos.s3.csi.ibm.io   Retain          Immediate           false                  17h
   

Deploying an app that uses IBM Cloud Object Storage

Setting up autorecovery for stale volumes

When the connection is lost between the ibm-object-csi-driver node server pods and application pods, you might see TransportEndpoint connection errors. One possible case for this error is when patch updates are applied. To help avoid the connection errors, set up autorecovery of stale volumes by completing the following steps.

1. Copy the following yaml and save it as a file called stale.yaml

   apiVersion: objectdriver.csi.ibm.com/v1alpha1
   kind: RecoverStaleVolume
   metadata:
     labels:
       app.kubernetes.io/name: recoverstalevolume
       app.kubernetes.io/instance: recoverstalevolume-sample
     name: recoverstalevolume-sample
     namespace: default
   spec:
     logHistory: 200
     data:
       - namespace: default # The namesapce where your app is deployed
         deployments: [<A comma separated list of all the apps you want to recover>]
   

2. Create the RecoverStaleVolume resource in your cluster.

   kubectl create -f stale.yaml
   

   Example output

   recoverstalevolume.objectdriver.csi.ibm.com/recoverstalevolume-sample created
   

3. Verify that the resource was created.

   kubectl get recoverstalevolume
   

   Example output

   NAME  AGE  recoverstalevolume-sample   41s
   

4. If the issue persists, contact support. Open a support case. In the case details, be sure to include any relevant log files, error messages, or command outputs.

Disabling the IBM Cloud Object Storage add-on

1. Run the following command to disable the add-on.

    ibmcloud ks cluster addon disable ibm-object-csi-driver --cluster CLUSTER
   

   Example output

   Data and resources that you created for the add-on might be deleted when the add-on is disabled. Continue? [y/N]> y
   Disabling add-on ibm-object-csi-driver for cluster XXX...
   OK
   

2. Verify the add-on was removed.

   ibmcloud ks cluster addon ls --cluster CLUSTER
   

Migrating from the Helm plug-in to the cluster add-on

1. Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.

2. Get the details of your PVCs and select one to migrate.

   kubectl get pvc --all-namespaces -o custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name' | tail -n +2 | while read namespace pvc; do kubectl describe pvc "$pvc" -n "$namespace" | grep 'volume.kubernetes.io/storage-provisioner: ibm.io/ibmc-s3fs' > /dev/null ; if [ $? -eq 0 ]; then echo "PVC: $pvc in Namespace: $namespace uses ibm.io/ibmc-s3fs storage provisioner"; fi; done
   

   Example output

   PVC: pvc-test in Namespace: default uses ibm.io/ibmc-s3fs storage provisioner
   

3. Describe the PVC and get the bucket name.

   kubectl describe pvc <pvc_name> | grep ibm.io/bucket:
   

   Example output

   ibm.io/bucket: test-s3
   

4. Create a secret that has the same name as your PVC.

   apiVersion: v1
   kind: Secret
   type: cos-s3-csi-driver
   metadata:
       name: test-s3 # Name your secret the same name your PVC
       namespace: default # Specify the namespace where you want to create the secret. In this example, the previous PVC and secret were in the default namespace.
   data:
       bucketName: <base64-encoded-bucket-name>
       apiKey: <base64-encoded-COS-Service-Instance-API-key>
       accessKey: <base64-encoded-HMAC-access-key>
       secretKey: <base64-encoded-HMAC-secret-key>
   stringData:
   # uid: "3000" # Optional: Provide a uid to run as non root user. This must match runAsUser in SecurityContext of pod spec.
   mountOptions: |
   

5. Find the storage class that was used in your PVC.

   kubectl describe pvc <pvc_name> | grep StorageClass:
   

   Example command for a PVC called test-s3.

   kubectl describe pvc test-s3 | grep StorageClass:
   

   Example output

   StorageClass:  ibmc-s3fs-smart-perf-regional
   

6. Review the new storage classes that are available with the add-on and select a replacement class.

   • If you used a flex class, choose one of the new smart classes.
   • If you used a standard classes, choose one of the new standard classes.
   • The cold and vault classes are no longer available with the add-on; choose a smart or standard class instead.

7. Review the details of your PVC.

   kubectl describe pvc test-s3
   

   Example output

   Name:          pvc-test
   Namespace:     default
   StorageClass:  ibmc-s3fs-smart-perf-regional
   Status:        Bound
   Volume:        pvc-c625474d-31f0-4929-bc3e-feace1fb42fb
   Labels:        <none>
   Annotations:   ibm.io/auto-create-bucket: true
               ibm.io/auto-delete-bucket: true
               ibm.io/bucket: bha-test-s23
               ibm.io/secret-name: satstoragesecret
               pv.kubernetes.io/bind-completed: yes
               pv.kubernetes.io/bound-by-controller: yes
               volume.beta.kubernetes.io/storage-provisioner: ibm.io/ibmc-s3fs
               volume.kubernetes.io/storage-provisioner: ibm.io/ibmc-s3fs
   Finalizers:    [kubernetes.io/pvc-protection]
   Capacity:      3Gi
   Access Modes:  RWO
   VolumeMode:    Filesystem
   Used By:       test-pod
   Events:        <none>
   

8. Create a replacement PVC that uses a new storage class and references the secret you created earlier.

   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
   name: test-s3 # Enter the same name as the secret you created earlier.
   spec:
   accessModes:
   - ReadWriteOnce
   resources:
       requests:
       storage: 3Gi
   storageClassName: ibm-object-storage-smart-s3fs
   

9. Verify the PVC is Bound.

   kubectl get pvc
   

10. Get the details of your app.

    kubectl get pods
    

11. Scale down your app to zero.

    kubectl scale deployment --replicas=0 my-app
    

12. Create a replacement deployment that references the PVC you created in the previous step.

13. After the new deployment is running, you can delete the old deployment.

14. Repeat these steps for each PVC that you want to migrate.

IBM Cloud Object Storage cluster add-on storage classes