Why is my app pod stuck in Container creating when trying to mount File Storage for VPC?

When you try to deploy an app that uses File Storage for VPC you see one or more of the following error messages.

Example command to get the ibm-vpc-file-csi driver logs.

kubectl logs ibm-vpc-file-csi-node-xxx -n kube-system -c iks-vpc-file-node-driver

Example output with error message.

ibmcsidriver/node.go:94","msg":"CSINodeServer-NodePublishVolume..."
ibmcsidriver/node.go:160","msg":"CSINodeServer-NodeUnpublishVolume..."

Example describe pod command.

kubectl describe pod ibm-vpc-file-csi-node-xxx -n kube-system -c iks-vpc-file-node-driver

Example output with error message.

Warning FailedMount 68s kubelet MountVolume.SetUp failed for volume "pvc-c37fe511-ec6d-44c1-8c55-1b5e2c21ec5b" : rpc error: code = DeadlineExceeded desc = context deadline exceeded
Warning FailedMount 65s kubelet Unable to attach or mount volumes: unmounted volumes=[test-persistent-storage], unattached volumes=[], failed to process volumes=[]: timed out waiting for the condition

Example command to get the pipelineruns logs.

kubectl logs cat -n pipelineruns

Example output with error message.

Error from server (BadRequest): container "cat" in pod "cat" is waiting to start: ContainerCreating

A temporary network outage might cause file shares to be unreachable and unmountable.

Complete the following steps to resolve the issue.

New security group rules were introduced in versions 1.25 and later. These rule changes mean you must sync your security groups before you can use File Storage for VPC. If your cluster was initially created at version 1.25 or earlier, run the following commands to sync your security group settings.

• If your cluster was initially created at version 1.25 or earlier:

  1. Get the ID of your cluster.

     ibmcloud ks cluster ls
     

  2. Get the ID of the kube-<clusterID> security group.

     ibmcloud is sg kube-<cluster-id>  | grep ID
     

  3. Sync the kube-<clusterID> security group by using the ID that you retrieved in the previous step.

     ibmcloud ks security-group sync -c <cluster ID> --security-group <ID>
     

• If your cluster was created at version 1.25 and later, verify that the worker node where pod is deployed is allowlisted in the VNI security group.

1. If the issue persists, contact support. Open a support case. In the case details, be sure to include any relevant log files, error messages, or command outputs.