I deleted a portable subnet and now my Classic cluster my Load Balancers are failing. How do I recover?

Classic

your cluster is experiencing network errors.

A portable subnet was deleted.

Complete the following steps.

1. For the cluster that is experiencing issues, run the following command and make a note of the cluster ID.

   ibmcloud ks cluster ls
   

2. List the subnets your cluster is using. In the output, make a note of the subnets that your cluster is using.

   ibmcloud ks subnets --provider classic | grep CLUSTER-ID
   

3. List the subnets in your account. Check whether any subnets that you found in step 1 are missing from the list of subnets in your account. This means your cluster is using a subnet that was deleted, which is causing your Load Balancer to fail.

   ibmcloud sl subnet list
   

   ibmcloud sl subnet detail SUBNET-ID
   

4. For each subnet that was deleted, re-create it. If you need public IP addresses for your ALBs or LoadBalancers, specify the public VLAN ID of your worker nodes; for private IPs specify the private VLAN ID of your worker nodes.

   ibmcloud ks cluster subnet create -c CLUSTER --size 8 --vlan VLAN-ID
   

5. Disable any ALBs that use the deleted portable subnets.

   1. List your ALBs and note the IDs of ALBS that you want to remove.

      ibmcloud ks ingress alb ls -c CLUSTER-ID
      

   2. Disable the ALBs to delete the Load Balancer for the ALB, which releases the portable IP address in the portable subnet you want to detach

      ibmcloud ks ingress alb disable -c CLUSTER-ID --alb ALB-ID
      

6. Complete the following steps to find any Load Balancers that use IP addresses from the deleted portable subnet(s), save their settings, and then delete them.

   1. Find the IP addresses your LoadBalancers use.

      kubectl get svc -A -o wide | grep LoadBalancer
      

   2. For each Load Balancer that you identified earlier, run the following command to export it a YAML file.

      kubectl get svc -o yaml -n LB-NAMESPACE LB-NAME > LB-NAMESPACE.LB-NAME.yaml
      

   3. After you saved the information to a YAML file, delete those LoadBalancers.

      kubectl delete svc -n LB-NAMESPACE LB-NAME
      

7. Detach the deleted portable subnets from your cluster with the following command.

   ibmcloud ks cluster subnet detach -c CLUSTER --subnet-id SUBNET-ID
   

8. Wait a few minutes for the detach command to finish, then check that they have been detached by reviewing the ibm-cloud-provider-vlan-ip-config ConfigMap that stores the subnets and IP addresses.

   kubectl get cm -n kube-system ibm-cloud-provider-vlan-ip-config -o yaml
   

   It might take 5-10 minutes for the detach to complete. While you wait, you can ensure that the new portable subnets you created for this cluster also appear in the ConfigMap.

9. Create ALBs by using the following command. Note that you can't re-enable the old ALBs because they still use the deleted portable subnet.

   ibmcloud ks ingress alb create classic --cluster CLUSTER-ID --type public|private --zone ZONE --vlan VLAN-ID
   

10. Re-create the LoadBalancers that you deleted by using the YAML file that you saved earlier.

    kubectl apply -f LB-NAMESPACE.LB-NAME.yaml