IBM Cloud Docs
VPC clusters: Why can't my app connect via load balancer?

VPC clusters: Why can't my app connect via load balancer?

Infrastructure provider: VPC

You exposed your app by creating a Kubernetes LoadBalancer service in your VPC cluster. When you try to connect to your app by using the hostname that is assigned to the Kubernetes LoadBalancer, the connection fails or times out.

When you run kubectl describe svc <kubernetes_lb_service_name>, you might see a warning message similar to one of the following in the Events section:

The VPC load balancer that routes requests to this Kubernetes `LoadBalancer` service is offline.

The VPC load balancer that routes requests to this Kubernetes `LoadBalancer` service was deleted from your VPC.

When you create a Kubernetes LoadBalancer service in your cluster, a VPC load balancer is automatically created in your VPC. The VPC load balancer routes requests only to the app that the Kubernetes LoadBalancer service exposes.

Requests can't be routed to your app in the following situations:

  • A VPC security group is blocking incoming traffic to your worker nodes, including incoming requests to your app.
  • The VPC load balancer is offline, such as due to load balancer provisioning errors or VSI connection errors.
  • The VPC load balancer is deleted through the VPC console or the CLI.
  • The VPC load balancer's DNS entry is still registering.
  • You reached the maximum number of VPC load balancers permitted per account. Check the VPC quotas documentation for VPC resource quotas across all your VPC clusters in your VPC.

Verify that no VPC security groups are blocking traffic to your cluster and that the VPC load balancer is available.

  1. Verify that the VPC load balancer for the Kubernetes LoadBalancer service exists. In the output, look for the VPC load balancer that is formatted kube-<cluster_ID>-<kubernetes_lb_service_UID>. You can get the Kubernetes LoadBalancer service UID by running kubectl get svc <service_name> -o yaml.

    ibmcloud is load-balancers
    • If the VPC load balancer is not listed, it does not exist for one of the following reasons:
    • You reached the maximum number of VPC load balancers permitted per account. Across all your VPC clusters in your VPC, a maximum of 20 VPC load balancers can be created. One VPC load balancer is created for each Kubernetes LoadBalancer service that you create, and it routes requests to that Kubernetes LoadBalancer service only.
    • The VPC load balancer was deleted through the VPC console or the CLI. To re-create the VPC load balancer for your Kubernetes LoadBalancer service, restart the Kubernetes master by running ibmcloud ks cluster master refresh --cluster <cluster_name_or_id>.

    If you want to remove the load balancing setup for an app in your VPC cluster, delete the Kubernetes LoadBalancer service by running kubectl delete svc <kubernetes_lb_service_name>. The VPC load balancer that is associated with the Kubernetes LoadBalancer service is automatically deleted from your VPC.

  2. If the load balancer exists, view the VPC security groups that are attached to it. If you have made any modifications to the kube-<vpc-id> security group, which is automatically attached to the load balancer, set the original rules back in the security group.

    • If the VPC load balancer is listed and you have not modified any attached security groups, it might not be responsive for the following reasons:
      • Its DNS entry might still be registering. When a VPC load balancer is created, the hostname is registered through a public DNS. Sometimes, it can take several minutes for this DNS entry to be replicated to the specific DNS that your client is using. You can either wait for the hostname to be registered in your DNS, or access the VPC load balancer directly by using one of its IP addresses. To find the VPC load balancer IP addresses, run ibmcloud is lb <LB_ID> and look for the Public IPs field.
      • If after several minutes you can't reach the load balancer, it might be offline due to provisioning or connection issues. Open an IBM Cloud support case. For the type, select Technical. For the category, select Network in the VPC section. In the description, include your cluster ID and the VPC load balancer ID.