Setting up a Private Path Network Load Balancer for VPC
Virtual Private Cloud 1.29 and later
In fully private VPC environments with no public Internet access, you can use a Private Path Network Load Balancer to balance the network traffic flowing to the applications running in your VPC clusters. For more information, see the Private Path service use cases.
Private Path Network Load Balancers for VPC are available in Beta and their functionality is subject to change.
Prerequisites
-
If you don't already have a running app, deploy an app to your cluster. Ensure that you add a label in the metadata section of your deployment configuration file. This custom label identifies all pods where your app runs to include them in the load balancing.
Configuring the LoadBalancer
service
-
Copy the following
LoadBalancer
configuration and save it to a file calledlb.yaml
.apiVersion: v1 kind: Service metadata: name: <app_name>-vpc-nlb-<VPC_zone> annotations: service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-lb-name: "my-load-balancer" service.kubernetes.io/ibm-load-balancer-cloud-provider-enable-features: "private-path" # Required service.kubernetes.io/ibm-load-balancer-cloud-provider-ip-type: "private" # Required service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-subnets: "<subnet_ID>" spec: type: LoadBalancer selector: <selector_key>: <selector_value> ports: - name: http protocol: TCP port: 8080 targetPort: 8080 # Optional. By default, the `targetPort` is set to match the `port` value unless specified otherwise. - name: https protocol: TCP port: 443 targetPort: 443 # Optional. By default, the `targetPort` is set to match the `port` value unless specified otherwise. externalTrafficPolicy: Local # Specify Local or Cluster.
-
Customize the fields for your use case. For a complete list of annotations, see Annotations and specifications.
-
Save your changes.
-
Deploy the Load Balancer service to your cluster.
kubectl apply -f lb.yaml
Creating a Private Path service
Follow the instructions to Create a Private Path service.
Setting up a Virtual Private Endpoint Gateway
Now that you've configured a Load Balancer service, you must set a Virtual Private Endpoint (VPE) Gateway to access the applications in your cluster.
For more information, see Creating an endpoint gateway in the UI.
Connecting to your apps through your VPE
For information on connecting to your apps through your VPE, see Accessing your virtual private endpoint after setting up your endpoint gateway.
Annotations and specifications
Review the required and optional VPC NLB annotations and specifications.
Required annotations and specifications
externalTrafficPolicy
- Specify
Local
orCluster
. - Set to
Local
to preserve the source IP address of client requests to your apps. This setting prevents the incoming traffic from being forwarded to a different node. This option also configures HTTP health checks. - If
Cluster
is set, DSR is implemented only from the worker node that the VPC NLB initially forwards the incoming request to. After the incoming request arrives, the request is forwarded to a worker node that contains the app pod, which might be in a different zone. The response from the app pod is sent to the original worker node, and that worker node uses DSR to send the response directly back to the client, bypassing the VPC NLB. This option also configures TCP health checks.
Optional annotations and specifications
service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-lb-name
- Include a unique name to make your VPC load balancer persistent. Persistent VPC load balancers are not deleted when the cluster they belong to is deleted. For more information, see Persistent VPC load balancers. This annotation can be set only on load balancer creation. It cannot be used in an update operation.
service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-health-check-protocol
- This annotation sets the health check protocol on the VPC load balancer resource associated with the Kubernetes load balancer service. Available options are
http
,https
, ortcp
. Usually, the VPC LB health check protocol is determined by the value of theexternalTrafficPolicy
setting in the Kubernetes load balancer service specification. However, this annotation overrides that logic. This annotation does not alter how Kubernetes, and kube-proxy in particular, behaves in regards to the various settings ofexternalTrafficPolicy
. service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-health-check-port
- The TCP port that is used for the health checks. This annotation applies only if
ibm-load-balancer-cloud-provider-vpc-health-check-protocol
is also specified. If the specified TCP port is outside of the Kubernetes node port range (30,000-32,767), the VPC security group applied to the cluster worker nodes must be modified to allow inbound traffic on the port. If this annotation is applied to a Kubernetes load balancer service associated with a VPC ALB, the outbound rules of the security group assigned to the VPC ALB must be modified to allow outbound traffic to the specified TCP port. For more information, see Managing security groups. service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-subnets
- Annotation to specify which subnet to use to assigned the IP addresses for the ppNLB. These IP addresses are only used internally. The value can be a VPC subnet ID, VPC subnet name, or VPC subnet CIDR. You must specify only one subnet. All
incoming traffic appears to be coming from these IP addresses. While all the addresses are in a single zone, the ppNLB still handles incoming traffic from all zones. If this specific zone goes done, incoming traffic from the other zones
still works. If you don not specify this annotation, the subnet is automatically selected and the cluster worker node subnet that has the most free IP addresses available is used. To see subnets in all resource groups, run
ibmcloud ks subnets --provider vpc-gen2 --vpc-id <vpc> --zone <zone>
. service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-node-selector
- Annotation to specify a worker node label selector. You can configure specific worker nodes in your cluster to receive traffic by specifying label selector keys. You can include only one label selector in the annotation, and that the selector
must be specified in the
"key=value"
format. If this annotation is not specified, all worker nodes in your cluster are configured to receive traffic from the VPC NLB. This annotation takes precedence over theservice.kubernetes.io/ibm-load-balancer-cloud-provider-zone
annotation, and anydedicated: edge
labels on worker nodes are ignored. To limit traffic to a specific zone, you can use this annotation to specify worker nodes in that zone. Note that setting a new label on a cluster worker node does not automatically configure the worker node to receive traffic; you must recreate or update the VPC NLB for the newly labeled worker node to receive traffic. service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-health-check-path
- The health check URL path for HTTP and HTTPs health checks. This annotation applies only if
ibm-load-balancer-cloud-provider-vpc-health-check-protocol
is set tohttp
orhttps
. The URL path must be in the format of an origin-form request target. If this annotation is not specified and theibm-load-balancer-cloud-provider-vpc-health-check-protocol
annotation is set tohttp
orhttps
, the default value/
is applied. service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-health-check-delay
- Optional. The number of seconds to wait between health check attempts. By default, this value is set to
5
, and has a minimum of2
and a maximum of60
. This value must be greater than theibm-load-balancer-cloud-provider-vpc-health-check-timeout
value, which is set to2
by default. service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-health-check-timeout
- Optional. The number of seconds to wait for a response to a health check. By default, this value is set to
2
, and has a minimum of1
and a maximum of59
. This value must be less than theibm-load-balancer-cloud-provider-vpc-health-check-delay
, which is set to5
by default. service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-health-check-retries
- The maximum number of health check retries for the VPC load balancer. By default, this value is set to
2
, and has a minimum of1
and a maximum of10
. service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-member-quota
- Optional. The number of worker nodes per zone that the load balancer routes to. The default value is 8. For a cluster with worker nodes in three zones, this results in the the load balancer routing to 24 total worker nodes. The total number of worker nodes across all zones that the load balancer routes to cannot exceed 50. If the cluster has fewer than 50 worker nodes across all zones, specify 0 to route to all worker nodes in a zone.
selector
- The label key (
<selector_key>
) and value (<selector_value>
) that you used in thespec.template.metadata.labels
section of your app deployment YAML. This custom label identifies all pods where your app runs to include them in the load balancing. port
- The port that the service listens on.
targetPort
- Optional: The port to which the service directs traffic. The application running in the pod must be listening for incoming TCP traffic on this target port. The target port is often statically defined in the image that is running in the application pod. The target port configured in the pod is different than the node port for the service and might also be different than the external port that is configured on the VPC LB.