Resilient and secure multi-region Kubernetes clusters with IBM Cloud Internet Services

Create a Kubernetes cluster

A minimal cluster with one (1) zone, one (1) worker node is sufficient for this tutorial.

When creating the following Kubernetes cluster:

1. Set Cluster name to my-us-cluster.

2. Locate in North America and Dallas

3. Open the Kubernetes clusters and click Create cluster.

4. Create a cluster on your choice of Infrastructure.

   • The following steps are if you select VPC for Kubernetes on VPC infrastructure. You are required to create a VPC and subnet(s) before creating the Kubernetes cluster. Reference the Creating VPC clusters documentation for more details.

     1. Click Create VPC.
     2. Under the Location section, select a Geography and Region, for example North America and Dallas.
     3. Enter a Name of your VPC, select a Resource group and optionally, add Tags to organize your resources.
     4. Uncheck Allow SSH and Allow ping from the Default security group.
     5. Uncheck Create subnet in every zone.
     6. Click on Create.
     7. Under Worker zones and subnets, uncheck the two zones for which the subnet wasn't created.
     8. Set the Worker nodes per zone to 1 and click on Change flavor to explore and change to the worker node flavor of your choice.
     9. Under Ingress, enable Ingress secrets management and select your existing Secrets Manager instance.
     10. Enter a Cluster name and select the same Resource group that you used for the VPC.
     11. Logging or Monitoring aren't required in this tutorial, disable those options and click on Create.
     12. While you waiting for the cluster to become active, attach a public gateway to the VPC. Navigate to the Virtual private clouds.
     13. Click on the name for the VPC used by the cluster and scroll down to subnets section.
     14. Click on the name of the subnet created earlier and in the Public Gateway section, click on Detached to change the state to Attached.

   • The following steps are if you select Classic for Kubernetes on Classic infrastructure. Reference the Creating a standard classic cluster documentation for more details.

     1. Under the Location section, select a Geography, multizone Availability, and Metro for example North America and Dallas.
     2. Under Worker zones and VLANs, uncheck all zones except for one.
     3. Set the Worker nodes per zone to 1 and click on Change flavor to explore and change to the worker node flavor of your choice.
     4. Under Master service endpoint, select Both private & public endpoints.
     5. Under Ingress, enable Ingress secrets management and select your existing Secrets Manager instance.
     6. Enter a Cluster name and select the Resource group to create these resources under.
     7. Logging or Monitoring aren't required in this tutorial, disable those options and click on Create.

While the cluster is getting ready, you are going to prepare the application.

Deploy the application to the Kubernetes cluster

The cluster should be ready. You can check its status in the Kubernetes Service console.

1. Gain access to your cluster as described on the Access tab of your cluster. Something like:

   MYCLUSTER=my-us-cluster
   ibmcloud ks cluster config --cluster $MYCLUSTER
   

2. Create the deployment using a pre-built image of the application. The application source code can be found in this GitHub repository.

   kubectl create deploy hello-world-deployment --image=icr.io/solution-tutorials/tutorial-scalable-webapp-kubernetes
   

   Example output: deployment "hello-world-deployment" created.

3. Make the application accessible within the cluster by creating a service:

   kubectl expose deployment/hello-world-deployment --type=ClusterIP --port=80 --name=hello-world-service --target-port=3000
   

   It returns message like service "hello-world-service" exposed. To see the services:

   kubectl get services
   

4. Run the application in the cluster with two replicas:

   kubectl scale deployment hello-world-deployment --replicas=2
   

5. You can check the status of the deployment with the following command:

   kubectl get pods
   

Get the Ingress Subdomain assigned to the cluster

When a Kubernetes cluster is created, it gets assigned an Ingress subdomain (for example, my-us-cluster.us-south.containers.appdomain.cloud) and a public Application Load Balancer IP address.

1. Retrieve the Ingress subdomain of the cluster:

   ibmcloud ks cluster get --cluster $MYCLUSTER
   

   Look for the Ingress Subdomain value.
2. Make note of this information for a later step.

This tutorial uses the Ingress Subdomain to configure the Global Load Balancer. You could also replace the Ingress Subdomain with the public Application Load Balancer, ALB of the cluster. An <IngressSubdomain> looks something like my-us-cluster-e7f2ca73139645ddf61a8702003a483a-0000.us-south.containers.appdomain.cloud

Configure the Ingress for your DNS subdomain

It will be required to have your own DNS domain name and a global load balancer subdomain will be created in the following task: <glb_name>.<your_domain_name>. Something like hello-world-service.example.com <glb_name> = hello-world-service and <your_domain_name> = example.com

1. Create the file glb-ingress.yaml and replace the placeholders with their respective values:

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
    name: <glb-name>
    annotations:
      spec.ingressClassName: "public-iks-k8s-nginx"
   spec:
    rules:
    - host: <glb-name>.<your_domain_name>
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: hello-world-service
              port:
                number: 80
   

2. Add the ingress instance:

   kubectl apply -f glb-ingress.yaml
   

   It can take a few minutes before ingress becomes available as indicated by a value in the ADDRESS column in the command:

   kubectl get ingress
   

3. Now test by configuring the curl Host http header with your DNS subdomain name to override the default of <IngressSubdomain>:

   curl --header 'Host: <glb_name>.<your_domain_name>' <IngressSubdomain>/hostname
   

   The curl command would look something like this: curl --header 'Host: hello-world-service.ibmom.com' my-us-cluster-e7f2ca73139645ddf61a8702003a483a-0000.us-south.containers.appdomain.cloud/hostname

Register a custom domain with IBM Cloud Internet Services

The first step is to create an instance of CIS and to point your custom domain to CIS name servers.

1. If you don't own a domain, you can buy one from a registrar.

2. Navigate to IBM Cloud Internet Services in the IBM Cloud catalog.

3. Pick a plan, set the service name and resource group, and click Create to create an instance of the service.

4. When the service instance is provisioned, click on Add domain.

5. Enter your domain name and click Next.

6. Setup your DNS records is an optional step and can be skipped for this tutorial. click on Next.

7. When the name servers are assigned, configure your registrar or domain name provider to use the name servers listed.

8. At this point you can click on Cancel to get back to the main page, after you've configured your registrar or the DNS provider, it may require up to 24 hours for the changes to take effect.

   When the domain's status on the Overview page changes from Pending to Active, you can use the dig <your_domain_name> ns command to verify that the new name servers have taken effect.

Configure Health Check for the Global Load Balancer

Health Checks monitor responses to HTTP/HTTPS requests from origin pools on a set interval. They are used with origin pools to determine if the pools are still running properly.

1. In the IBM Cloud Internet Services dashboard, use the navigation menu to select Reliability > Global Load Balancers.

2. Select the Health checks tab and click Create.

   1. Set Name to hello-world-service
   2. Set Monitor Type to HTTP.
   3. Set Port to 80.
   4. Set Path to /.
   5. In the Configure request headers (optional) add Header name: Host and Value: <glb_name>.<your_domain_name>
   6. Click Create.

   When building your own applications, you could define a dedicated health endpoint such as /heathz where you would report the application state.

Define Origin Pools

A pool is a group of origin servers that traffic is intelligently routed to when attached to a GLB. With clusters in the United Kingdom and United States, you can define location-based pools and configure CIS to redirect users to the closest clusters based on the geographical location of the user requests.

Create the Global Load Balancer

With the origin pools defined, you can complete the configuration of the load balancer.

1. Select the Load balancers tab and click Create.

2. Enter a name, <glb_name>, under Name for the Global Load Balancer. This name will also be part of your universal application URL (http://<glb_name>.<your_domain_name>), regardless of the location.

3. Under Geo routes, click Add route.

   1. Select Default from the Region drop down.
   2. Select the pool US.
   3. Click Add.

   Repeat the process to create the following:

   List of Geo routes to create

   Region	Origin Pool
   Default	US
   Western Europe	UK
   Eastern Europe	UK
   Northeast Asia	UK
   Southeast Asia	UK
   Western North America	US
   Eastern North America	US

   With this configuration, users in Europe and in Asia will be redirected to the cluster in London, users in US to the Dallas cluster. When a request does not match any of the defined route, it will be redirected to the pool(s) in the Default region.

4. Click Create

At this stage, you have successfully configured a Global Load Balancer with Kubernetes clusters across many locations. You can access the GLB URL http://<glb_name>.<your_domain_name>/hostname to view your application. Based on your location, you are redirected to the closest cluster or a cluster from the default pool if CIS was not able to map your IP address to a specific location.

Turn the Web Application Firewall On

The Web Application Firewall(WAF) protects your web application against ISO Layer 7 attacks. Usually, it is combined with grouped rule-sets, these rule-sets aim to protect against vulnerabilities in the application by filtering out malicious traffic.

1. In the IBM Cloud Internet Services dashboard, navigate to Security, then on the WAF.
2. Ensure the WAF is On.
3. Click OWASP Rule Set. From this page, you can review the OWASP Core Rule Set and individually enable or disable rules. When a rule is enabled, if an incoming request triggers the rule, the global threat score will be increased. The Sensitivity setting will decide whether an Action is triggered for the request.
   1. Leave default OWASP rule sets as it is.
   2. Set Sensitivity to Low.
   3. Set Action to Simulate to log all the events.
4. Click CIS Rule Set. This page shows additional rules based on common technology stacks for hosting websites.

For a secured connection with HTTPS, you can either obtain a certificate from Let's Encrypt as described in the following IBM Cloud® blog or through IBM Cloud Secrets Manager.

Increase performance and protect from Denial of Service attacks

A distributed denial of service (DDoS) attack is a malicious attempt to disrupt normal traffic of a server, service, or network by overwhelming the target or its surrounding infrastructure with a flood of internet traffic. CIS is equipped to protect your domain from DDoS.

1. In the CIS dashboard, select Reliability > Global Load Balancer.

2. Locate the GLB you created in the Load Balancers table.

3. Enable the Security and Performance features in the Proxy column:

   

Your GLB is now protected. An immediate benefit is that the origin IP addresses of your clusters will be hidden from the clients. If CIS detects a threat for an upcoming request, the user may see a screen like this one before being redirected to your application:

In addition, you can now control what content gets cached by CIS and how long it stays cached. Go to Performance > Caching to define the global caching level and the browser expiration. You can customize the global security and caching rules with Page Rules. Page Rules enable fine-grained configuration using specific domain paths. As example with Page Rules, you could decide to cache all contents under /assets for 3 days:

Remove Kubernetes Cluster resources

1. Remove the Ingress, you can do so by running the following command:

   kubectl delete -f glb-ingress.yaml
   

2. Remove the service, you can do so by running the following command:

   kubectl delete service hello-world-service
   

3. Remove the deployment, you can do so by running the following command:

   kubectl delete deployment hello-world-deployment
   

4. Delete the clusters if you created them specifically for this tutorial.

Remove CIS resources

1. Remove the GLB.
2. Remove the origin pools.
3. Remove the health checks.
4. Update the DNS for your custom domain.
5. Delete the CIS instance if you created it specifically for this tutorial.