Deploy microservices with Red Hat OpenShift on IBM Cloud

Initialize a Cloud Shell

The Red Hat OpenShift Container Platform CLI exposes commands for managing your applications, as well as lower level tools to interact with each component of your system. The CLI is available using the oc command.

To avoid installing the command line tools, the recommended approach is to use the IBM Cloud Shell.

IBM Cloud Shell is a cloud-based shell workspace that you can access through your browser. It's preconfigured with the full IBM Cloud CLI and many plug-ins and tools that you can use to manage apps, resources, and infrastructure.

In this step, you'll use the IBM Cloud shell and configure oc to point to the cluster assigned to you.

1. When the cluster is ready, click the button (next to your account) in the upper right corner to launch a Cloud shell. Make sure you don't close this window/tab.

2. Check the version of the OpenShift CLI:

   oc version
   

   The version needs to be at minimum 4.12.x, otherwise install the latest version by following these instructions.

3. Validate your cluster is shown when listing all clusters:

   ibmcloud oc clusters
   

4. Initialize the oc command environment by replacing the placeholder <your-cluster-name>:

   ibmcloud oc cluster config -c <your-cluster-name> --admin
   

5. Verify the oc command is working:

   oc get projects
   

Create Project

A project is a collection of resources managed by a DevOps team. An administrator will create the project and the developers can create applications that can be built and deployed.

1. Navigate to the Red Hat OpenShift web console by clicking the OpenShift web console button in the selected Cluster.
2. On the left navigation pane, under the Administrator perspective, select Home > Projects view to display all the projects.
3. Create a new project by clicking Create Project. In the pop up Name the project example-health, leave Display Name and Description blank, click Create.
4. The new project's Project Details page is displayed. Observe that your context is Administrator > Home > Projects on the left and Projects > Project details > example-health on the top.

Build and Deploy Application

1. Switch from the Administrator to the Developer perspective. Your context should be Developer > +Add on the left and Project: example-health on the top.

   

2. Let's build and deploy the application by selecting Import from Git.
3. Enter the repository https://github.com/IBM-Cloud/patient-health-frontend.git in the Git Repo URL field.
   • Note the green check Builder image detected and the Node.js 16 (UBI 8).
   • Note that the builder image automatically detected the language Node.js. If not detected, select Node.js from the provided list.
   • Builder Image Version leave at the default.
   • Application Name delete all of the characters and leave it empty (this will default to the Name)
   • Name : patient-health-frontend.
   • Click the Resource type link and choose DeploymentConfig.
   • Leave defaults for other selections.
4. Click Create at the bottom of the window to build and deploy the application.

View Application

1. You should see the app you just deployed. Notice that you are in the Topology view of the example-health project in the Developer perspective. All applications in the project are displayed.

2. Select the node patient-health-frontend to bring up the details view of the DeploymentConfig. Note the DC next to patient-health-frontend. The Pods, Builds, Services and Routes are visible.

   

   • Pods: Your Node.js application containers
   • Builds: The auto-generated build that created a Docker image from your Node.js source code, deployed it to the Red Hat OpenShift container registry, and kicked off your deployment config
   • Services: Tells Red Hat OpenShift how to access your Pods by grouping them together as a service and defining the port to listen to
   • Routes: Exposes your services to the outside world using the LoadBalancer provided by the IBM Cloud network

3. Click on View Logs next to your completed Build. This shows you the process that Red Hat OpenShift took to install the dependencies for your Node.js application and build/push a Docker image. The last entry should looks like this:

   Successfully pushed image-registry.openshift-image-registry.svc:5000/example-health/patient-health-frontend@sha256:f9385e010144f36353a74d16b6af10a028c12d005ab4fc0b1437137f6bd9e20a
   Push successful
   

4. Click back to the Topology and select your app again.

5. Click on the URL under Routes to visit your application. Enter any string for username and password, for instance test:test because the app is running in demonstration mode.

The Node.js app has been deployed to Red Hat OpenShift Container Platform. To recap:

• The "Example Health" Node.js application was deployed directly from GitHub into your cluster.
• The application was examined in the Red Hat OpenShift on IBM Cloud console.
• A Build Configuration was created - a new commit can be both built and deployed by clicking Start Build in the Builds section of the application details.

Simulate Load on the Application

Create a script to simulate load.

1. Make sure you're connected to the project where you deployed your app.

   oc project example-health
   

2. Retrieve the public route to access your application:

   oc get routes
   

   Output looks similar to this, note your value for Host:

   NAME         HOST/PORT                                                                                                 PATH      SERVICES     PORT       TERMINATION   WILDCARD
   patient-health-frontend   patient-health-frontend-example-health.roks07-872b77d77f69503584da5a379a38af9c-0000.eu-de.containers.appdomain.cloud             patient-health-frontend   8080-tcp                 None
   

3. Define a variable with the host:

   HOST=$(oc get routes -o json | jq -r '.items[0].spec.host')
   

4. Verify access to the application. It outputs patient information:

   curl -s -L http://$HOST/info
   

   Output should look like:

   $ curl -s -L http://$HOST/info
   {"personal":{"name":"Ralph DAlmeida","age":38,"gender":"male","street":"34 Main Street","city":"Toronto","zipcode":"M5H 1T1"},"medications":["Metoprolol","ACE inhibitors","Vitamin D"],"appointments":["2018-01-15 1:00 - Dentist","2018-02-14 4:00 - Internal Medicine","2018-09-30 8:00 - Pediatry"]}
   

5. Run the following script which will endlessly send requests to the application and generates traffic:

   while sleep 0.2; do curl --max-time 2 -s -L http://$HOST/info >/dev/null; echo -n "."
   done
   

   To stop the script, hit CTRL + c on your keyboard

Red Hat OpenShift on IBM Cloud Logging

Since there is only one pod, viewing the application logs will be straight forward.

1. Ensure that you're in the Topology view of the Developer perspective.

2. Navigate to your Pod by selecting your app.

3. Click on View Logs next to the name of the Pod under Pods to see streaming logs from your running application. If you're still generating traffic, you should see log messages for every request being made.

   

Red Hat OpenShift on IBM Cloud Terminal

One of the great things about Kubernetes is the ability to quickly debug your application pods with SSH terminals. This is great for development, but generally is not recommended in production environments. Red Hat OpenShift makes it even easier by allowing you to launch a terminal directly in the dashboard.

1. Switch from the Logs tab to the Terminal tab.
2. Run the following Shell commands:

Red Hat OpenShift on IBM Cloud Monitoring

When deploying new apps, making configuration changes, or simply inspecting the state of your cluster, the project-scope dashboard gives a Developer clear insights.

1. Access the dashboard in the Developer perspective by clicking Observe on the left side menu.
2. You can also dive in a bit deeper by clicking the Events tab. Events are useful for identifying the timeline of events and finding potential error messages. When tracking the state of a new rollout, managing existing assets, or even something simple like exposing a route, the Events view is critical in identifying the timeline of activity. This becomes even more useful when considering that multiple operators may be working against a single cluster.

Almost all actions in Red Hat OpenShift result in an event being fired in this view. As it is updated real-time, it's a great way to track changes to state.

Dashboards

Red Hat OpenShift comes with predefined dashboards to monitor your projects.

1. Get started by switching from the Developer perspective to the Administrator perspective:
2. Navigate to Observe > Dashboards in the left-hand bar.
3. Select Kubernetes / Compute Resources / Namespace (Pods) from the dropdown and Namespace to example-health.
4. Notice the CPU and Memory usage for your application. In production environments, this is helpful for identifying the average amount of CPU or Memory your application uses, especially as it can fluctuate through the day. Auto-scaling is one way to handle fluctuations and will be demonstrated a little later.

Metrics

Red Hat OpenShift provides a web interface to run queries and examine the metrics visualized on a plot. This functionality provides an extensive overview of the cluster state and enables you to troubleshoot problems.

1. Navigate to Observe > Metrics.

2. Enter the following expression and click Run queries. You should see the value and the graph associated with the query.

   sum(container_cpu_usage_seconds_total{container="patient-health-frontend"})
   

   

Enable Resource Limits

Before autoscaling maximum CPU and memory resource limits must be established.

The dashboards earlier showed you that the load was consuming anywhere between ".002" to ".02" cores. This translates to 2-20 "millicores". To be safe, let's bump the higher-end up to 30 millicores. In addition, the data showed that the app consumes about 25-65 MB of RAM. The following steps will set the resource limits in the deploymentConfig

1. Make sure the script to generate traffic is running.

2. Switch to the Administrator perspective.

3. Navigate to Workloads > DeploymentConfigs.

4. Select the example-health project.

5. From the Actions menu (the three vertical dots) of patient-health-frontend, choose Edit DeploymentConfig.

   

6. Under the YAML view, find the section spec > template > spec > containers, add the following resource limits into the empty resources. Replace the resources {}, and ensure the spacing is correct -- YAML uses strict indentation.

             resources:
               limits:
                 cpu: 30m
                 memory: 100Mi
               requests:
                 cpu: 3m
                 memory: 40Mi
   

   Here is a snippet after you have made the changes:

          ports:
            - containerPort: 8080
              protocol: TCP
          resources:
            limits:
              cpu: 30m
              memory: 100Mi
            requests:
              cpu: 3m
              memory: 40Mi
          terminationMessagePath: /dev/termination-log
   

7. Save to apply the changes.

8. Verify that the replication controller has been changed by navigating to Events tab:

   

Enable Autoscaler

Now that resource limits are configured, the pod autoscaler can be enabled.

By default, the autoscaler allows you to scale based on CPU or Memory. Pods are balanced between the minimum and maximum number of pods that you specify. With the autoscaler, pods are automatically created or deleted to ensure that the average CPU usage of the pods is below the CPU request target as defined. In general, you probably want to start scaling up when you get near 50-90% of the CPU usage of a pod. In our case, 1% can be used with the load being provided.

1. Navigate to Administrator perspective Workloads > HorizontalPodAutoscalers, then click Create HorizontalPodAutoscaler.

   

   Replace the contents of the editor with this yaml:

   apiVersion: autoscaling/v2
   kind: HorizontalPodAutoscaler
   metadata:
     name: patient-hpa
     namespace: example-health
   spec:
     scaleTargetRef:
       apiVersion: apps.openshift.io/v1
       kind: DeploymentConfig
       name: patient-health-frontend
     minReplicas: 1
     maxReplicas: 10
     metrics:
       - type: Resource
         resource:
           name: cpu
           target:
             averageUtilization: 1
             type: Utilization
   

2. Click Create.

Test Autoscaler

If you're not running the script to simulate load, the number of pods should stay at 1.

1. Check by opening the Overview page of the deployment config. Click Workloads > DeploymentConfigs and click patient-health-frontend and make sure the Details panel is selected.

2. Start simulating load (see previous section to simulate load on the application).

   

   It can take a few minutes for the autoscaler to make adjustments.

That's it! You now have a highly available and automatically scaled front-end Node.js application. Red Hat OpenShift is automatically scaling your application pods since the CPU usage of the pods greatly exceeded 1% of the resource limit, 30 millicores.

Autoscaling from the command line

You can also can delete and create resources like autoscalars with the command line.

1. Start by verifying the context is your project:

   oc project example-health
   

2. Get the autoscaler that was created earlier:

   oc get hpa
   

3. Delete the autoscaler made earlier:

   oc delete hpa/patient-hpa
   

4. Create a new autoscaler with a max of 9 pods:

   oc autoscale deploymentconfig/patient-health-frontend --name patient-hpa --min 1 --max 9 --cpu-percent=1
   

5. Revisit the Workloads > DeploymentConfigs Details page for patient-health-frontend deployment and watch it work.

Enable the IBM Cloud Operator

Let's understand exactly how Operators work. In the first exercise, you used a builder to deploy a simple application using a DeploymentConfig a default resource type that come with Red Hat OpenShift. A custom resource definition allows you to create resource types that do not come preinstalled with Red Hat OpenShift on IBM Cloud such an IBM Cloud service. Operators manage the lifecycle of resources and create Custom Resource Descriptors, CRDs, allowing you to manage custom resources the native "Kubernetes" way.

1. In the Administrator perspective, and click Operators > OperatorHub.
2. Find the IBM Cloud Operator, and click Install.
3. Keep the default options and click Install.
4. After a few seconds installed operator - ready for use should be displayed.

Create a Cloudant Service and Bind using the CRDs

Click to open it. Scroll down to the Prerequisites section.

An API key with the appropriate permissions to create a IBM Cloudant database is required in this section. The API key is going to be stored in a Kubernetes Secret resource. This will need to be created using the shell. There are instructions in the Prerequisites section of the installed operator. Steps:

1. Use the same resource group and region that is associated with your cluster.

   ibmcloud target -g <resource_group> -r <region>
   

   To see the the resource groups in your account, run ibmcloud resource groups command

2. Verify that the resource group and region matches your cluster. The following command should return your cluster.

   ibmcloud oc cluster ls
   

Output looks something like this:

$ ibmcloud oc cluster ls
OK
Name      ID                     State    Created        Workers   Location   Version                  Resource Group Name   Provider
osmicro   ck68svdd0vvcfs6ad9ag   normal   18 hours ago   2         Dallas     4.12.26_1562_openshift   default               vpc-gen2

3. Use the helper script provided by IBM to create the following resources:

   • IBM Cloud API key that represents you and your permissions to use IBM Cloud
   • Kubernetes Secret named secret-ibm-cloud-operator in the default namespace. This secret has the keys api-key and region. The operator will use this data to create the cloudant service instance.
   • Kubernetes ConfigMap resource with the name config-ibm-cloud-operator in the default namespace to hold the region and resource group

   Use the supplied curl command:

   curl -sL https://raw.githubusercontent.com/IBM/cloud-operators/master/hack/configure-operator.sh | bash
   

4. Back in the Red Hat OpenShift web console, click the Create service under the Service tab on the Installed Operators of the IBM Cloud Operator page and select YAML view to bring up the yaml editor.

5. Make the suggested substitutions where the serviceClass is cloudantnosqldb and the plan can be lite or standard (only one lite plan is allowed per account). Replace <your-initials>:

   apiVersion: ibmcloud.ibm.com/v1
   kind: Service
   metadata:
     annotations:
      ibmcloud.ibm.com/self-healing: enabled
     name: <your-initials>-cloudant-service
     namespace: example-health
   spec:
     serviceClass: cloudantnosqldb
     plan: standard
   

6. Click Create to create a IBM Cloudant database instance. Your context should be Operators > Installed Operators > IBM Cloud Operator in the Administrator perspective with Project: example-health in the Service panel.

7. Click on the service just created, <your-initials>-cloudant-service and over time the state field will change from provisioning to Online meaning it is good to go.

8. Create a Binding resource and a Secret resource for the cloudant Service resource just created. Navigate back to Operators > Installed Operators > IBM Cloud Operator > Binding tab. Open the Binding tab, click Create Binding and select YAML View. Create a cloudant-binding associated with the serviceName <your-initials>-cloudant-service, (this is the the name provided for the Service created earlier).

   apiVersion: ibmcloud.ibm.com/v1
   kind: Binding
   metadata:
     name: cloudant-binding
     namespace: example-health
   spec:
     serviceName: <your-initials>-cloudant-service
   

9. Optionally dig a little deeper to understand the relationship between the Red Hat OpenShift resources: Service, service Binding, binding Secret and the IBM Cloud resources: Service, service Instance and the instance's Service credentials. Using the cloud shell:

   ibmcloud resource service-instances --service-name cloudantnosqldb
   

   YOURINITIALS=<your-initials>
   

   ibmcloud resource service-instance $YOURINITIALS-cloudant-service
   

   ibmcloud resource service-keys --instance-name $YOURINITIALS-cloudant-service --output json
   

   Output looks something like this:

   youyou@cloudshell:~$ ibmcloud resource service-instances --service-name cloudantnosqldb
   Retrieving instances with type service_instance in all resource groups in all locations under ..
   OK
   Name                           Location   State    Type
   <your-initials>-cloudant-service               us-south   active   service_instance
   youyou@cloudshell:~$ ibmcloud resource service-instance <your-initials>-cloudant-service
   Retrieving service instance <your-initials>-cloudant-service in all resource groups under ...
   OK
   
   Name:                  <your-initials>-cloudant-service
   ID:                    crn:v1:bluemix:public:cloudantnosqldb:us-south:a/0123456789507a53135fe6793c37cc74:SECRET
   GUID:                  SECRET
   Location:              us-south
   Service Name:          cloudantnosqldb
   Service Plan Name:     standard
   Resource Group Name:   Default
   State:                 active
   Type:                  service_instance
   Sub Type:
   Created at:            2020-05-06T22:39:25Z
   Created by:            youyou@us.ibm.com
   Updated at:            2020-05-06T22:40:03Z
   Last Operation:
                       Status       create succeeded
                       Message      Provisioning is complete
                       Updated At   2020-05-06 22:40:03.04469305 +0000 UTC
   
   youyou@cloudshell:~$ ibmcloud resource service-keys --instance-name $YOURINITIALS-cloudant-service --output json
   [
       {
           "guid": "01234560-902d-4078-9a7f-20446a639aeb",
           "id": "crn:v1:bluemix:public:cloudantnosqldb:us-south:a/0123456789507a53135fe6793c37cc74:SECRET",
           "url": "/v2/resource_keys/01234560-902d-4078-9a7f-20446a639aeb",
           "created_at": "2020-05-06T23:03:43.484872077Z",
           "updated_at": "2020-05-06T23:03:43.484872077Z",
           "deleted_at": null,
           "name": "cloudant-binding",
           "account_id": "0123456789507a53135fe6793c37cc74",
           "resource_group_id": "01234567836d49029966ab5be7fe50b5",
           "source_crn": "crn:v1:bluemix:public:cloudantnosqldb:us-south:a/0123456789507a53135fe6793c37cc74:SECRET",
           "state": "active",
           "credentials": {
               "apikey": "SECRET",
               "host": "SECRET",
               "iam_apikey_description": "Auto-generated for key SECRET",
               "iam_apikey_name": "cloudant-binding",
               "iam_role_crn": "SECRET",
               "iam_serviceid_crn": "SECRET",
               "password": "SECRET",
               "port": 443,
               "url": "https://01234SECRET",
               "username": "01234567-SECRET"
           },
           "iam_compatible": true,
           "resource_instance_url": "/v2/resource_instances/SECRET",
           "crn": "crn:v1:bluemix:public:cloudantnosqldb:us-south:a/0123456789507a53135fe6793c37cc74:SECRET"
       }
   ]
   

Deploy the Node.js Patient Backend Database App

Now you'll create the Node.js app that will populate your Cloudant DB with patient data. It will also serve data to the front-end application deployed earlier.

1. Make sure you're your context is the project example-health:

   oc project example-health
   

2. The following new-app commmand will make a build configuration and Deployment Configuration. The following demonstrates the CLI invocation of the add application (remember using the GUI console for the frontend):

   oc new-app --name=patient-health-backend --as-deployment-config registry.access.redhat.com/ubi9/nodejs-20-minimal:latest~https://github.com/IBM-Cloud/patient-health-backend
   

3. Back in the console, and in the Topology view of the Developer perspective, open the patient-health-backend app and wait for the build to complete. Notice that the Pod is failing to start. Click on the Pod logs to see:

   > node app.js
   
   /opt/app-root/src/app.js:23
            throw("Cannot find Cloudant credentials, set CLOUDANT_URL.")
            ^
   Cannot find Cloudant credentials, set CLOUDANT_URL.
   

4. Let's fix this by setting the environment variable of the DeploymentConfig to the cloudant-binding secret created earlier in the operator binding section. Navigate to the deployment config for the patient-health-backend app by clicking the app, and then selecting the name next to DC:

   

5. Go to the Environment tab, click Add from ConfigMap or Secret and create a new environment variable named CLOUDANT_URL. Choose the cloudant-binding secret, then choose url for the Key. Click Save.

   

6. Go back to the Topology tab, and click the patient-health-backend. Check out the Pods section, which should indicate Running shortly. Click on View logs next to the running pod and notice the databases created.

Configure Patient Health Frontend App to use Patient Health Backend App

The patient-health-frontend application has an environment variable for the backend microservice url.

1. Set the API_URL environment variable to default in the frontend DeploymentConfig. Navigate to the deployment config for the patient-health-frontend app by clicking the frontend app in the Topology view, and then selecting the name next to DC:

2. Go to the Environment tab, and in the Single values (env) section add a name API_URL and value default. Click Save then Reload. This will result in a connection to http://patient-health-backend:8080/ which you can verify by looking at the pod logs. You can verify this is the correct port by scanning for the Pod Template / Containers / Port output of this command:

   oc describe dc/patient-health-backend
   

Your application is now backed by the mock patient data in the Cloudant DB! You can now log-in using any user-id/password in the Cloudant DB, use "opall:opall".

1. In a real-world application, these passwords should not be stored as plain-text. To review the patients (and alternate logins) in the Cloudant DB, navigate to your services in IBM Cloud Resource List. Click <your-initials>-cloudant-service.
2. Launch the Cloudant dashboard by clicking on Launch Dashboard button and then click the patients db.
3. Click through the different patients you can log in as.

View pre-defined monitoring views and dashboards

Use views and dashboards to monitor your infrastructure, applications, and services. You can use pre-defined dashboards. You can also create custom dashboards through the Web UI or programmatically. You can backup and restore dashboards by using Python scripts.

The following table lists the different types of pre-defined dashboards:

View the Monitoring dashboard

1. Navigate to Red Hat OpenShift on IBM Cloud clusters and notice the Red Hat OpenShift clusters
2. Click on your cluster and verify the Overview tab on the left is selected
3. In the Integrations section next to Monitoring, click the Launch button.

Initial data may NOT be available on newly created Monitoring instances.

• After a few minutes, raw data will be displayed
• After about an hour, indexing will provides the detail required to proceed with this tutorial

1. Under the Dashboards section, select Kubernetes > Pod Status & Performance to view raw metrics for all workloads running on the cluster.
2. Set the namespace filter to example-health to focus on the pods of your application.
3. Under Dashboards on the left pane, expand Applications in Dashboard Templates. Then select HTTP to get a global view of the cluster HTTP load.

Explore the cluster and the node capacity

1. Select Dashboards, check out the two dashboard templates:

   • Containers > Container Resource Usage
   • Host Infrastructure > Host Resource Usage

2. Select Kubernetes > Pod Rightsizing & Workload Capacity Optimization template. This dashboard helps you to optimize your infrastructure and better control cluster spend by ensure pods are sized correctly. Understand if you can free up resources by reducing memory and/or CPU requests.

Explore the Application

1. Select Dashboards and the template Kubernetes > Workload Status & Performance.

   A detailed dashboard showing all the pods in the cluster.

2. Create a customized dashboard and then scope it to a specific namespace.

   • In the upper right click Copy to my Dashboards and name it Workload Status & Performanceapp example-health
   • Click Create and Open to create your own dashboard.
   • Edit the dashboard scope.
   • Set the filter for kube_namespace_name, is, example-health.
   • Click Save.

   The dashboard now shows information focused on the example-health namespace.

   Scroll down to the TimeCharts for HTTP Requests, Latency, Error, ... to understand the performance of the application.

   

Find more about IBM Cloud Monitoring in the IBM Cloud documentation.