VMware Solutions and Red Hat OpenShift overview

Red Hat OpenShift overview

The Red Hat OpenShift platform is a platform that is designed to orchestrate containerized workloads across a cluster of nodes. The platform uses Kubernetes as the core container orchestration engine, which manages the Docker container images and their lifecycle.

The operating system of the nodes is Red Hat® Enterprise Linux® CoreOS, which is the container host version of Red Hat Enterprise Linux (RHEL) and features an RHEL kernel with SELinux enabled by default. RHEL CoreOS includes kubelet, which is the Kubernetes node agent, and the CRI-O container runtime, which is optimized for Kubernetes. In Red Hat OpenShift 4.7, you must use RHEL CoreOS for all control plane machines, but you can use Red Hat Enterprise Linux (RHEL) as the operating system for compute, or worker machines. If you choose to use RHEL workers, you must perform more system maintenance than if you use RHEL CoreOS for all of the cluster machines.

The reference architecture and this build process use RHEL CoreOS. The nodes must have direct Internet access to complete the following tasks.

• Access the Red Hat OpenShift Infrastructure Providers page to download the installation program.
• Access quay.io to obtain the packages that are required to install the cluster.
• Obtain the packages that are required to perform cluster updates.
• Access Red Hat’s software as a service page to perform subscription management.

In the reference architecture, the following components are installed and configured in the build process:

• Bastion node - This RHEL VM acts as a "jump-server" on the overlay network to enable the build process. It is accessed by using SSH through the private network. It also hosts a webserver to help the build process of the cluster.
• Bootstrap node - As each node in the cluster requires information about the cluster when it is provisioned, a temporary bootstrap node is used. The bootstrap node creates the control plane nodes that make up the control plane. The control plane nodes then create the worker nodes. After the cluster initializes, the bootstrap node can be destroyed.
• Control-plane nodes - These nodes run services that are required to control the Kubernetes cluster. They contain more than just the Kubernetes services for managing the cluster. The terms "primary" and "control-plane" are used interchangeably.
• Compute nodes - In a Kubernetes cluster, the compute nodes are where the workloads are run. The compute nodes advertise their capacity to the control-plane nodes.
• DNS - A correct DNS setup is imperative for a functioning Red Hat OpenShift cluster. The AD DNS server of the vCenter Server instance hosts the required DNS records.
• Load-balancer - An NSX ESG load-balancer service is used to front end the Red Hat OpenShift APIs, both internal and external, and the Red Hat OpenShift router. The load balancer is configured so that Port 6443 and 22623 point to the bootstrap and control plane nodes, while ports 80 and 443 are configured to point to the worker nodes.
• Webserver - A web server is needed to hold the ignition configurations and installation images for the installation of RHEL CoreOS. NGINX is installed on the bastion node to provide this function.
• Persistent storage - To support the persistent storage requirements, the vSphere cloud provider is used to provide storage volumes up to the Red Hat OpenShift platform backed by any supported vSphere datastore (VMware® vSAN™, NFS, or iSCSI). Red Hat OpenShift can deliver storage through static or dynamic provisioning. The preferred method is to use dynamic provisioning. Dynamic provisioning automatically triggers the creation of the persistent volume and its backend VMDK file. For dynamic provisioning, a default StorageClass for the Red Hat OpenShift cluster is defined and a PersistentVolumeClaim in Kubernetes is created.

Access to the environment for this build process is done through a "jump-server" or remote device:

• You can have a Microsoft Windows® or Linux Virtual Server Instance (VSI) installed alongside your vCenter Server instance to provide administrative access to the environment. This VSI has internet access for the remote connection to the server and for downloading files. It also has private network access for connecting to vCenter and to the bastion node.
• You can have a remote device (laptop or desktop) connected through the IBM Cloud SSL VPN to the IBM Cloud Private network. This remote device has access to the internet to download the required files and can connect to vCenter and the bastion node through the SSL VPN.

Scripts overview

This build process uses the following scripting tools and scripts:

• govc is a vSphere CLI pre-compiled for Linux, OSX, and Windows. The CLI is useful to complete various vCenter Server and vSphere operations and it's used in this process to complete the following tasks:
  • Upload the ISO and OVA files to a vSphere datastore.
  • Create a persistent volume for the Red Hat OpenShift cluster to use.
• Red Hat Installer - The installer creates the Ignition files that are used by Terraform. Ignition is a first boot installer and configuration tool that is designed specifically for CoreOS Container Linux to partition disks, format partitions, writing files and configuring users. On first boot, Ignition reads its configuration from a remote URL and applies the configuration to the VM.
• Terraform - Terraform is a tool that codifies APIs into declarative configuration files. It uses a "provider" to interact with the infrastructure. A .tf file is used to describe the infrastructure as code. A .tvars file is used to store the variables for the deployment. The main.tf file holds the build instructions.
• PowerShell is used on the AD DNS server to create the DNS entries.
• PowerShell Core is available for Windows, Linux, and macOS systems. This tool is optional and it can be used on the jump-server or remote device to enable the use of PowerCLI. For more information, see PowerShell Core.
• PowerCLI - PowerCLI is a PowerShell interface for managing VMware vSphere®. You can automate all aspects of vSphere management, including network, storage, VMs, and guest OS. This tool is optional and it can be used on the jump-server or remote device to help the preparation of the vCenter Server instance. These steps can be done by using the GUI. For more information about how to install this tool, see Install PowerCLI.
• PowerNSX - PowerNSX is a PowerShell module that abstracts the VMware NSX API to a set of easily used PowerShell functions. This tool is optional and it can be used on the jump-server or remote device to help the preparation of NSX on the vCenter Server instance. These steps can be done by using the GUI. For more information, see powernsx.

Build process overview

This documentation describes the process to install Red Hat OpenShift v4.7 on to an existing vCenter Server instance. The process installs and configures:

• One bastion node.
• One bootstrap node.
• Three control plane nodes.
• Three worker nodes.

The deployment approach is described in the following phases:

• Phase 1 - vCenter Server instance preparation.
  • Using the IBM Cloud® for VMware Solutions console, order a vCenter Server instance, which can include NFS or vSAN storage. You can use an existing instance if the instance has enough capacity.
  • Using the IBM Cloud console, order more private and public subnets to be used by the Red Hat OpenShift cluster.
  • Download RHEL 8.0 ISO for the OS of the bastion or deployment node and the Red Hat Enterprise Linux CoreOS (RHCOS) OVA image. This step is described in Prerequisites for installation.
  • Using govc, the OVA and ISO are uploaded to a datastore on the vCenter Server instance. This step is described in Prerequisites for installation.
  • Add logical switches - Two logical switches are created: OpenShift-LS - the network that the Red Hat OpenShift VMs are deployed onto and OpenShift-DLR-Transit - the uplink between the DLR and the Edge.
  • Add an ESG - An external services gateway (ESG) is a virtual appliance that provides North-South routing, and other network functions. In this architecture, the ESG is used for; routing, NAT, firewall, and load-balancing. As the ESGs are configured as an active-passive pair, DRS anti-affinity rules are used to ensure that NSX Edges do not run on the same host. Static routes are used to direct traffic to either the internet or the IBM private Network. This step is described in Red Hat OpenShift NSX Edge configuration.
  • Add a DLR - A distributed logical router (DLR) is a virtual appliance that contains the routing control plane and it distributes the data plane in kernel modules to each hypervisor host. The DLR provides East-West distributed routing and is the default gateway for the Red Hat OpenShift VMs that are installed on the Red Hat OpenShift logical switch. The NSX DLR virtual machines are configured as an active - passive pair, and vSphere Distributed Resource Scheduler (DRS) anti-affinity rules are created to ensure that the DLR VMs do not run on the same host. This step is described in Red Hat OpenShift NSX DLR configuration.
  • Update DNS - The infrastructure DNS, provisioned with the vCenter Server instance is updated with the names and IP addresses for the Red Hat OpenShift components by using a PowerShell script. This step is described in VMware Solutions DNS configuration.
• Phase 2 - Red Hat OpenShift installation. These steps are described in Red Hat OpenShift 4.7 user provider infrastructure installation.
  • A Red Hat virtual machine, the bastion node, is provisioned to run the Red Hat OpenShift installer and to host an HTTP Server. It is registered with Red Hat by using your subscription, and the Red Hat OpenShift installer is downloaded.
  • On the bastion node, the install-config.yaml file is populated with the required Red Hat OpenShift parameters and Red Hat OpenShift ignition is used to generate a number of files used for the installation of the bootstrap, control plane, and worker machines.
  • Terraform, on the bastion node, uses the files that are created by Ignition to create the Red Hat OpenShift VMs.
• Phase 3 - Post deployment activities. Configure a persistent volume for use by the Red Hat OpenShift cluster. This step is described in Red Hat OpenShift 4.7 additional configuration.

Related links

• Video - What is Red Hat OpenShift?
• Red Hat OpenShift 4 Release Update
• Installing a cluster on vSphere with user-provisioned infrastructure
• Getting started with IBM Cloud Virtual Private Networking