IBM Cloud Docs
Creating and managing VPC security groups

Creating and managing VPC security groups

Virtual Private Cloud

4.14 and earlier: If you modify the default VPC security group, you must, at minimum, include the required rules so that your cluster works properly.

Adding security groups during cluster creation

When you create a VPC cluster, the default worker security group, which is named kube-<clusterID>, is automatically created and applied to all workers in the cluster. You can also attach additional security groups alongside the default worker security group. The security groups applied to the workers in the cluster are a combination of the security groups applied when you create the cluster and when you create the worker pool. A total of five security groups can be applied to workers, including the default security groups and any security groups applied to the worker pool. Note that the option to add custom security groups to workers is only available in the CLI.

The security groups applied to a cluster cannot be changed once the cluster is created. You can change the rules of the security groups that are applied to the cluster, but you cannot add or remove security groups at the cluster level. If you apply the incorrect security groups at cluster create time, you must delete the cluster and create a new one.

If you only want the default security groups

VPC security group Worker security group

Note that this is the default behavior at cluster create time.

When you create your cluster, do not specify any additional security groups.

Example command to create a VPC cluster with only the default VPC and kube-<clusterID> cluster security groups:

 ibmcloud oc cluster create vpc-gen2 --name <cluster-name> --zone <zone> --vpc-id <vpc-id> --subnet-id <subnet-id>

The following security groups are applied:

  • The default VPC security group (randomly generated name)
  • The kube-<clusterID> worker security group.

If you want the cluster security group and your own additional security groups

VPC security group Worker security group Your own security groups

When you create the cluster, specify --cluster-security-group cluster and up to four additional security groups that you own. You must include a separate --cluster-security-group option for each individual security group you want to add. Note that at maximum of five security groups can be applied to workers, including the security groups that are applied by default.

Example command to create a VPC cluster with the kube-<clusterID> cluster security group and your own additional security groups:

 ibmcloud oc cluster create vpc-gen2 --name <cluster-name> --zone <zone> --vpc-id <vpc-id> --subnet-id <subnet-id> --cluster-security-group cluster --cluster-security-group <group-id-1> --cluster-security-group <group-id-2> --cluster-security-group <group-id-3>

The following security groups are applied:

  • The default VPC security group (randomly generated name).
  • kube-<clusterID> worker security group
  • Up to four of your own additional security groups, for a maximum of five total security groups.

If you only want your own security groups

VPC security group Your own security groups

This option is only available in clusters created at versions 4.14 and earlier. Beginning with 4.15, all clusters get the worker node security group as well.

When you create the cluster, specify up to five security groups that you own. You must include a separate --cluster-security-group option for each individual security group you want to add.

Example command to create a VPC cluster with only your own security groups:

 ibmcloud oc cluster create vpc-gen2 --name <cluster-name> --zone <zone> --vpc-id <vpc-id> --subnet-id <subnet-id> --cluster-security-group <group-id-1> --cluster-security-group <group-id-2> --cluster-security-group <group-id-3>

The following security groups are applied:

  • The default VPC security group (randomly generated name)
  • Up to four of your own additional security groups, for a maximum of five total security groups.

Adding security groups to worker pools during creation

By default, the security groups applied to a worker pool are the same security groups that are indicated at cluster create time. However, you can specify additional security groups to apply to a worker pool. If you apply additional security groups to the worker pool, then the security group applied to the workers on the cluster is a combination of the security groups applied at cluster create and the security groups applied to the worker pool.

A maximum of five security groups can be applied to a worker, including the security groups applied by default.

The security groups applied to a worker pool cannot be changed once the worker pool is created. You can change the rules of the security groups that are applied to the worker pool, but you cannot add or remove security groups at the worker pool level. If you apply the incorrect security groups at worker pool create time, you must delete the worker pool and create a new one.

If you do not want to attach additional security groups to the worker pool

When you create the worker pool, do not specify any additional security groups.

This scenario only applies to versions 4.14 and earlier.

Example command to create a worker pool with no security groups applied:

ibmcloud oc worker-pool create vpc-gen2 --name <worker_pool_name> --cluster <cluster_name_or_ID> --flavor <flavor> --size-per-zone <number_of_workers_per_zone>

Only the security groups applied to the cluster are applied to the workers.

If you want to attach extra security groups to the worker pool

When you create the worker pool specify extra security groups at worker pool create time. You must include a separate --security-group option for each individual security group you want to add.

Example command to create a worker pool with your own security groups applied:

ibmcloud oc worker-pool create vpc-gen2 --name <worker_pool_name> --cluster <cluster_name_or_ID> --flavor <flavor> --size-per-zone <number_of_workers_per_zone> --security-group <group-id-1> --security-group <group-id-2> --security-group <group-id-3>

The security groups applied to the workers in the worker pool are a combination of those applied to the cluster that the worker pool is attached to and those applied to the worker pool at create time.

Viewing security groups

Follow the steps to view details about the VPC security groups.

  1. List your clusters and note the ID of the cluster that you are working in.

    To check what VPC a cluster is in, run ibmcloud oc cluster get --cluster <cluster_name_or_id> and check the VPC ID in the output.

    ibmcloud oc cluster ls --provider vpc-gen2

  2. List the security groups attached to the VPC that your cluster is in. The VPC security group is assigned a randomly generated name, such as trench-hexagon-matriarch-flower. The cluster security group is named in the format of kube-<cluster-ID>. The VPC security group is named in the format of kube-<vpc-ID>.

    ibmcloud is sgs | grep <vpc_name>

    Example output

    ID                                          Name                                             Rules   Network interfaces   VPC                          Resource group   

r006-111aa1aa-1a1a-1a11-1111-a111aaa1a11a   trench-hexagon-matriarch-flower                    4       0                    my-vpc                      default   
r006-222aa2aa-2a2a-2a22-2222-a222aaa2a22a   kube-a111a11a11aa1aa11a11                          4       0                    my-vpc                      default   
r006-333aa3aa-3a3a-3a33-3333-a333aaa3a33a   kube-r006-111a11aa-aaa1-1a1a-aa11-1a1a111aa11      4       0                    my-vpc                      default

  3. Get the details of a security group. Find the Rules section in the output to view the inbound and outbound rules attached to the security group.

    ibmcloud is sg GROUP

    Example output

    ...
Rules      
ID                                          Direction   IP version   Protocol                  Remote   
r006-111bb1bb-1b1b-1b11-1111-b111bbb1b11b   outbound    ipv4         all                       0.0.0.0/0   
r006-222bb2bb-2b2b-2b22-2222-b222bbb2b22b   inbound     ipv4         all                       behind-unbuilt-guidable-anthill   
r006-333bb3bb-3b3b-3b33-3333-b333bbb3b33b   inbound     ipv4         icmp Type=8               0.0.0.0/0   
r006-444bb4bb-4b4b-4b44-4444-b444bbb4b44b   inbound     ipv4         tcp Ports:Min=22,Max=22   0.0.0.0/0

Viewing security groups in the console

  1. From the Security groups for VPC dashboard, find the security groups that are attached to the VPC that your cluster is in.

  2. Click on the security group.

    To sort the security groups by the VPC they are attached to, click the Virtual Private Cloud column heading in the table.

  3. To view the inbound and outbound rules attached to the security group, click the Rules tab.

Creating security group rules in the console

  1. From the Virtual private cloud dashboard, click the name of the Default Security Group for the VPC that your cluster is in.
  2. Click the Rules tab.
    • To create new inbound rules to control inbound traffic to your worker nodes, in the Inbound rules section, click Create.
    • To create new rules to control outbound traffic to your worker nodes, in the Outbound rules section, delete the default rule that allows all outbound traffic. Then, in the Outbound rules section, click Create.

Keep in mind that in addition to any rules you create, you must also create the required inbound and outbound rules.

Creating security group rules in the command line

Use the IBM Cloud CLI to add inbound and outbound rules to the default security group for your cluster.

  1. Install the infrastructure-service plug-in. The prefix for running commands is ibmcloud is.
    ibmcloud plugin install infrastructure-service
  2. Target the region that your VPC is in.
    ibmcloud target -r REGION
  3. Get your cluster's ID.
    ibmcloud oc cluster get -c CLUSTER
  4. List your security groups and note the ID of the default security group for your VPC. Note that the default security group uses a randomly generated name and not the format kube-<cluster_ID>. 
    ibmcloud is sgs
    Example output with the default security group for the VPC of a randomly generated name, chamomile-dislodge-showier-unfilled.
    ID                                          Name                                       Rules   Network interfaces   VPC          Resource group
1a111a1a-a111-11a1-a111-111111111111        chamomile-dislodge-showier-unfilled        5       2                    events-vpc   default
2b222b2b-b222-22b2-b222-222222222222        kube-df253b6025d64744ab99ed63bb4567b6      5       3                    gen2-vpn     default
  5. Store the security group ID as an environment variable.
    sg=GROUP
  6. Review the default rules for the security group. Keep in mind that in addition to any rules you create, you must also create the required inbound and outbound rules.
    ibmcloud is sg $sg

    • To create inbound traffic rules, use the ibmcloud is sg-rulec <sg> inbound command.

      ibmcloud is sg-rulec $sg inbound <protocol> [--remote <remote_address> | <CIDR_block> | <security_group_ID>] [--icmp-type <icmp_type> [--icmp-code <icmp_code>]] [--port-min <port_min>] [--port-max <port_max>]

    • To create outbound traffic rules, use the ibmcloud is sg-rulec <sg> outbound command.

      ibmcloud is sg-rulec $sg outbound <protocol> [--remote <remote_address> | <CIDR_block> | <security_group_ID>] [--icmp-type <icmp_type> [--icmp-code <icmp_code>]] [--port-min <port_min>] [--port-max <port_max>]

      1. After you create custom outbound rules, get the ID of the default rule that allows all outbound traffic.

        ibmcloud is sg $sg

        Example output

        Rules
ID                                          Direction   IP version   Protocol                        Remote
r010-e3a34cbb-d5e8-4713-a57e-3e35a7458272   inbound     ipv4         all                             freeload-flavored-surging-repaying
r010-036c3a13-1c16-4425-9667-a4ec34b1702b   inbound     ipv4         icmp Type=8                     0.0.0.0/0
r010-15591636-6976-493f-a94f-70721702860a   inbound     ipv4         tcp Ports:Min=22,Max=22         0.0.0.0/0

      2. Delete the default rule that allows all outbound traffic.

        ibmcloud is security-group-rule-delete $sg <rule_ID>

Keep in mind that you can create your own security groups as well as add or remove rules addition to any rules you create, you must also create the required inbound and outbound rules.

Allow worker nodes to connect to the public service endpoint IP addresses for the Oauth service

Required for VPC clusters with a public service endpoint. This scenario applies only to versions 4.14 and earlier.

Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.

  1. Get the URL used to connect to the oauth service in the form https://cXXX-e.<region>.containers.cloud.ibm.com:<PORT>. Make a note of the region and port.

    oc get --raw /.well-known/oauth-authorization-server | grep issuer

    Example output

    "issuer": "https://c104-e.us-east.containers.cloud.ibm.com:31062",

  2. Run dig +short or nslookup to get the IPs associated with the hostname in the URL. For single-campus multizone regions there is only one public IP associated with the hostname. For multizone regions, there are 3. Look up the individual IPs by replacing e with 1, 2, and 3.

    dig +short cXXX-1.<region>.containers.cloud.ibm.com
dig +short cXXX-2.<region>.containers.cloud.ibm.com
dig +short cXXX-3.<region>.containers.cloud.ibm.com

    Example output

    169.63.111.82

  3. Add a security group rule for each of the IPs that allows an outbound TCP connection to the destination IP and PORT.

    ibmcloud is sg-rulec <sg> outbound tcp --port-min 31062 --port-max 31062 --remote 169.63.111.82

    Example output

    Direction              outbound   
IP version             ipv4   
Protocol               tcp   
Min destination port   31062   
Max destination port   31062   
Remote                 169.63.111.82

Allow worker nodes to connect to the Ingress LoadBalancer

Follow the steps to allow worker nodes to connect to the Ingress LoadBalancer.

This scenario applies only to versions 4.14 and earlier.

  1. Get the EXTERNAL-IP of the LoadBalancer service.

    oc get svc -o wide -n openshift-ingress router-default

  2. Run dig on the EXTERNAL-IP to get the IP addresses associated with the LoadBalancer.

    dig <EXTERNAL-IP> +short

    Example output

    150.XXX.XXX.XXX
169.XX.XXX.XXX

  3. Create outbound security rules to each of the IP address that you retrieved earlier and port 443.

    ibmcloud is sg-rulec <sg> outbound tcp --port-min 443 --port-max 443 --remote 150.XXX.XXX.XXX

If the Ingress or console operators fail their health checks, you can repeat these steps to see if the LoadBalancer IP addresses changed. While rare, if the amount of traffic to your LoadBalancers varies widely, these IP addresses might change to handle the increased or decreased load.