Creating and managing VPC security groups
Virtual Private Cloud
4.14 and earlier: If you modify the default VPC security group, you must, at minimum, include the required rules so that your cluster works properly.
Adding security groups during cluster creation
When you create a VPC cluster, the default worker security group, which is named kube-<clusterID>
, is automatically created and applied to all workers in the cluster. You can also attach additional security groups alongside
the default worker security group. The security groups applied to the workers in the cluster are a combination of the security groups applied when you create the cluster and when you create the worker pool. A total of five security groups
can be applied to workers, including the default security groups and any security groups applied to the worker pool. Note that the option to add custom security groups to workers is only available in the CLI.
The security groups applied to a cluster cannot be changed once the cluster is created. You can change the rules of the security groups that are applied to the cluster, but you cannot add or remove security groups at the cluster level. If you apply the incorrect security groups at cluster create time, you must delete the cluster and create a new one.
If you only want the default security groups
VPC security group Worker security group
Note that this is the default behavior at cluster create time.
When you create your cluster, do not specify any additional security groups.
Example command to create a VPC cluster with only the default VPC and kube-<clusterID>
cluster security groups:
ibmcloud oc cluster create vpc-gen2 --name <cluster-name> --zone <zone> --vpc-id <vpc-id> --subnet-id <subnet-id>
The following security groups are applied:
- The default VPC security group (randomly generated name)
- The
kube-<clusterID>
worker security group.
If you want the cluster security group and your own additional security groups
VPC security group Worker security group Your own security groups
When you create the cluster, specify --cluster-security-group cluster
and up to four additional security groups that you own. You must include a separate --cluster-security-group
option for each individual security
group you want to add. Note that at maximum of five security groups can be applied to workers, including the security groups that are applied by default.
Example command to create a VPC cluster with the kube-<clusterID>
cluster security group and your own additional security groups:
ibmcloud oc cluster create vpc-gen2 --name <cluster-name> --zone <zone> --vpc-id <vpc-id> --subnet-id <subnet-id> --cluster-security-group cluster --cluster-security-group <group-id-1> --cluster-security-group <group-id-2> --cluster-security-group <group-id-3>
The following security groups are applied:
- The default VPC security group (randomly generated name).
kube-<clusterID>
worker security group- Up to four of your own additional security groups, for a maximum of five total security groups.
If you only want your own security groups
VPC security group Your own security groups
This option is only available in clusters created at versions 4.14 and earlier. Beginning with 4.15, all clusters get the worker node security group as well.
When you create the cluster, specify up to five security groups that you own. You must include a separate --cluster-security-group
option for each individual security group you want to add.
Example command to create a VPC cluster with only your own security groups:
ibmcloud oc cluster create vpc-gen2 --name <cluster-name> --zone <zone> --vpc-id <vpc-id> --subnet-id <subnet-id> --cluster-security-group <group-id-1> --cluster-security-group <group-id-2> --cluster-security-group <group-id-3>
The following security groups are applied:
- The default VPC security group (randomly generated name)
- Up to four of your own additional security groups, for a maximum of five total security groups.
Adding security groups to worker pools during creation
By default, the security groups applied to a worker pool are the same security groups that are indicated at cluster create time. However, you can specify additional security groups to apply to a worker pool. If you apply additional security groups to the worker pool, then the security group applied to the workers on the cluster is a combination of the security groups applied at cluster create and the security groups applied to the worker pool.
A maximum of five security groups can be applied to a worker, including the security groups applied by default.
The security groups applied to a worker pool cannot be changed once the worker pool is created. You can change the rules of the security groups that are applied to the worker pool, but you cannot add or remove security groups at the worker pool level. If you apply the incorrect security groups at worker pool create time, you must delete the worker pool and create a new one.
If you do not want to attach additional security groups to the worker pool
When you create the worker pool, do not specify any additional security groups.
This scenario only applies to versions 4.14 and earlier.
Example command to create a worker pool with no security groups applied:
ibmcloud oc worker-pool create vpc-gen2 --name <worker_pool_name> --cluster <cluster_name_or_ID> --flavor <flavor> --size-per-zone <number_of_workers_per_zone>
Only the security groups applied to the cluster are applied to the workers.
If you want to attach additional security groups to the worker pool
When you create the worker pool specify additional security groups at worker pool create time. You must include a separate --security-group
option for each individual security group you want to add.
Example command to create a worker pool with your own security groups applied:
ibmcloud oc worker-pool create vpc-gen2 --name <worker_pool_name> --cluster <cluster_name_or_ID> --flavor <flavor> --size-per-zone <number_of_workers_per_zone> --security-group <group-id-1> --security-group <group-id-2> --security-group <group-id-3>
The security groups applied to the workers in the worker pool are a combination of those applied to the cluster that the worker pool is attached to and those applied to the worker pool at create time.
Viewing security groups
Follow the steps to view details about the VPC security groups.
-
List your clusters and note the ID of the cluster that you are working in.
To check what VPC a cluster is in, run
ibmcloud oc cluster get --cluster <cluster_name_or_id>
and check the VPC ID in the output.ibmcloud oc cluster ls --provider vpc-gen2
-
List the security groups attached to the VPC that your cluster is in. The VPC security group is assigned a randomly generated name, such as
trench-hexagon-matriarch-flower
. The cluster security group is named in the format ofkube-<cluster-ID>
. The VPC security group is named in the format ofkube-<vpc-ID>
.ibmcloud is sgs | grep <vpc_name>
Example output
ID Name Rules Network interfaces VPC Resource group r006-111aa1aa-1a1a-1a11-1111-a111aaa1a11a trench-hexagon-matriarch-flower 4 0 my-vpc default r006-222aa2aa-2a2a-2a22-2222-a222aaa2a22a kube-a111a11a11aa1aa11a11 4 0 my-vpc default r006-333aa3aa-3a3a-3a33-3333-a333aaa3a33a kube-r006-111a11aa-aaa1-1a1a-aa11-1a1a111aa11 4 0 my-vpc default
-
Get the details of a security group. Find the Rules section in the output to view the inbound and outbound rules attached to the security group.
ibmcloud is sg GROUP
Example output
... Rules ID Direction IP version Protocol Remote r006-111bb1bb-1b1b-1b11-1111-b111bbb1b11b outbound ipv4 all 0.0.0.0/0 r006-222bb2bb-2b2b-2b22-2222-b222bbb2b22b inbound ipv4 all behind-unbuilt-guidable-anthill r006-333bb3bb-3b3b-3b33-3333-b333bbb3b33b inbound ipv4 icmp Type=8 0.0.0.0/0 r006-444bb4bb-4b4b-4b44-4444-b444bbb4b44b inbound ipv4 tcp Ports:Min=22,Max=22 0.0.0.0/0
Viewing security groups in the console
-
From the Security groups for VPC dashboard, find the security groups that are attached to the VPC that your cluster is in.
-
Click on the security group.
To sort the security groups by the VPC they are attached to, click the Virtual Private Cloud column heading in the table.
-
To view the inbound and outbound rules attached to the security group, click the Rules tab.
Creating security group rules in the console
- From the Virtual private cloud dashboard, click the name of the Default Security Group for the VPC that your cluster is in.
- Click the Rules tab.
- To create new inbound rules to control inbound traffic to your worker nodes, in the Inbound rules section, click Create.
- To create new rules to control outbound traffic to your worker nodes, in the Outbound rules section, delete the default rule that allows all outbound traffic. Then, in the Outbound rules section, click Create.
Keep in mind that in addition to any rules you create, you must also create the required inbound and outbound rules.
Creating security group rules in the command line
Use the IBM Cloud CLI to add inbound and outbound rules to the default security group for your cluster.
- Install the
infrastructure-service
plug-in. The prefix for running commands isibmcloud is
.ibmcloud plugin install infrastructure-service
- Target the region that your VPC is in.
ibmcloud target -r REGION
- Get your cluster's ID.
ibmcloud oc cluster get -c CLUSTER
- List your security groups and note the ID of the default security group for your VPC. Note that the default security group uses a randomly generated name and not the format
kube-<cluster_ID>
.
Example output with the default security group for the VPC of a randomly generated name,ibmcloud is sgs
chamomile-dislodge-showier-unfilled
.ID Name Rules Network interfaces VPC Resource group 1a111a1a-a111-11a1-a111-111111111111 chamomile-dislodge-showier-unfilled 5 2 events-vpc default 2b222b2b-b222-22b2-b222-222222222222 kube-df253b6025d64744ab99ed63bb4567b6 5 3 gen2-vpn default
- Store the security group ID as an environment variable.
sg=GROUP
- Review the default rules for the security group. Keep in mind that in addition to any rules you create, you must also create the required inbound and outbound rules.
ibmcloud is sg $sg
-
To create inbound traffic rules, use the
ibmcloud is sg-rulec <sg> inbound
command.ibmcloud is sg-rulec $sg inbound <protocol> [--remote <remote_address> | <CIDR_block> | <security_group_ID>] [--icmp-type <icmp_type> [--icmp-code <icmp_code>]] [--port-min <port_min>] [--port-max <port_max>]
-
To create outbound traffic rules, use the
ibmcloud is sg-rulec <sg> outbound
command.ibmcloud is sg-rulec $sg outbound <protocol> [--remote <remote_address> | <CIDR_block> | <security_group_ID>] [--icmp-type <icmp_type> [--icmp-code <icmp_code>]] [--port-min <port_min>] [--port-max <port_max>]
-
After you create custom outbound rules, get the ID of the default rule that allows all outbound traffic.
ibmcloud is sg $sg
Example output
Rules ID Direction IP version Protocol Remote r010-e3a34cbb-d5e8-4713-a57e-3e35a7458272 inbound ipv4 all freeload-flavored-surging-repaying r010-036c3a13-1c16-4425-9667-a4ec34b1702b inbound ipv4 icmp Type=8 0.0.0.0/0 r010-15591636-6976-493f-a94f-70721702860a inbound ipv4 tcp Ports:Min=22,Max=22 0.0.0.0/0
-
Delete the default rule that allows all outbound traffic.
ibmcloud is security-group-rule-delete $sg <rule_ID>
-
-
Keep in mind that you can create your own security groups as well as add or remove rules addition to any rules you create, you must also create the required inbound and outbound rules.
Allow worker nodes to connect to the public service endpoint IP addresses for the Oauth service
Required for VPC clusters with a public service endpoint. This scenario applies only to versions 4.14 and earlier.
-
Get the URL used to connect to the oauth service in the form
https://cXXX-e.<region>.containers.cloud.ibm.com:<PORT>
. Make a note of the region and port.oc get --raw /.well-known/oauth-authorization-server | grep issuer
Example output
"issuer": "https://c104-e.us-east.containers.cloud.ibm.com:31062",
-
Run
dig +short
ornslookup
to get the IPs associated with the hostname in the URL. For single zone regions there is only one public IP associated with the hostname. For multizone regions, there are 3. Look up the individual IPs by replacinge
with 1, 2, and 3.dig +short cXXX-1.<region>.containers.cloud.ibm.com dig +short cXXX-2.<region>.containers.cloud.ibm.com dig +short cXXX-3.<region>.containers.cloud.ibm.com
Example output
169.63.111.82
-
Add a security group rule for each of the IPs that allows an outbound TCP connection to the destination IP and PORT.
ibmcloud is sg-rulec <sg> outbound tcp --port-min 31062 --port-max 31062 --remote 169.63.111.82
Example output
Direction outbound IP version ipv4 Protocol tcp Min destination port 31062 Max destination port 31062 Remote 169.63.111.82
Allow worker nodes to connect to the Ingress LoadBalancer
Follow the steps to allow worker nodes to connect to the Ingress LoadBalancer.
This scenario applies only to versions 4.14 and earlier.
-
Get the
EXTERNAL-IP
of the LoadBalancer service.oc get svc -o wide -n openshift-ingress router-default
-
Run
dig
on theEXTERNAL-IP
to get the IP addresses associated with the LoadBalancer.dig <EXTERNAL-IP> +short
Example output
150.XXX.XXX.XXX 169.XX.XXX.XXX
-
Create outbound security rules to each of the IP address that you retrieved earlier and port 443.
ibmcloud is sg-rulec <sg> outbound tcp --port-min 443 --port-max 443 --remote 150.XXX.XXX.XXX
If the Ingress or console operators fail their health checks, you can repeat these steps to see if the LoadBalancer IP addresses changed. While rare, if the amount of traffic to your LoadBalancers varies widely, these IP addresses might change to handle the increased or decreased load.