Resolving permission and credential errors
When you try to make changes to your cluster, such as creating or deleting worker nodes or other components, you receive an error message similar to one of the following examples.
The infrastructure authentication credentials are not authorized for the request.
We were unable to connect to your Softlayer account.
Creating a standard cluster requires that you have either a
Pay-As-You-Go account that is linked to an IBM Cloud infrastructure
account term or that you have used the Kubernetes service
CLI to set your Infrastructure API keys.
'Item' must be ordered with permission.
The worker node instance '<ID>' can't be found. Review '<provider>' infrastructure user permissions.
The worker node instance can't be found. Review '<provider>' infrastructure user permissions.
The worker node instance can't be identified. Review '<provider>' infrastructure user permissions.
The IAM token exchange request failed with the message: <message>
IAM token exchange request failed: <message>
The cluster could not be configured with the registry. Make sure that you have the Administrator role for Container Registry.
The infrastructure credentials that are set for the region and resource group are missing the appropriate infrastructure permissions, or the credentials are not recognized.
There are multiple reasons why this can occur.
-
You do not have the required infrastructure permissions.
-
The resource group and region are mismatched.
- Credentials and the IAM API key are set to a region and a resource group. The region is specified when you run the
ibmcloud oc credential set
command. The resource group applied is whichever resource group is targeted when you run this command. If you do not explicitly target a resource group with theibmcloud target
command before you runibmcloud oc credential set
, it is possible that the API key's resource group is not the one you expected, resulting is mismatched credentials that are not recognized.
- Credentials and the IAM API key are set to a region and a resource group. The region is specified when you run the
-
Credentials were added to or removed from the cluster.
- If you created a cluster with a linked IBM Cloud infrastructure account and then later added or removed credentials with the
ibmcloud oc credential set
oribmcloud oc credential unset
, the credentials might not match the specifications for the linked account. This can result in the credentials being unrecognized.
- If you created a cluster with a linked IBM Cloud infrastructure account and then later added or removed credentials with the
The account owner must set up the infrastructure account credentials properly. The credentials depend on what type of infrastructure account you are using.
Before you begin, Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster..
-
Identify what user credentials are used for the region and resource group's infrastructure permissions.
- Check the API key for a region and resource group of the cluster.
Example outputibmcloud oc api-key info --cluster <cluster_name_or_ID>
Getting information about the API key owner for cluster <cluster_name>... OK Name Email <user_name> <name@email.com>
- Check if the classic infrastructure account for the region and resource group is manually set to use a different IBM Cloud infrastructure account.
Example output if credentials are set to use a different classic account. In this case, the user's infrastructure credentials are used for the region and resource group that you targeted, even if a different user's credentials are stored in the API key that you retrieved in the previous step.ibmcloud oc credential get --region <us-south>
Example output if credentials are not set to use a different classic account. In this case, the API key owner that you retrieved in the previous step has the infrastructure credentials that are used for the region and resource group.OK Infrastructure credentials for user name <1234567_name@email.com> set for resource group <resource_group_name>.
FAILED No credentials set for resource group <resource_group_name>.: The user credentials could not be found. (E0051)
- Check the API key for a region and resource group of the cluster.
-
Validate the infrastructure permissions that the user has.
-
List the suggested and required infrastructure permissions for the region and resource group.
ibmcloud oc infra-permissions get --region <region>
For console and CLI commands to assign these permissions, see Classic infrastructure roles.
-
Make sure that the infrastructure credentials owner for the API key or the manually set account has the correct permissions. You can change the API key or manually set infrastructure credentials owner for the region and resource group.
-
-
Try again to perform the infrastructure operation, such as deleting the cluster or worker node. If you still run into the permissions or credentials error, review these additional troubleshooting pages.
- If the worker node is not removed, review the State and Status fields and the common issues with worker nodes to continue debugging.
- If you manually set credentials and still can't see the cluster's worker nodes in your infrastructure account, you might check whether the cluster is orphaned.
-
If the issue persists, gather the following information to submit to IBM Cloud support. Save the outputs from each command. Make sure that you have the correct resource group targeted with the
ibmcloud target -g <resource_group>
command.-
API key info.
ibmcloud ks api-key info --cluster <cluster_name_or_id>
-
Account details.
ibmcloud target
-
Credential details for the expected region and resource group.
ibmcloud oc credential get --region <region>
-
Infrastructure permissions details.
ibmcloud oc infra-permissions get --region <region>
-
-
[Open an issue with IBM Cloud support](/docs/openshift?topic=openshift-get-help. Be sure to include all the information and command outputs gathered in the previous step.
Invalid API key
Classic infrastructure
When you try to complete an action that requires you to specify an API key, you get an error similar to the following.
Error: SoftLayer_Exception_User_Customer_Unauthorized: Invalid API key
Additionally, running the ibmcloud ks api-key reset
command does not resolve the issue and the output of the ibmcloud oc infra-permissions get
command does not indicate any problems with permissions.
There are multiple reasons why this error can occur.
-
The action you are attempting requires you to specify a classic infrastructure API key rather than a Kubernetes API key. If a classic infrastructure API key is set for the region, it takes precedence over all other credentials.
-
The classic infrastructure API key you specified does not exist. It might have been deleted, or the API key owner might have left the organization.
If you know the classic infrastructure API key, specify it. If you do not know the infrastructure API key or you think it might have been deleted, follow these steps.
-
Reset the classic infrastructure API key.
ibmcloud oc credential set classic --infrastructure-api-key API_KEY --infrastructure-username USERNAME --region REGION [-q]
-
Run the command to update the credential on the cluster.
ibmcloud oc api-key reset --region REGION [-q]
-
Try again to complete the action.