Are you stuck with webidentityerr error using AWS Load Balancer Controller? We can help you.

Here, at Bobcares, we assist our customers with several AWS queries as part of our AWS Support Services.

Today, let us see how our Support techs assist with this query.

How to resolve webidentityerr error using AWS Load Balancer Controller?

Typically, error looks as shown below:

"failed to find existing LoadBalancer due to WebIdentityErr: failed to retrieve credentials\ncaused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity\n\tstatus code: 403"

With AWS Load Balancer Controller, worker nodes perform the tasks.

These worker nodes must grant access to the AWS Application Load Balancer or AWS Network Load Balancer resources using IAM permissions.

The IAM permissions can either be set up using IAM roles for the service account, or they can attach directly to the worker node’s IAM roles.

Today, let us see how our Support Techs resolve it.

Incorrect service account configurations

To check whether your service account configurations are properly configure, perform the following steps:

1.Firstly, verify the service account (SA) name define in your deployment:

kubectl describe deploy <aws-load-balancer-controller> -n kube-system | grep -i "Service Account"

2.Then, describe the service account:

kubectl describe sa <aws-load-balancer-controller> -n kube-system

3. Next, verify the SA annotation for the IAM role:

Annotations: eks.amazonaws.com/role-arn: arn:aws:iam::xxxxxxxxxx:role/<AmazonEKSLoadBalancerControllerRole>

4. If this annotation is missing or incorrect, then update the annotation.

Make sure that you properly associate the IAM role to a service account, like this:

kubectl annotate serviceaccount -n <SERVICE_ACCOUNT_NAMESPACE> <SERVICE_ACCOUNT_NAME> \ eks.amazonaws.com/role-arn=arn:aws:iam::<ACCOUNT_ID>:role/<IAM_ROLE_NAME>

5.Then, delete and recreate any existing Amazon EKS pods that are associate with the service account.

Make sure to apply the credential environment variables:

kubectl delete pods <aws-loadbalancer-controller-pod> -n <kube-system>

Incorrect trust relationship between the AWS Identity Access Management (IAM) role used and service account

The following examples show some common mistakes that can occur when establishing the trust relationship between your IAM role and service account.

Example 1: IAM role or trust relationship isn’t properly defined for the “sts:AssumeRoleWithWebIdentity” action

Check to make sure that the trust relationship is properly define for the “sts:AssumeRoleWithWebIdentity” action.

Here’s an example of a trust relationship that isn’t properly defined:

{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "xxxxx.amazonaws.com"
},
"Action": "sts:AssumeRole"
}

To correct the trust relationship, make sure to define the trust relationship for the “sts:AssumeRoleWithWebIdentity” action:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::<AWS-ACCOUNT>:oidc-provider/oidc.eks.<REGION>.amazonaws.com/id/<Your-EKS-cluster-OIDC-Provider-ID>"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.<REGION>.amazonaws.com/id/<Your-EKS-cluster-OIDC-Provider-ID>": "system:serviceaccount:kube-system:<aws-load-balancer-controller-serviceaccount>"
}
}
}
]
}

Note: Make sure to replace all variables with your own values.

Example 2: Incorrect OIDC provider ID when creating an Amazon EKS cluster

Make sure to properly create and verify an OIDC provider for your Amazon EKS cluster.

Verify that the OIDC provider ID and the associate AWS Region are correctly list. Otherwise, you receive a WebIdentityErr error.

Example 3: Incorrect service account (SA) name and its namespace when configuring the AWS Load Balancer Controller deployment

Make sure to enter the correct SA name and its namespace when you update your AWS Load Balancer Controller deployment.

Example 4: Missing “sts.amazonaws.com” steps from the trust relationship

If the service role associate with your EKS pod is unable to perform the STS operation on the “AssumeRoleWithWebIdentity” action, then update the trust relationship.

The trust relationship must include “sts.amazonaws.com” to perform an STS operation.

For example:

"Condition": {
"StringEquals": {
"oidc.eks.<REGION>.amazonaws.com/id/<Your-EKS-cluster-OIDC-Provider-ID>:sub": "system:serviceaccount:kube-system:<aws-load-balancer-controller-serviceaccount>",
"oidc.eks.<REGION>.amazonaws.com/id/<Your-EKS-cluster-OIDC-Provider-ID>:aud": "sts.amazonaws.com"
}
}

[Need help with the process? We’d be happy to assist]

Conclusion

In short, we saw how our Support Techs resolve webidentityerr error.