com.amazonaws.services.s3.model.amazons3exception: access denied error resolved with ease. 

At Bobcares, we offer solutions for every query, big and small, as a part of our Server Management Service.

Let’s take a look at how our Support Team recently helped a customer when their Amazon EMR application failed, resulting in a com.amazonaws.services.s3.model.amazons3exception: access denied error.

com.amazonaws.services.s3.model.amazons3exception: access denied

If you have been coming across the HTTP 403 “Access Denied” AmazonS3Exception, our Support Engineers have a solution for you.

This error draws attention to the application that attempted the failed Amazon S3 operation due to one of the following reasons:

• Role or credentials specified in the application code.
• Amazon S3 VPC endpoint policy.
• Policy attached to the Amazon EC2 instance profile role.
• Amazon S3 source as well as destination bucket policies.

How to resolve com.amazonaws.services.s3.model.amazons3exception: access denied error

First, we have to run the command seen below on the EMR cluster’s master node after replacing s3://doc-example-bucket/xyz/ with the Amazon S3 path:

aws s3 ls s3://doc-example-bucket/xyz/

In case the command is successful, it indicates that the role or credentials mentioned in the application code are the reason behind the error. In other words, the role or credentials needs to have access to the Amazon S3 path.

However, if the command fails, our Support Techs recommend verifying whether we are using the latest AWS CLI version.

How to check the policy for Amazon EC2 instance profile role

Application, by default, inherits Amazon S3 access from the IAM role for the EC2 instance profile. Furthermore, the IAM policies attached to this role have to allow the required Amazon S3 operations on the source as well as destination buckets.

In case we are using EMRFS role mapping, the application will inherit Amazon S3 permissions from the IAM role for the specific user who submitted the application.

Moreover, this particular IAM user needs to have an IAM policy that permits required Amazon S3 operations on the source as well as destination buckets.

How to check the Amazon S3 VPC endpoint policy

Next, we will check whether the EMR cluster’s subnet route table has a route to an S3 VPC endpoint. After that, we will confirm whether the endpoint policy permits Amazon S3 operations.

Our Support Techs recommend using AWS CLI or Amazon VPC console to verify and modify the endpoint policy.

This may resolve the com.amazonaws.services.s3.model.amazons3exception: access denied error. If not, try the other strategies suggested below by our Support Team.

For AWS CLI:

aws ec2 describe-vpc-endpoints --vpc-endpoint-ids "vpce-9f28e3f6"

Here, we will replace vpce-9f28e3f6 with the VPC ID. We can also run a command to upload a modified endpoint policy as seen below:

aws ec2 modify-vpc-endpoint --vpc-endpoint-id "vpce-9f28e3f6" --policy-document file://policy.json

For Amazon VPC console:

1. First, we have to open the Amazon VPC console.
2. Then, we will select Endpoints in the navigation pane.
3. After that, we will choose the Amazon S3 endpoint and select the Policy tab in order to review the endpoint policy.
4. Finally, we have to add the required Amazon S3 actions and select Edit Policy.

How to check the S3 bucket policies

According to our Support Team, bucket policies specify actions that are denied or allowed for which principals.

Furthermore, the bucket policies have to allow EC2 instance profile role as well as the mapped IAM role to go about Amazon S3 operations.

For AWS CLI:

aws s3api get-bucket-policy --bucket doc-example-bucket

Here, we will replace doc-example-bucket with the source or destination bucket name. We can also run a command to upload a modified bucket policy as seen below:

aws s3api put-bucket-policy --bucket doc-example-bucket --policy file://policy.json

For Amazon S3 console:

1. First, open the Amazon S3 console.
2. Then, select the bucket.
3. After that, we have to select the Permissions tab.
4. Finally, we have to select Bucket Policy in order to review as well as modify the bucket policy.

How to access S3 buckets in another account

In case our application accesses an S3 bucket that belongs to a different AWS account, the account owner has to permit our IAM role on the bucket policy.

For instance, the example below demonstrated how the bucket policy offers all IAM roles as well as users in the emr-account full access to s3://doc-example-bucket/myfolder/:

{
    "Id": "MyCustomPolicy",
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowRootAndHomeListingOfCompanyBucket",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::emr-account:root"
                ]
            },
            "Action": [
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::doc-example-bucket"
            ],
            "Condition": {
                "StringEquals": {
                    "s3:prefix": [
                        "",
                        "myfolder/"
                    ],
                    "s3:delimiter": [
                        "/"
                    ]
                }
            }
        },
        {
            "Sid": "AllowListingOfUserFolder",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::emr-account:root"
                ]
            },
            "Action": [
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::doc-example-bucket"
            ],
            "Condition": {
                "StringLike": {
                    "s3:prefix": [
                        "myfolder/*"
                    ]
                }
            }
        },
        {
            "Sid": "AllowAllS3ActionsInUserFolder",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::emr-account:root"
                ]
            },
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::doc-example-bucket/myfolder/*",
                "arn:aws:s3:::doc-example-bucket/myfolder*"
            ]
        }
    ]
}

[Need further assistance? We are available 24/7.]

Conclusion

To sum up, the skilled Support Engineers at Bobcares demonstrated how to deal with com.amazonaws.services.s3.model.amazons3exception: access denied error.