Wondering how to resolve S3 endpoint connection test failures with AWS DMS?

Often our AWS customers report to us that they are facing S3 endpoint connection failures when using AWS DMS. Here, at Bobcares, we handle similar requests from our customers as a part of our AWS Support Services.

Today let’s see how our Support Engineers fix this for our AWS customers.

S3 endpoint connection test failures with AWS DMS

AWS Database Migration Service, better known as AWS DMS helps us to migrate databases securely with no downtime.

Our AWS customers report to us that while trying to connect the DMS source endpoint to the AWS S3 for data migration, they receive the following ‘Test Endpoint failed’ errors.

Test Endpoint failed: Application-Status: 1020912, Application-Message: Failed to connect to database.

Test Endpoint failed: Application-Status: 1020912, Application-Message: Failed to connect to S3 endpoint. Access denied.

Usually, this might happens due to insufficient permission. If we notice any of the test failure error messages, we must make sure that the IAM role has the minimum permission to access the Amazon S3 bucket.

Note: For using Amazon S3 as a source for DMS, we must make sure that the source S3 bucket and the DMS replication instance are in the same AWS Region.

How to fix S3 endpoint connection failures with AWS DMS

Now, let’s see how our Support Techs fix this test failure issue.

To fix these errors, we need to grant the required IAM permissions to access the Amazon S3 endpoint.  Also, note that there are different permissions for Amazon S3 source and target endpoints.

To fix permission errors:

1. Firstly, we need to confirm that the IAM role has the required permissions to access the Amazon S3 endpoint.

The following shows IAM policy examples for both Amazon S3 source and target endpoints:

Amazon S3 source endpoint:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",

"Action": [
"s3:GetObject",
"s3:ListBucket"
],

"Resource": [
"arn:aws:s3:::awstestbucket",
"arn:aws:s3:::awstestbucket/*"
]
}
]
}

Amazon S3 target endpoint:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:DeleteObject",
"s3:PutObjectTagging"
],
"Resource": [
"arn:aws:s3:::awstestbucket/*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::awstestbucket"
]
}
]
}

2. Always make sure that the bucket name in the S3 endpoint matches the resource allowed in the IAM role. Here the IAM role allows the bucket awstestbucket. So the same bucket name should specify in the Amazon S3 endpoint also. otherwise, the test connection will fail.

3. Also we need to confirm that the DMS (dms.amazonaws.com) is the trusted entity associated with the IAM role.

Trust policy for AWS DMS:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "dms.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}

4. We must confirm that the IAM role used here is a part of an organization that allows access to Amazon S3 if we are using AWS organization. If the organization doesn’t allow access, then DMS can’t connect to the S3 endpoint using that particular IAM role even though the role has all the required permissions.

[Need assistance with more AWS queries? We can help you]

Conclusion

To conclude, today, we saw the steps that our Support Techs follow to fix S3 endpoint connection test failures with AWS DMS.