Wondering how to automate the AWS DMS error log deletion process? We can help you with this!

Here at Bobcares, we handle similar requests from our customers as a part of our AWS Support Services.

Today let’s see how our Support Engineers help our customers to automate AWS DMS error log deletion.

Automate AWS DMS error log deletion process

If the storage space of the replication instance becomes full due to AWS DMS error logs, we can fix the storage space issue by deleting the error logs.

We can delete the error logs by using any of the following options:

1. Delete error logs using AWS CLI.
2. Automated log deletion using API.
3. Delete error logs manually using AWS DMS console.

Now let’s see steps followed by our Support Techs to automate the AWS DMS error log deletion process.

Delete error logs using AWS CLI

The AWS DMS error log can be deleted by altering the task settings by putting DeleteTaskLogs to true. For that we can use the following AWS CLI command:

aws dms modify-replication-task --replication-task-arn <DMS task ARN> --replication-task-settings '{"Logging": { "DeleteTaskLogs": true}}'

After adding the DeleteTaskLogs parameter to true, all logs for that particular task are deleted. Always confirm that the logs are deleted after running the command. Also, the task setting removes the parameter after the log deletion. We have to use this command to delete the logs periodically. So we can delete the logs periodically using cronjob by scheduling this command.

Automated log deletion using API

The task logs will delete in every week for all DMS replication instances. We can also create a script and automate the deletion of logs as per our wish. The following is an example of python code used to automate the error log deletion process using AWS Lambda:

Steps to set up Lambda function

1. Log in to the AWS and open the Lambda console.

2. Select Create function from Functions and enter a function name.

3. Then select python 3.8 as Runtime.

4. Select Create a new role with basic Lambda permissions for the execution role. (note the AWS Identity and IAM role name)

5. Select Create function.

6. After that open the IAM console and then open the IAM role which has already been created.

7. Then create an IAM policy using the following JSON. (Here the task name is EXAMPLE)

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"dms:ModifyReplicationTask"
],
"Resource": 'example:aws:dms:us-west-1:0987654321:task:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
}
]
}

8. After that, we need to attach the policy to the role.

9. Then open the Lambda console, and then select the Lambda function.

10. Type the following code on the Function code section and then click the Deploy option.

import boto3
import json
client = boto3.client('dms')
def lambda_handler(event, context):
# specific task EXAMPLE
taskarn = 'example:aws:dms:us-west-1:0987654321:task:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA'
# modify the DMS task
response = client.modify_replication_task(
ReplicationTaskArn=taskarn,
ReplicationTaskSettings='{"Logging": { "DeleteTaskLogs": true}}'
)

11. Select Test, and then enter Event name with the default template.

13. Then select Create.

14. Again select Test, and the code will run without errors.

15. After completing the steps, the DMS task status changes to Modifying. We can also see that the task error logs under the replication instance are deleted.

Steps to automate deletion using Lambda Scheduler

Let’s see the steps followed by our Support Techs to automate the deletion of task error logs using Lambda Scheduler.

1. Log in to the AWS and open the Lambda console.
2. Select Add trigger for Designer.
3. Then select EventBridge for Cloudwatch Events.
4. Select Create a new rule for Rule and enter the Rule name and Rule description.
5. For schedule expression, to run the code every two days, enter the rate as 2 days.
6. Finally, click Add.
7. Now, the automated deletion is set up is completed and the logs will be cleared every two days

[Need assistance with more AWS queries? We can help you]

Conclusion

To conclude, today we saw the steps followed by our Support Techs to automate the AWS DMS error log deletion process.