AWS waiting for terminate lifecycle action made easy with Bobcares by your side. 

At Bobcares, we offer solutions for every query, big and small, as a part of our Server Management Service.

Let’s take a look at how our Support Team recently helped a customer who wanted their AWS to wait before terminating lifecycle action.

How to Increase AWS waiting for terminate lifecycle action

If an Amazon EC2 instance is marked unhealthy, it gets moved to the “Auto Scaling Terminating” state automatically. Our Support Engineers are here with a way to delay Auto Scaling termination.

In other words, it allows us to troubleshoot unhealthy Amazon EC2 instances before termination.

The solution involves adding a lifecycle hook to the AWS Auto Scaling group. This moves instances from Terminating state to Terminating:Wait state. Additionally, it allows us to access instances prior to termination.

Furthermore, an instance remains in the Terminating:Wait state for 3600 seconds by default. We can increase this time value via the heartbeat-timeout parameter in the put-lifecycle-hook API call.

Moreover, the maximum time permitted for this parameter is 100 times the heartbeat timeout or 48 hours, whichever is smaller.

How to increase AWS waiting for terminate lifecycle action

The solution to this specific issue involves configuring a lifecycle hook via the AWS CLI. After that, we have to create Amazon Simple Notification Service topics as well as IAM permissions.

We can also perform these actions via the AWS Management Console.

• Create an Amazon SNS topic
• Configure IAM permissions
• Configure the lifecycle hook
• Test the lifecycle hook

How to create an Amazon SNS topic

1. First, we will create a topic where AWS Auto Scaling is able to send lifecycle notifications. For instance, this example illustrates calling create-topic command in order to create ASNotifications topic:

   $ aws sns create-topic --name ASNotifications

   As a result, an ARN similar to the one below is returned:

   "TopicArn": "arn:aws:sns:us-west-2:123456789022:ASNotifications"

2. Next, we will create a subscription to the topic. This allows us to receive the LifeCycleActionToken that comes in handy in extending the heartbeat timeout of the pending state or completing the lifecycle action.

   For instance, this example makes use of the subscribe command to create a subscription that relies on SMPT with an endpoint email address:

   $ aws sns subscribe --topic-arn arn:aws:sns:us-west-2:123456789012:ASNotifications --protocol email --notification-endpoint user@amazon.com

How to configure IAM permissions

1. First, open a text editor to create a file as seen below:

   $ sudo vi assume-role.txt

2. Then, paste the following in the file and save the file:

   {
     "Version": "2012-10-17",
     "Statement": [{
         "Sid": "",
         "Effect": "Allow",
         "Principal": {
           "Service": "autoscaling.amazonaws.com"
         },
         "Action": "sts:AssumeRole"
       }
     ]
   }

3. After that, we will create AS-Lifecycle-Hook-Role, an IAM role, from the policy saved in assume-role.txt:

   $ aws iam create-role --role-name AS-Lifecycle-Hook-Role --assume-role-policy-document file://assume-role.txt

   As a result, we get the ARN for the role as output. Our Support Engineers would like to point that you need to save the ARN of the IAM role as well as the SNS topic.
4. Next, we will add permission to the role. Thereby, allowing AWS Auto Scaling to send notifications when the lifecycle hook event takes place.

   For instance,

   $ aws iam attach-role-policy --role-name AS-Lifecycle-Hook-Role --policy-arn arn:aws:iam::aws:policy/service-role/AutoScalingNotificationAccessRole

   This grants the following permissions:

   {
     "Version": "2012-10-17",
     "Statement": [{
         "Effect": "Allow",
         "Resource": "*",
         "Action": [
           "sqs:SendMessage",
           "sqs:GetQueueUrl",
           "sns:Publish"
         ]
       }
     ]
   }

   If you are looking for a way to restrict AWS Auto Scaling’s access to certain SNS topics or queues, our Support Techs recommend the following sample policy:

   {
     "Version": "2012-10-17",
     "Statement": [{
         "Effect": "Allow",
         "Resource": "arn:aws:sns:us-west-2:123456789012:ASNotifications",
          "Action": [
            "sqs:SendMessage",
            "sqs:GetQueueUrl",
            "sns:Publish"
          ]
        }
      ]
   }

   How to configure the lifecycle hook?

   Next, we will use the put-lifecycle-hook command in order to configure the lifecycle hook as seen below:

   aws autoscaling put-lifecycle-hook --lifecycle-hook-name AStroublshoot --auto-scaling-group-name MyASGroup
           --lifecycle-transition autoscaling:EC2_INSTANCE_TERMINATING
           --notification-target-arn arn:aws:sns:us-west-2:123456789012:ASNotifications
           --role-arn arn:aws:iam::123456789012:role/AS-Lifecycle-Hook-Role

   Remember to replace the AWS Auto Scaling group name, SNS target ARN as well as the IAM role ARN before executing the command.

   Additionally this command:

   • Identifies the AWS Auto Scaling group associated with lifestyle hook.
   • Names the lifecycle hook.
   • Specifies the IAM role’s ARN.
   • Specifies the SNS topic’s ARN.

   How to test the lifecycle hook

   We can test the lifecycle hook by selecting an instance and using the terminate-instance-in-auto-scaling group to terminate it. This will force AWS Auto Scaling to terminate the instance.

   When the instance moves to Terminating:Wait state we can keep the instance in the same state with record-lifecycle-action-heartbeat. We can also opt to allow the termination to proceed by using complete-lifecycle-action.

   aws autoscaling complete-lifecycle-action --lifecycle-hook-name my-lifecycle-hook
           --auto-scaling-group-name MyASGroup --lifecycle-action-result CONTINUE
           --instance-id i-0e7380909ffaab747

   [Need further assistance? We are available 24/7.]

   Conclusion

   To sum up, the skilled Support Engineers at Bobcares demonstrated how to delay Auto Scaling termination of unhealthy Amazon EC2 instances.

   PREVENT YOUR SERVER FROM CRASHING!

   Never again lose customers to poor server speed! Let us help you.

   Our server experts will monitor & maintain your server 24/7 so that it remains lightning fast and secure.

   GET STARTED