AWS OpsWorks auto healing allows stacks to automatically replace failed instances in the layer. As part of our AWS Support Service, Bobcares offers solutions to every query that comes our way.

AWS OpsWorks Auto Healing

AWS OpsWorks Stacks agents are installed on each instance and communicate frequently with the service. That communication is used by AWS OpsWorks Stacks to track the health of each instance. AWS OpsWorks Stacks determines that the instance has failed if an agent has not communicated with the service for more than five minutes.

At the layer level, auto healing is configured. By editing layer settings, we can modify the auto healing setting. A single instance can belong to several layers. AWS OpsWorks Stacks does not repair the instance if any of those layers has auto healing disabled.

AWS OpsWorks Stacks automatically replaces the layer’s failed instances as follows if auto healing is enabled for a layer, which is the default setting:

Instance store-backed instance

1. Firstly, stops the Amazon EC2 instance and make sure it has actually ended.
2. Then removes the content from the root volume.
3. Then a new Amazon EC2 instance is created with the same hostname, settings, and layer membership.
4. Reattaches all Amazon EBS volumes, including those that were initially attached to the old instance but later detached.
5. Then make a new public and private IP Address assignment.
6. Associates the new instance with the same IP address as the old instance, if the latter was an elastic IP address.

Amazon EBS-backed instance

1. Firstly stops and confirms that the Amazon EC2 instance has stopped.
2. Then start an EC2 instance.

AWS OpsWorks Stacks

It causes a Configure life cycle event to be triggered on all of the stack’s instances after the auto-healed instance has been brought back online. The instance’s public and private IP addresses are included in the associated stack configuration and deployment attributes. The new IP addresses can be retrieved by Custom Configure recipes from the node object.

AWS OpsWorks Stacks creates a new volume and attaches it to each instance when the instance is started if we specify an Amazon EBS volume for a layer’s instances. Use the Resources page to later detach the volume from an instance if necessary.

One of a layer’s instances is auto-healed by AWS OpsWorks Stacks, and volumes are handled as follows:

• If the volume was already attached to the instance when it failed, AWS OpsWorks Stacks saves the volume and its data and attaches it to the new instance.
• AWS OpsWorks Stacks creates a new, empty volume with the configuration specified by the layer and attaches it to the new instance if the volume was not attached to the instance when the instance failed.

All layers have auto healing turned on by default, but we can turn it off by editing the layer’s general settings.

Steps to follow if auto healing is turned on

• To stop instances, only use the AWS OpsWorks Stacks console, CLI, or API.

  AWS OpsWorks Stacks considers an instance to have failed if we stop it in any other way, such as by using the Amazon EC2 console and automatically heals the instance.
• Any data that we don’t want to lose if the instance is auto-healed should be stored on Amazon EBS volumes.

  Any data not stored on an Amazon EBS volume is destroyed when the old Amazon EC2 instance is stopped by auto healing. Any stored data is kept safe thanks to the reattachment of Amazon EBS volumes to the new instance.

How to Configure Auto Healing Notifications for AWS OpsWorks Stacks in Amazon CloudWatch Events

In this article, we go over how to configure Amazon CloudWatch Events so that we are alerted whenever an Amazon EC2 instance’s stop and start events are triggered by AWS OpsWorks Stacks auto healing.

CloudWatch Events are now supported by AWS OpsWorks Stacks, according to recent news. We can now send OpsWorks Stack state changes, like instances being stopped or deployments failing, to CloudWatch Events.

Incorporating resource provisioning, Amazon EBS volume setup, configuration management, application deployment, monitoring, and access control, AWS OpsWorks Stacks offers us a seamless management experience that covers the entire application lifecycle.

We can respond selectively to events in the cloud as well as in our applications thanks to Amazon CloudWatch Events. More specifically, we are able to build CloudWatch Events rules that match event patterns and respond to those patterns. Both custom events and events provided by AWS can be processed using CloudWatch Events.

Instance auto-healed scenario

In our example, an organization has multiple instances in various OpsWorks Stacks, and all layers can have auto healing enabled. The way auto healing works is by telling instances it thinks have failed to stop and start. When an OpsWorks Stacks instance is stopped and started by auto healing and the company is unsure of what or who did it, a problem results.

Solution

Amazon CloudWatch Events now offers AWS OpsWorks Stacks event sources. These event sources can be used to start events or send out notifications, including events started by auto healing.

Before, in order to initiate events in response to OpsWorks Stacks events, we had to orchestrate calls to numerous APIs. Now, when an OpsWorks Stacks instance, deployment, or command state changes, we can use CloudWatch Events to start an event. Additionally, we can use CloudWatch Events to show that a service error for OpsWorks Stacks has occurred. Patterns of events are represented as JSON objects in CloudWatch Events.

Set up the solution

We must first create a target before we can create CloudWatch rules. We’ll use an Amazon SNS topic in this solution, but other options include Amazon SQS and AWS Lambda functions.

Make an OpsWorksAutoHealingNotifier SNS topic. We must subscribe to an endpoint to the topic in order to receive messages posted to it. Open the confirmation email from AWS Notifications, click the link to complete the subscription, and then close the message. We can now start building our CloudWatch Rule.

Utilizing the AWS Management Console, write the rule as follows:

1. Firstly, go to the Amazon CloudWatch page in the AWS Management Console.
2. Then select Events from the left navigation panel.
3. After selecting Create Rule, select Show advanced options.
4. Then select Edit the pattern’s JSON representation.
5. Then copy and paste the OpsWorksAutoHealingPattern.json file’s contents into the text box.
6. Under the Targets section, select Add target. OpsWorksAutoHealingNotifier can be chosen from the list of SNS topics.
7. Then select Configure details.
8. OpsWorks-AutoHealing should be the name of the rule.

[Looking for a solution to another query? We are just a click away.]

Conclusion

In conclusion, Auto healing is configured at the layer level. By editing layer settings, as seen in the following screenshot, we can modify the auto healing setting. A single instance can belong to several layers. AWS OpsWorks Stacks does not repair the instance if any of those layers has auto healing disabled.