Reducing AWS Storage Costs Through Automated EBS Snapshot Lifecycle Management
The client used Amazon EC2 with Amazon EBS snapshots as the primary backup mechanism across Test, Pre-production, and Production environments. Although automated backup policies were in place, outdated retention periods, accumulated manual snapshots, and orphaned snapshots were increasing storage usage and AWS costs.
Bobcares reviewed the existing Amazon Data Lifecycle Manager policies, audited EBS snapshots, optimized retention periods, and identified obsolete backup data. An AWS Lambda-based cleanup process was also introduced alongside DLM to manage manual and orphaned snapshots that were outside the existing lifecycle policies.
Business Challenge
The client's EBS snapshot inventory had grown beyond current backup requirements due to extended retention periods, historical manual snapshots, and orphaned snapshots. This increased storage consumption and AWS costs while requiring manual reviews and cleanup without compromising required backup and disaster recovery coverage.
Technology Used
Blueprint
Amazon Data Lifecycle Manager continued managing policy-based snapshots, while AWS Lambda supplemented it by identifying orphaned and obsolete manual snapshots. Bobcares first optimized retention policies, audited existing snapshots, and validated deletion candidates before introducing automated cleanup across Test, Pre-production, and Production environments.
Key Outcomes
- OOM-related deployment failures were eliminated during deployment windows.
- Free disk space increased from under 10% to consistently above 20%.
- Deployments became more stable and repeatable.
- Rollbacks became rare and controlled.
- MTTR was significantly reduced through centralized logs and alarms.
- The client continued its high-frequency release schedule with less manual intervention.
Customer Profile
The Challenges
- Snapshot retention periods exceeded current business requirements.
- Historical manual snapshots had accumulated through migration, upgrades, testing, maintenance, and one-off backups.
- Orphaned snapshots remained after their associated EC2 instances or EBS volumes were decommissioned.
- Manual snapshot cleanup required time-consuming reviews and introduced the risk of removing useful recovery points.
- Existing DLM policies did not manage manually created snapshots.
- Uncontrolled snapshot growth increased AWS storage consumption and monthly costs.
Why Bobcares?
- The approach addressed both policy-managed and manually created snapshots.
- Existing DLM policies were reviewed rather than replaced unnecessarily.
- Retention periods were adjusted according to the client’s current requirements.
- Snapshot deletion was preceded by validation to avoid affecting required recovery points.
- Automation was introduced only after validation in the Test environment.
- The solution combined existing AWS lifecycle management with targeted cleanup automation.
What We Delivered
- Reviewed existing Amazon Data Lifecycle Manager policies.
- Optimized snapshot retention for Test, Pre-production, and Production environments.
- Audited EBS snapshots across the AWS environments.
- Classified snapshots as managed, manual, or orphaned.
- Documented snapshot IDs, creation dates, types, associated workloads, and recommended actions.
- Identified obsolete manual snapshots for cleanup.
- Validated deletion candidates with the client.
- Designed an AWS Lambda automation process for snapshot cleanup.
- Automated identification of orphaned and obsolete manual snapshots.
- Rolled out the automation from Test to Pre-production and Production.
- Verified that scheduled backups continued during implementation.
The Results
Snapshot Retention
- Before: Some snapshots were retained longer than required by current business needs.
- After: Retention was optimized to the approved requirements, with Test set to 7 days and Pre-production and Production set to 14 days.
Snapshot Management
- Before: Snapshot management was partially automated, with manual and policy-managed snapshots requiring separate handling.
- After: DLM and AWS Lambda provided a broader automated lifecycle management approach covering policy-managed, obsolete manual, and orphaned snapshots.
Manual Snapshots
- Before: Historical manual snapshots had accumulated over time without a consistent retention strategy.
- After: Obsolete manual snapshots were identified, validated, and cleaned up.
Orphaned Snapshots
- Before: Snapshots associated with decommissioned workloads remained in the AWS accounts and consumed storage.
- After: The Lambda-based process automatically identified and removed snapshots that were no longer linked to live workloads.
AWS EBS Storage Cost
- Before: Storage costs increased as unnecessary snapshot data accumulated.
- After: Optimized retention and automated cleanup reduced unnecessary EBS storage consumption and associated costs.
Operational Effort
- Before: Snapshot reviews and cleanup required manual effort.
- After: Regular identification and cleanup of unwanted snapshots were automated.
The Business Impact
- Reduced unnecessary AWS EBS storage consumption.
- Improved control over backup retention across environments.
- Reduced the operational effort required for snapshot reviews and cleanup.
- Limited the accumulation of obsolete and orphaned backup data.
- Maintained required backup and disaster recovery coverage during optimization.
- Established a more scalable approach to managing snapshot growth.
Ongoing Collaboration
- Review of snapshot retention requirements.
- Monitoring of snapshot lifecycle management.
- Validation of backup and recovery requirements.
- Assessment of obsolete and orphaned snapshots.
- Review of storage growth and optimization opportunities.
- Scaling assessments for the automated cleanup process.
