Bobcares

wesupport

CLIENT AREACall Us 1-800-383-5193
Bobcares

wesupport

Call Us 1-800-383-5193
Bobcares

wesupport

Call Us 1-800-383-5193

Need help?

Our experts have had an average response time of 11.7 minutes in August 2021 to fix urgent issues.

We will keep your servers stable, secure, and fast at all times for one fixed price.

All about AWS EMR managed scaling

by | Oct 1, 2021

Stuck with AWS EMR managed scaling? Read on to find out what our expert Support Engineers have to say.

Learn how to use AWS CLI to configure managed scaling from the skilled Support Team at Bobcares.

Dealing with AWS EMR managed scaling

Let’s take a look at look at how to configure managed scaling for EMR with CLI commands. You can opt to use a shorthand syntax with the JSON configuration inline included in the relevant commands or reference the file with the configuration JSON.

Enabling the managed scaling during cluster launch

This example demonstrates how to enable managed scaling during cluster launch,

aws emr create-cluster \
 --service-role EMR_DefaultRole \
 --release-label emr-5.33.0 \
 --name EMR_Managed_Scaling_Enabled_Cluster \
 --applications Name=Spark Name=Hbase \
 --ec2-attributes KeyName=keyName,InstanceProfile=EMR_EC2_DefaultRole \
 --instance-groups InstanceType=m4.xlarge,InstanceGroupType=MASTER,InstanceCount=1 InstanceType=m4.xlarge,InstanceGroupType=CORE,InstanceCount=2 \
 --region us-east-1 \
 --managed-scaling-policy ComputeLimits='{MinimumCapacityUnits=2,MaximumCapacityUnits=4,UnitType=Instances}'

Furthermore, you can specify the managed policy configuration via the –managed-scaling-policy option while using create-cluster.

Applying managed scaling policy to an existing cluster

The following example applies managed scaling to an existing cluster:

aws emr put-managed-scaling-policy  
--cluster-id j-123456  
--managed-scaling-policy ComputeLimits='{MinimumCapacityUnits=1,
MaximumCapacityUnits=10,  MaximumOnDemandCapacityUnits=10, UnitType=Instances}'

Moreover, this can be done also via the AWS EMR put-managed-scaling-policy command. For instance, this example makes a reference to managedscaleconfig.json, a JSON file.

aws emr put-managed-scaling-policy --cluster-id j-123456 --managed-scaling-policy file://./managedscaleconfig.json

Here is a look at the contents of the managedscaleconfig.json file. It defines the managed scaling policy.

{
    "ComputeLimits": {
        "UnitType": "Instances",
        "MinimumCapacityUnits": 1,
        "MaximumCapacityUnits": 10,
        "MaximumOnDemandCapacityUnits": 10
    }
}

Retrieve an EMR managed scaling policy configuration

In order to retrieve the policy configuration, use the GetManagedScalingPolicy command. For instance, it is used here to retrieve the configuration of the cluster with the cluster id j-123455.

aws emr get-managed-scaling-policy --cluster-id j-123455

This results in the following output:

{
   "ManagedScalingPolicy": { 
      "ComputeLimits": { 
         "MinimumCapacityUnits": 1,
         "MaximumOnDemandCapacityUnits": 10,
         "MaximumCapacityUnits": 10,
         "UnitType": "Instances"
      }
   }
}

Removing EMR managed scaling policy

The RemoveManagedScalingPolicy is used to remove the managed scaling policy. For instance, this command removes the policy of the cluster with cluster id j-123455:

aws emr remove-managed-scaling-policy --cluster-id j-123455

[Need help with Server Management? We are here to help.]

Conclusion

In brief, we learned how to use AWS CLI to configure AWS EMR managed scaling. With the guidance of the skilled Support Engineers at Bobcares, Server management is now a breeze.

PREVENT YOUR SERVER FROM CRASHING!

Never again lose customers to poor server speed! Let us help you.

Our server experts will monitor & maintain your server 24/7 so that it remains lightning fast and secure.

GET STARTED

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *