wesupport

Need help?

Our experts have had an average response time of 13.14 minutes in February 2024 to fix urgent issues.

We will keep your servers stable, secure, and fast at all times for one fixed price.

Enable special parameters in AWS Glue job – How to do it

by | Oct 25, 2021

Wondering how to enable special parameters in AWS Glue job? We can help you.

As part of our Server Management Services, we assist our customers with several AWS queries.

Today, let us see how our Support techs proceed to enable it.

How to enable special parameters in AWS Glue job?

Sometimes, when we try to enable it, such as –enable-metrics, for job in AWS we get a template validation or “null values” error from AWS CloudFormation.

To enable special parameters for your job in AWS Glue, you must supply a key-value pair for the DefaultArguments property of the AWS::Glue::Job resource in AWS CloudFormation.

If you supply a key only in your job definition, then AWS CloudFormation returns a validation error.

Today, let us see the simple steps followed by our Support Techs to resolve it.

1. Firstly, in your AWS CloudFormation template, for the DefaultArguments property of your job definition, set the value of your special parameter to an empty string.

In the following example JSON and YAML templates, the value of –enable-metrics is set to an empty string.

JSON:

"MyJob": {
"Type": "AWS::Glue::Job",
"Properties": {
"Command": {
"Name": "glueetl",
"ScriptLocation": "s3://my-test//test-job1"
},
"DefaultArguments": {
"--job-bookmark-option": "job-bookmark-enable",
"--enable-metrics": ""
},
"ExecutionProperty": {
"MaxConcurrentRuns": 2
},
"MaxRetries": 0,
"Name": "cf-job3",
"Role": {
"Ref": "MyJobRole"
}
}
}

YAML:

MyJob:
Type: 'AWS::Glue::Job'
Properties:
Command:
Name: glueetl
ScriptLocation: 's3://my-test//test-job1'
DefaultArguments:
'--job-bookmark-option': job-bookmark-enable
'--enable-metrics': ''
ExecutionProperty:
MaxConcurrentRuns:
MaxRetries: 0
Name: cf-job3
Role: !Ref MyJobRole

The empty string validates the template and launches the resource configured with the special parameter.

2. Then, to activate your special parameter, run your job.

[Stuck in between? We’d be glad to assist you]

Conclusion

In short, today we saw steps followed by our Support Techs to enable special parameters.

PREVENT YOUR SERVER FROM CRASHING!

Never again lose customers to poor server speed! Let us help you.

Our server experts will monitor & maintain your server 24/7 so that it remains lightning fast and secure.

GET STARTED

2 Comments

  1. Anna

    Hi, I don’t seem to get the job name , job run Id, type, statistics from the get-metric-data or get-metric-statistics using lambda function boto3 for glue job. Can you please advise.
    Also, I have developed custom dashboard using lambda function but I don’t see any data points on my dashboard.

    Reply
    • Hiba Razak

      Hi,
      Please contact our support team via live chat(click on the icon at right-bottom)

      Reply

Submit a Comment

Your email address will not be published. Required fields are marked *

Categories

Tags