Wondering how to enable special parameters in AWS Glue job? We can help you.

As part of our Server Management Services, we assist our customers with several AWS queries.

Today, let us see how our Support techs proceed to enable it.

How to enable special parameters in AWS Glue job?

Sometimes, when we try to enable it, such as –enable-metrics, for job in AWS we get a template validation or “null values” error from AWS CloudFormation.

To enable special parameters for your job in AWS Glue, you must supply a key-value pair for the DefaultArguments property of the AWS::Glue::Job resource in AWS CloudFormation.

If you supply a key only in your job definition, then AWS CloudFormation returns a validation error.

Today, let us see the simple steps followed by our Support Techs to resolve it.

1. Firstly, in your AWS CloudFormation template, for the DefaultArguments property of your job definition, set the value of your special parameter to an empty string.

In the following example JSON and YAML templates, the value of –enable-metrics is set to an empty string.

JSON:

"MyJob": {
"Type": "AWS::Glue::Job",
"Properties": {
"Command": {
"Name": "glueetl",
"ScriptLocation": "s3://my-test//test-job1"
},
"DefaultArguments": {
"--job-bookmark-option": "job-bookmark-enable",
"--enable-metrics": ""
},
"ExecutionProperty": {
"MaxConcurrentRuns": 2
},
"MaxRetries": 0,
"Name": "cf-job3",
"Role": {
"Ref": "MyJobRole"
}
}
}

YAML:

MyJob:
Type: 'AWS::Glue::Job'
Properties:
Command:
Name: glueetl
ScriptLocation: 's3://my-test//test-job1'
DefaultArguments:
'--job-bookmark-option': job-bookmark-enable
'--enable-metrics': ''
ExecutionProperty:
MaxConcurrentRuns:
MaxRetries: 0
Name: cf-job3
Role: !Ref MyJobRole

The empty string validates the template and launches the resource configured with the special parameter.

2. Then, to activate your special parameter, run your job.

[Stuck in between? We’d be glad to assist you]

Conclusion

In short, today we saw steps followed by our Support Techs to enable special parameters.