CODEWITHSUNDEEP
amazon-web-services
madhurishastry
madhurishastry

Reputation: 41

AWS GlueJob Error - Command failed with exit code 137

I am executing a AWS-Gluejob with python shell. It fails inconsistently with the error "Command failed with exit code 137" and executes perfectly fine with no changes sometimes.

What does this error signify? Are there any changes we can do in the job configuration to handle the same?

Error Screenshot

Upvotes: 4

Views: 2214

Answers (1)

Ali
Ali

Reputation: 434

Adding the worker type to the Job Properties will resolve the issue.Based on the file size please select the worker type as below:

  • Standard – When you choose this type, you also provide a value for Maximum capacity. Maximum capacity is the number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. The Standard worker type has a 50 GB disk and 2 executors.
  • G.1X – When you choose this type, you also provide a value for Number of workers. Each worker maps to 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.
  • G.2X – When you choose this type, you also provide a value for Number of workers. Each worker maps to 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs and jobs that run ML transforms.

Upvotes: 3

Related Questions