CODEWITHSUNDEEP
amazon-emrprestotrinostarburst
yahoo
yahoo

Reputation: 331

Prestosql/trino on EMR scaling down kills the queries

I have a presto cluster on EMR. We also have auto scaling policy in place based on AWS Clouldwatch metrics. We have a issue when cluster scale-down the nodes, it doesn't wait for queries running on those nodes to complete. It just kills all queries and scale down. There is a property which graciously shut down the nodes, can be used in cluster configuration, but this has its limitations. It doesn't work with trino or prestoSql.This only works for Prestodb.

https://docs.aws.amazon.com/emr/latest/ReleaseGuide/presto-graceful-autoscale.html

I have also explored the cooldown period in autoscaling policies but none of them seems to be working for our cluster. I am wondering is this a known issue for prestosql/trino

Upvotes: 1

Views: 997

Answers (1)

Hongbo Miao
Hongbo Miao

Reputation: 49944

AWS created a GitHub repo Amazon EMR - Trino Autoscale in March 2023 which I think will help.

In the readme, it mentions

This project provides a custom auto-autoscaling for Amazon EMR on EC2 clusters running with Trino. The package support out of the box Instance Groups and Instance Fleets clusters with On Demand / SPOT instances.

...

In order to use this utility, it is recommended to enable Trino fault-tolerant execution to avoid query failures during resize operations.

Upvotes: 0

Related Questions