AWS EMR Presto集群意外终止错误通常是由于Spot实例被终止导致的。为了解决这个问题,你可以采取以下步骤:
[
{
"InstanceCount": 1,
"InstanceRole": "MASTER",
"InstanceType": "m5.xlarge",
"Market": "ON_DEMAND",
"Name": "Master"
},
{
"InstanceCount": 2,
"InstanceRole": "CORE",
"InstanceType": "m5.xlarge",
"Market": "ON_DEMAND",
"Name": "Core"
}
]
[
{
"Classification": "presto-config",
"Properties": {
"discovery.uri": "http://:8889",
"query.max-memory": "50GB",
"query.max-memory-per-node": "10GB",
"query.max-total-memory-per-node": "12GB",
"query.max-total-memory": "100GB",
"query.max-history": "1GB",
"query.max-stage-count": "200",
"query.initial-hash-partitions": "32",
"query.initial-splits-per-node": "8",
"spill-enabled": "true",
"spill-path": "/mnt/var/lib/presto/data"
}
},
{
"Classification": "presto",
"Properties": {
"spot-instance-interruption-behavior": "terminate-cluster"
}
}
]
这将确保当Spot实例被终止时,EMR集群会自动终止,以避免资源浪费。
[
{
"Classification": "emrfs-site",
"Properties": {
"fs.s3.consistent.retryPeriodSeconds": "10",
"fs.s3.consistent": "true",
"fs.s3.consistent.retryCount": "5",
"fs.s3.consistent.metadata.tableName": "EmrFSMetadata"
}
}
]
通过以上步骤,你可以有效地解决AWS EMR Presto集群意外终止错误,并确保集群的可靠性和稳定性。