将Athena数据集的查询异步化,在ECS上使用Apache Superset。
以下是操作步骤和代码示例:
在ECS实例上安装并配置Apache Superset
配置Athena数据源并连接
使用Flask-Executor库将查询异步提交,避免等待时间过长
代码示例:
DATABASES = {
'my_athena_engine':{
'engine':'presto',
'name':'athena',
'metadata':{'schemas_allowed_for': ['athena',]},
'presto_catalog_description':{'host':'aws-region.presto.amazonaws.com','port':443,'protocol':'https'},
'presto_catalog':{'host':'aws-region.presto.amazonaws.com','port':443,'protocol':'https'},
'user_name':'
@app.route('/', methods=['GET']) def index(): executor.submit(run_query) return 'Query submitted'
def run_query(): query = 'SELECT * FROM my_table' results = superset.db.engine.execute(query).fetchall() writeFile(results)
def writeFile(results): with open('query_result.txt', 'w') as file: for row in results: file.write(str(row) + '\n')
if name == 'main': executor = Executor(app) app.run(host='0.0.0.0', port=5000)
上一篇:ApacheSuperset连接MSSQLServer失败
下一篇:ApacheSupersetPostgreSQL'functiondate_trunc(unknown,bigint)doesnotexist'