CODEWITHSUNDEEP
javajdbcamazon-redshift
iec2011007
iec2011007

Reputation: 1846

Bulk unload from redshift async

I want to fire an unload query to redshift. But is using jdbc connection for doing this is the best way to go around?

As far as i have done my POC the call is blocking. Now many things can go wrong here, maybe the query dumps too much result and the jdbc connection may timeout.

So is there anyway in which we can submit a query to red shift asynchronously and then poll the api to see the result of the query.

PS: Using jdbc is not the only requirement. But the thing is the redshift should be connected by JAVA code

EDIT : Then if anyone has to fire a long running unload query what is the best way to go around ?

Upvotes: 4

Views: 1270

Answers (1)

Joe Harris
Joe Harris

Reputation: 14045

There is not currently a "submit and poll for results" feature in Redshift.

I'd recommend using a workflow server. Something that can run the jobs and track success and failure.

Look at AWS Data Pipeline, Apache Airflow, or Azkaban

Upvotes: 1

Related Questions