CODEWITHSUNDEEP
pythonamazon-web-servicesamazon-redshift
OHS Catalog Deduplication
OHS Catalog Deduplication

Reputation: 1

How to upload data to Amazon Redshift

I want to upload data to a Redshift automatically.

I know how to access redshift and extract data. but I don't know how to load data into a table.

I heard that you need a commit feature. I want to know how to commit or the name of the module (like pycopg2).

Upvotes: 0

Views: 82

Answers (1)

John Rotenstein
John Rotenstein

Reputation: 269081

The recommended method for loading data into Amazon Redshift is to place the data in an Amazon S3 bucket, then use the COPY command to load the data from S3. This method is very efficient and utilizes the parallel processing capabilities of the entire cluster.

It is recommended NOT to use an INSERT command for large quantities of data (but it is okay for a few lines).

Therefore, you automation steps are:

  • Store the data in an S3 bucket
  • Issue a COPY command to the Redshift cluster (via SQL, so you could use psycopg2)

Upvotes: 1

Related Questions