CODEWITHSUNDEEP
pythonpostgresqlairflow
David Masip
David Masip

Reputation: 2521

Most python way to write from GCS to Postgresql in airflow

I am building an airflow dag that takes csv files from GCS and inserts them to a postgresql table in cloud SQL. I have several options:

Which is the most pythonic way to do so?

Upvotes: 1

Views: 843

Answers (1)

balderman
balderman

Reputation: 23825

You sould go with COPY.

See https://www.postgresql.org/docs/current/sql-copy.html

COPY moves data between PostgreSQL tables and standard file-system files. COPY TO copies the contents of a table to a file, while COPY FROM copies data from a file to a table (appending the data to whatever is in the table already).

Upvotes: 1

Related Questions