CODEWITHSUNDEEP
pythonpostgresqlpython-3.xpsycopg2data-pipeline
skybunk
skybunk

Reputation: 863

Python psycopg2: Copy result of query to another table

I am having some problem with psycopg2 in python
I have two disparate connections with corresponding cursors:

1. Source connection - source_cursor
2. Destination connection - dest_cursor

Lets say there is a select query that I want to execute on source_cursor:

SELECT id, date
FROM source_table
WHERE id > 200

The result of this query should be inserted into a table at the dest_cursor

I have looked into copy_from and copy_to and examples such as this. They make use of external files to copy the data, instead I need to copy it directly without using files. Direct connection between the source and destination databases is not possible, otherwise I would have done it at the database side.

I think it is possible using fetchall() and an INSERT.. or some method which directly enables us to copy data.

Any help with links to similar examples greatly appreciated

Upvotes: 2

Views: 3232

Answers (1)

skybunk
skybunk

Reputation: 863

Realized its a repeat question. Found a way to do it using the method given in Tuning Postgresql performance and memory use in a python workflow

Related:

  1. psycopg2: insert multiple rows with one query
  2. Psycopg2, Postgresql, Python: Fastest way to bulk-insert
  3. psycopg2 copy
  4. How to do virtual file processing? - we can use this to use psycopg2's copy

Upvotes: 1

Related Questions