CODEWITHSUNDEEP
pythonsqlpandasteradataread-sql
Felix313
Felix313

Reputation: 1

Optimizing pandas.read_sql for teradata

How can one read a teradata sql into a tempfile? The goal is to improve performance when ingesting data from an sql query into a pandas df.

On https://towardsdatascience.com/optimizing-pandas-read-sql-for-postgres-f31cd7f707ab Tristan Crockett shows how this is done for postgres.

def read_sql_tmpfile(query, db_engine):
    with tempfile.TemporaryFile() as tmpfile:
        copy_sql = "COPY ({query}) TO STDOUT WITH CSV {head}".format(
           query=query, head="HEADER"
        )
        conn = db_engine.raw_connection()
        cur = conn.cursor()
        cur.copy_expert(copy_sql, tmpfile)
        tmpfile.seek(0)
        df = pandas.read_csv(tmpfile)
        return df

I couldn't figure out how to rewrite this code to make it work with a teradata server.

Upvotes: 0

Views: 643

Answers (0)

Related Questions