CODEWITHSUNDEEP
pythonpandassqlalchemy
horatio1701d
horatio1701d

Reputation: 9169

Handling Duplicate Columns in Pandas DataFrame constructor from SQLAlchemy Join

I know that read_csv has mangle_dup_cols but how can I do the same from a sql join in sqlalchemy after issuing:

pd.DataFrame(result.fetchall(), columns=result.keys())

which gives me an error when using df.info() because of dupe col names.

Upvotes: 2

Views: 2207

Answers (1)

van
van

Reputation: 77012

You can create your own helper function which mangles column names. The code below I copied from the io.parsers._infer_columns:

def mangle_dupe_cols(columns):
    counts = {}
    for i, col in enumerate(columns):
        cur_count = counts.get(col, 0)
        if cur_count > 0:
            columns[i] = '%s.%d' % (col, cur_count)
        counts[col] = cur_count + 1
    return columns

pd.DataFrame(result.fetchall(), columns=mangle_dupe_cols(result.keys()))

Upvotes: 5

Related Questions