CODEWITHSUNDEEP
pythonpandas
sayth
sayth

Reputation: 7048

Fixing Future Warning Concat on excel files

I am using glob to retrieve a list of csv files and combine them to append to excel.

When I run the process I receive a future warning.

FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.

What do I need to do to this section of my code (below) to future proof it?

QInboundCSV = [file for file in glob.glob("Queue Inbound\*.csv")]
QOutboundCSV = [file for file in glob.glob("Queue Outbound\*.csv")]

df_queue_inbound = pd.concat(map(pd.read_csv, QInboundCSV), ignore_index=True)
df_queue_outbound = pd.concat(map(pd.read_csv, QOutboundCSV), ignore_index=True)

Upvotes: 1

Views: 88

Answers (1)

jezrael
jezrael

Reputation: 863351

You can filter out empty DataFrames:

df_queue_inbound = pd.concat((x for x in map(pd.read_csv, QInboundCSV)
                              if not x.empty), ignore_index=True)

Or:

df_queue_inbound = pd.concat(filter(lambda x: not x.empty, 
                             map(pd.read_csv, QInboundCSV)), ignore_index=True)

Upvotes: 1

Related Questions