CODEWITHSUNDEEP
pythonperformancepandasdictionarydataframe
user9606984
user9606984

Reputation:

Faster way of remapping dictionary to dataframe

I am trying to remap data from dictionary to the respective values in dataframe and I am successful in it.

It is fine when I am using few thousands of data but gets radically slow and sometimes never ending if I try to do it with few millions of data.

Here is the code I am using and works fine for less data: 

def remap(data,dict_labels):
    for field,values in dict_labels.items():
        data.replace({field:values},inplace=True)
    print("DONE")
    return data
dataframe1 = remap(df,dataDict)

Is there any efficient way of using something else for the same task so that it is faster for large data?

Upvotes: 2

Views: 209

Answers (1)

jpp
jpp

Reputation: 164693

You should use pd.DataFrame.applymap for this:

dataframe1 = df.applymap(dict_labels.get).fillna(df)

Note we use fillna to recover unmapped values.

As per the documentation:

DataFrame.applymap(func)

Apply a function to a DataFrame that is intended to operate elementwise, i.e. like doing map(func, series) for each series in the DataFrame

Performance note: Replace values in a pandas series via dictionary efficiently

Upvotes: 2

Related Questions