CODEWITHSUNDEEP
pythonpandasdataframereindex
Mainland
Mainland

Reputation: 4564

Pandas Timeseries reindex producing NaNs

I am surprised that my reindex is producing NaNs in whole dataframe when the original dataframe does have numerical values init. Don't know why?

Code:

df = 
                               A  ...                            D
Unnamed: 0                        ...                                
2022-04-04 11:00:05          NaN  ...                          2419.0
2022-04-04 11:00:10          NaN  ...                          2419.0

## exp start and end times
exp_start, exp_end = '2022-04-04 11:00:00','2022-04-04 13:00:00'
## one second index
onesec_idx = pd.date_range(start=exp_start,end=exp_end,freq='1s')
## map new index to the df
df = df.reindex(onesec_idx)

Result:

df = 
                               A  ...                            D
2022-04-04 11:00:00          NaN  ...                             NaN
2022-04-04 11:00:01          NaN  ...                             NaN
2022-04-04 11:00:02          NaN  ...                             NaN
2022-04-04 11:00:03          NaN  ...                             NaN
2022-04-04 11:00:04          NaN  ...                             NaN
2022-04-04 11:00:05          NaN  ...                             NaN

Upvotes: 1

Views: 483

Answers (1)

NeuroWinter
NeuroWinter

Reputation: 511

From the documentation you can see that df.reindex() will Places NA/NaN in locations having no value in the previous index.

However you can also provide a value that you want to replace missing values with (It defaults to NaN):

df.reindex(onesec_idx, fill_value='')

If you want to replace the NaN in a particular column or even in the whole dataframe you can run something like after doing a reindex:

df.fillna('',inplace=True)  # for replacing NaN in the entire df with ''

df['d'].fillna(0, inplace=True)  # if you want to replace all NaN in the D column with 0

Sources:

Documentation for reindex: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.reindex.html

Documentation for fillna: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.fillna.html

Upvotes: 1

Related Questions