CODEWITHSUNDEEP
pythonpandas
user1642513
user1642513

Reputation:

Pandas: unable to change column data type

I was following the advice here to change the column data type of a pandas dataframe. However, it does not seem to work if I reference the columns by index numbers instead of column names. Is there a way to do this correctly?

In [49]: df.iloc[:, 4:].astype(int)
Out[49]: 
&ltclass 'pandas.core.frame.DataFrame'&gt
Int64Index: 5074 entries, 0 to 5073
Data columns (total 3 columns):
5    5074  non-null values
6    5074  non-null values
7    5074  non-null values
dtypes: int64(3) 

In [50]: df.iloc[:, 4:] = df.iloc[:, 4:].astype(int)

In [51]: df
Out[51]: 
&ltclass 'pandas.core.frame.DataFrame'&gt
Int64Index: 5074 entries, 0 to 5073
Data columns (total 7 columns):
1    5074  non-null values
2    5074  non-null values
3    5074  non-null values
4    5074  non-null values
5    5074  non-null values
6    5074  non-null values
7    5074  non-null values
dtypes: object(7) 

In [52]:

Upvotes: 6

Views: 12618

Answers (1)

Jeff
Jeff

Reputation: 128978

Do it like this

In [49]: df = DataFrame([['1','2','3','.4',5,6.,'foo']],columns=list('ABCDEFG'))

In [50]: df
Out[50]: 
   A  B  C   D  E  F    G
0  1  2  3  .4  5  6  foo

In [51]: df.dtypes
Out[51]: 
A     object
B     object
C     object
D     object
E      int64
F    float64
G     object
dtype: object

Need to assign columns one-by-one

In [52]: for k, v in df.iloc[:,0:4].convert_objects(convert_numeric=True).iteritems():
    df[k] = v
   ....:     

In [53]: df.dtypes
Out[53]: 
A      int64
B      int64
C      int64
D    float64
E      int64
F    float64
G     object
dtype: object

Convert objects usually does the right thing, so easiest to do this

In [54]: df = DataFrame([['1','2','3','.4',5,6.,'foo']],columns=list('ABCDEFG'))

In [55]: df.convert_objects(convert_numeric=True).dtypes
Out[55]: 
A      int64
B      int64
C      int64
D    float64
E      int64
F    float64
G     object
dtype: object

assigning via df.iloc[:,4:] with a series on the right-hand side copies the data changing type as needed, so I think this should work in theory, but I suspect that this is hitting a very obscure bug that prevents the object dtype from changing to a real (meaning int/float) dtype. Should probably raise for now.

Heres the issue to track this: https://github.com/pydata/pandas/issues/4312

Upvotes: 2

Related Questions