CODEWITHSUNDEEP
pythonpandas
Marcin Tomal
Marcin Tomal

Reputation: 33

Pandas df.isna().sum() not showing all column names

I have simple code in databricks:

import pandas as pd

data_frame = pd.read_csv('/dbfs/some_very_large_file.csv')
data_frame.isna().sum()

Out[41]:
A    0
B    0
C    0
D    0
E    0
     ..
T     0
V     0
X     0
Z     0
Y     0
Length: 287, dtype: int64

How can i see all column (A to Y) names along with is N/A values? Tried setting pd.set_option('display.max_rows', 287) and pd.set_option('display.max_columns', 287) but this doesn't seem to work here. Also isna() and sum() methods do not have any arguments that would allow me to manipulate output as far as i can say.

Upvotes: 3

Views: 4869

Answers (1)

Boskosnitch
Boskosnitch

Reputation: 774

The default settings for pandas display options are set to 10 rows maximum. If the df to be displayed exceeds this number, it will be centrally truncated. To view the entire frame, you need to change the display options.

To display all rows of df:

pd.set_option('display.max_rows',None)

Ex:

>>> df
     A   B   C
0    4   8   8
1   13  17  13
2   19  13   2
3    9   9  16
4   14  19  19
..  ..  ..  ..
7    7   2   2
8    5   7   2
9   18  12  17
10  10   5  11
11   5   3  18

[12 rows x 3 columns]
>>> pd.set_option('display.max_rows',None)
>>> df
     A   B   C
0    4   8   8
1   13  17  13
2   19  13   2
3    9   9  16
4   14  19  19
5    3  17  12
6    9  13  17
7    7   2   2
8    5   7   2
9   18  12  17
10  10   5  11
11   5   3  18

Documentation: pandas.set_option

Upvotes: 3

Related Questions