CODEWITHSUNDEEP
python-3.xpandas
aviral sanjay
aviral sanjay

Reputation: 983

Pandas not reading headers properly

I have a csv file which is:

Sl. No,City,Name of the Scheme,Completed
1,Chennai,Collector Nagar + 8 slums,285
2,Chennai,Telugu & Tamil Harijan Colony ,138
3,Chennai,Kannabiran Koil Street + 2 Slums,95
4,Coimbatore,Machampalayam Mariamman Koil Stret and Kurichi Boyer Street,223

My code:

import sys

import pandas as pd

path = "/home/aviral/dev/local_sink/socialcops/datasets/opendata/config/file-upload-csv_kZCcyHvHM.csv/5bf7e7cf0ca1c9000712f3c1.csv"
df = pd.read_csv(path)

headers = list(df)
counter = 0
for row in df.itertuples(index=False):
    row = row._asdict()
    print(row)
    if counter > 10:
        sys.exit()
    counter += 1

The output that I am getting is:

OrderedDict([('_0', '1'), ('City', 'Chennai'), ('_2', 'Collector Nagar + 8 slums'), ('Completed', 285)])
OrderedDict([('_0', '2'), ('City', 'Chennai'), ('_2', 'Telugu & Tamil Harijan Colony '), ('Completed', 138)])
OrderedDict([('_0', '3'), ('City', 'Chennai'), ('_2', 'Kannabiran Koil Street + 2 Slums'), ('Completed', 95)])
OrderedDict([('_0', '4'), ('City', 'Coimbatore'), ('_2', 'Machampalayam Mariamman Koil Stret and Kurichi Boyer Street'), ('Completed', 223)])
OrderedDict([('_0', '5'), ('City', 'Erode '), ('_2', 'Kamaraj Nagar '), ('Completed', 50)])
OrderedDict([('_0', '6'), ('City', 'Dindigul '), ('_2', 'East Mariyanathapuram '), ('Completed', 168)])
OrderedDict([('_0', '7'), ('City', 'Madurai'), ('_2', 'Anaiyur '), ('Completed', 305)])
OrderedDict([('_0', '8'), ('City', 'Madurai'), ('_2', 'Ahimsapuram + 2 slums'), ('Completed', 498)])
OrderedDict([('_0', '9'), ('City', 'Salem'), ('_2', 'Anna Nagar + 8 Slums '), ('Completed', 1073)])
OrderedDict([('_0', '10'), ('City', 'Trichy '), ('_2', 'Keela Devadhanam + 13 slums'), ('Completed', 938)])
OrderedDict([('_0', '11'), ('City', 'Thoothukudi '), ('_2', 'Kakanji Nagar + 3 slums'), ('Completed', 205)])
OrderedDict([('_0', '12'), ('City', 'Tirunelveli '), ('_2', 'Thiruvannathapuram Pottal + 11 Slums'), ('Completed', 208)])

Note: There is no key in any of the rows above named 'Sl. No' whereas in the list of the headers, the dataframe does show the same column.

Upvotes: 1

Views: 1065

Answers (1)

Toby Petty
Toby Petty

Reputation: 4660

The reason for this is because there are invalid characters in your column names that aren't recognised by df.itertuples. From the pandas documentation for itertuples:

The column names will be renamed to positional names if they are invalid Python identifiers, repeated, or start with an underscore. With a large number of columns (>255), regular tuples are returned.

If you add the following lines before the loop in your code to remove spaces and periods, then it will print column names:

df.rename(columns={c:c.replace(" ", "_") for c in df.columns}, inplace=True)
df.rename(columns={c:c.replace(".", "") for c in df.columns}, inplace=True)

Upvotes: 1

Related Questions