CODEWITHSUNDEEP
pythonpandasnumpy
Avij
Avij

Reputation: 694

How to know CSV line count before loading in python?

I am new to python and have a requirement to load dataframes from various CSV files. It turns out that there is a business logic depending on the number of rows in csv. Can i know this beforehand if i can know CSV total row numbers without writing read_csv?

Upvotes: 4

Views: 5005

Answers (2)

MaxU - stand with Ukraine
MaxU - stand with Ukraine

Reputation: 210842

yes, you can:

lines = sum(1 for line in open('/path/to/file.csv'))

but be aware that Pandas will read the whole file again

if you are sure that the whole file will fit into memory we can do this:

with open('/path/to/file.csv') as f:
    data = f.readlines()
    lines = len(data)
    df = pd.read_csv(data, ...)

Upvotes: 5

Stuart Buckingham
Stuart Buckingham

Reputation: 1774

You can read the file without saving the contents. Try:

with open(filename, "r") as filehandle:
    number_of_lines = len(filehandle.readlines())

Upvotes: 0

Related Questions