CODEWITHSUNDEEP
pythonfilecsv
Ethan Kershner
Ethan Kershner

Reputation: 131

CSV reader not reading entire file

I have looked at previous answers to this question, but in each of those scenarios the questioners were asking about something specific they were doing with the file, but the problem occurs for me even when I am not.

I have a .csv file of 27,204 rows. When I open the python interpreter:

python
import csv
o = open('btc_usd1hour.csv','r')
p = csv.reader(o)
for row in p:
  print(row)

I then only see roughly the last third of the document displayed to me.

Upvotes: 3

Views: 4171

Answers (3)

Andrea Perelli
Andrea Perelli

Reputation: 186

Try so, at me works: 

             with open(name) as csvfile:
                reader = csv.DictReader(csvfile)
                for row in reader:
                  print(row)

reference:

https://docs.python.org/3.6/library/csv.html#csv.DictReader

Upvotes: 1

pepr
pepr

Reputation: 20794

Try the following code

import csv

fname = 'btc_usd1hour.csv'
with open(fname, newline='') as f:
    reader = csv.reader(f)
    for row in reader:
        print(row)

It is difficult to tell what is the problem without having the sample. I guess the problem would be removed if you add that newline='' for opening the file.

Use the with construct to close the file automatically. Use the f name for a file object when no further explanation is needed. Store the file name to fname to make future modifications easier (and also for easy copying the code fragment for your later programs).

olisch may be right that the console just scrolled so fast you could not see the result. You can write the result to another text file like this:

with open(fname, newline='') as fin,\
     open('output.txt', 'w') as fout:
    reader = csv.reader(fin)
    for row in reader:
        fout.write(repr(row) + '\n')

The repr function converts the row list into its string representation. The print calls that function internally, so you will have the same result that you otherwise observe on screen.

Upvotes: 1

olisch
olisch

Reputation: 1000

maybe your scrollback buffer is just to short to see the whole list?

In general your csv.reader call should be working fine, except your 27k rows aren't extremly long so that you might be able to hit any 64bit boundaries, which would be quite uncommon.

len(o) might be interesting to see.

Upvotes: 0

Related Questions