CODEWITHSUNDEEP
pythonnumpytext
user3601754
user3601754

Reputation: 3862

Loadtext for specific number of lines

I have a huge file to load so I can't open it directly. I think I have to read it in multiple parts.

For instance, in order to use lines 1 to 50, I tried with something like that, but it doesn't work:

import numpy as np

with open('test.txt') as f:
    lines = (line for line in f if line < 50.)
    FH = np.loadtxt(lines, delimiter=',', skiprows=1)

Upvotes: 1

Views: 6048

Answers (4)

Martin Evans
Martin Evans

Reputation: 46759

As you are using numpy, you can use the islice() function in the built in itertools library to load just the first 50 lines as follows:

import numpy as np
import itertools

with open('test.txt') as f_input:
    FH = np.loadtxt(itertools.islice(f_input, 0, 50), delimiter=',', skiprows=1)

Upvotes: 4

Xavier Guihot
Xavier Guihot

Reputation: 61646

Starting Numpy 1.16, np.loadtxt takes an optional parameter max_rows which limits the number of lines to read:

import numpy as np

np.loadtxt('file.txt', max_rows=50, delimiter=',', skiprows=1)

Upvotes: 2

Remi Guan
Remi Guan

Reputation: 22272

In Python 3, please try this:

a = 0
f = open('test.txt')
while a < 50:
    a = a + 1
    print(f.readline(), end='')
else:
    f.close()

In Python 2, use this:

a = 0
f = open('test.txt')
while a < 50:
    a = a + 1
    print f.readline(),
else:
    f.close()


or choose this way, use readlines():

with open('test.txt') as f:
    text = f.readlines()

readlines() will create a list, and one line is one object. so if you want the 20 - 50 lines, you can do this:

for i in text[20:50]:
    print(i)

Upvotes: 1

mmachine
mmachine

Reputation: 926

I use function to read arbitrary first, last N-lines from file:

def read_first_last_N_lines_from_file(in_file,N,last=False):
    with open(in_file) as myfile:
        if last:
            return [x.strip() for x in list(myfile)][-N:]
        return [next(myfile).strip() for x in range(N)]

Upvotes: 1

Related Questions