CODEWITHSUNDEEP
pythonbashcutcsv
alvas
alvas

Reputation: 122142

How to cut 2nd and 3rd column out of a textfile? python

I have a tab-delimited file with lines as such:

foo bar bar <tab>x y z<tab>a foo foo
...

Imagine 1,000,000 lines, with up to 200 words per line. each word on average of 5-6 characters.

To the 2nd and 3rd column, I can do this:

with open('test.txt','r') as infile:
  column23 = [i.split('\t')[1:3] for i in infile]

or i could use unix, How can i get 2nd and third column in tab delim file in bash?

import os
column23 = [i.split('\t') os.popen('cut -f 2-3 test.txt').readlines()]

Which is faster? Is there any other way to extract the 2nd and 3rd column?

Upvotes: 0

Views: 2284

Answers (2)

chepner
chepner

Reputation: 531798

Use neither. Unless it proves to be too slow, use the csv module, which is far more readable.

import csv
with open('test.txt','r') as infile:
    column23 = [ cols[1:3] for cols in csv.reader(infile, delimiter="\t") ]

Upvotes: 3

Tim Pietzcker
Tim Pietzcker

Reputation: 336378

If there can be hundreds of tab-delimited entries per line, and you only want the second and third, then you don't need to split all of them; there is a maxsplit parameter you can use that should speed things up:

with open('test.txt','r') as infile:
    column23 = [i.split('\t', 3)[1:3] for i in infile]

And who knows, maybe a clever regex would even be faster:

import re
regex = re.compile("^[^\t\n]*\t([^\t\n]*)\t([^\t\n]*)", re.MULTILINE)
with open('test.txt','r') as infile:
    columns23 = regex.findall(infile.read())

Upvotes: 1

Related Questions