CODEWITHSUNDEEP
python
xera
xera

Reputation: 143

How to split /t without create a new line using Python

I have lists of text in one folder :

My O
name O
is O
Alex B
. O

I O
am O
from O
London B
. O

This is my codes:

import re

def read_file(filename):

file = open(filename).read().strip().split("\n\n")
lines = []
for line in file:

 lines.append(re.split(r'\t|\n', line))

return lines

train_sents = read_file(("train.txt"))

train_sents [0]

The output is:

 [ 'My',
 'O',
 'name',
 'O',
 "is',
 'O',
 'Alex',
 'B',
 '.',
 'O']

My question is..is it possible to split \t without splitting it to the new line? For the example the output will be like:

[('My', 'O'),
 ('name', 'O'),
 ("is', 'O'),
 ('Alex', 'B'),
 ('.', 'O')]

Upvotes: 2

Views: 130

Answers (2)

Padraic Cunningham
Padraic Cunningham

Reputation: 180481

Just split each line:

with open(filename) as f:
    print([tuple(line.split()) for line in f])
[('My', 'O'), ('name', 'O'), ('is', 'O'), ('Alex', 'B'), ('.', 'O')]

To separate the lines by empty lines append to the last sublist or else add a new list if we meet an empty line:

with open(infile) as f:
    l = [[]]
    for line in f:
        if line.strip():
            l[-1].append(tuple(line.split()))
        else:
            l.append([])
print(l[0])
print(l[1])

[('My', 'O'), ('name', 'O'), ('is', 'O'), ('Alex', 'B'), ('.', 'O')]
[('I', 'O'), ('am', 'O'), ('from', 'O'), ('London', 'B'), ('.', 'O')]

You could also use itertools.groupby grouping using empty lines as the delimiter:

from itertools import groupby
with open(infile) as f:
     print([list(map(str.split, v))
       for k, v in groupby(f, key=lambda x: x.strip() != "") if k])


[[['My', 'O'], ['name', 'O'], ['is', 'O'], ['Alex', 'B'], ['.', 'O']], [['I', 'O'], ['am', 'O'], ['from', 'O'], ['London', 'B'], ['.', 'O']]]

You can map to tuple if necessary.

Upvotes: 3

Avinash Raj
Avinash Raj

Reputation: 174806

You may try this,

def read_file(filename):
    fil = open(filename).read().strip().split("\n\n")
    lines = []
    for line in fil:
        s = []
        m = line.split('\n')
        for i in m:
            s.append(tuple(re.split(r'\t', i)))
        lines.append(s)    
    return lines

train_sents = read_file("file")

print train_sents[0]

Output:

[('My', 'O'), ('name', 'O'), ('is', 'O'), ('Alex', 'B'), ('.', 'O')]

Upvotes: 1

Related Questions