Reputation: 321

Match multiline regex in file object

How can I extract the groups from this regex from a file object (data.txt)?

import numpy as np
import re
import os
ifile = open("data.txt",'r')

# Regex pattern
pattern = re.compile(r"""
                ^Time:(\d{2}:\d{2}:\d{2})   # Time: 12:34:56 at beginning of line
                \r{2}                       # Two carriage return
                \D+                         # 1 or more non-digits
                storeU=(\d+\.\d+)
                \s
                uIx=(\d+)
                \s
                storeI=(-?\d+.\d+)
                \s
                iIx=(\d+)
                \s
                avgCI=(-?\d+.\d+)
                """, re.VERBOSE | re.MULTILINE)

time = [];

for line in ifile:
    match = re.search(pattern, line)
    if match:
        time.append(match.group(1))

The problem in the last part of the code, is that I iterate line by line, which obviously doesn't work with multiline regex. I have tried to use pattern.finditer(ifile) like this:

for match in pattern.finditer(ifile):
    print match

... just to see if it works, but the finditer method requires a string or buffer.

I have also tried this method, but can't get it to work

matches = [m.groups() for m in pattern.finditer(ifile)]

Any idea?

After comment from Mike and Tuomas, I was told to use .read().. Something like this:

ifile = open("data.txt",'r').read()

This works fine, but would this be the correct way to search through the file? Can't get it to work...

for i in pattern.finditer(ifile):
    match = re.search(pattern, i)
    if match:
        time.append(match.group(1))

Solution

# Open file as file object and read to string
ifile = open("data.txt",'r')

# Read file object to string
text = ifile.read()

# Close file object
ifile.close()

# Regex pattern
pattern_meas = re.compile(r"""
                ^Time:(\d{2}:\d{2}:\d{2})   # Time: 12:34:56 at beginning of line
                \n{2}                       # Two newlines
                \D+                         # 1 or more non-digits
                storeU=(\d+\.\d+)           # Decimal-number
                \s
                uIx=(\d+)                   # Fetch uIx-variable
                \s
                storeI=(-?\d+.\d+)          # Fetch storeI-variable
                \s
                iIx=(\d+)                   # Fetch iIx-variable
                \s
                avgCI=(-?\d+.\d+)           # Fetch avgCI-variable
                """, re.VERBOSE | re.MULTILINE)

file_times = open("output_times.txt","w")
for match in pattern_meas.finditer(text):
    output = "%s,\t%s,\t\t%s,\t%s,\t\t%s,\t%s\n" % (match.group(1), match.group(2), match.group(3), match.group(4), match.group(5), match.group(6))
    file_times.write(output)
file_times.close()

Maybe it can be written more compact and pythonic though....

Upvotes: 13

Answers (3)

SilentGhost

Reputation: 319531

times = [match.group(1) for match in pattern.finditer(ifile.read())]

finditer yield MatchObjects. If the regex doesn't match anything times will be an empty list.

You can also modify your regex to use non-capturing groups for storeU, storeI, iIx and avgCI, then pattern.findall will contain only matched times.

Note: naming variable time might shadow standard library module. times would be a better option.

Upvotes: 3