CODEWITHSUNDEEP
pythonregextextpython-3.x
Eden Crow
Eden Crow

Reputation: 16194

Python 3: Searching A Large Text File With REGEX

I wish to search a large text file with regex and have set-up the following code:

import re

regex = input("REGEX: ")

SearchFunction = re.compile(regex)

f = open('data','r', encoding='utf-8')

result = re.search(SearchFunction, f)

print(result.groups())

f.close()

Of course, this doesn't work because the second argument for re.search should be a string or buffer. However, I cannot insert all of my text file into a string as it is too long (meaning that it would take forever). What is the alternative?

Upvotes: 8

Views: 13544

Answers (2)

Steven
Steven

Reputation: 28666

You can use a memory-mapped file with the mmap module. Think of it as a file pretending to be a string (or the opposite of a StringIO). You can find an example in this Python Module of the Week article about mmap (direct link) by Doug Hellman.

Upvotes: 7

Mariusz Jamro
Mariusz Jamro

Reputation: 31653

You check if the pattern matches for each line. This won't load the entire file to the memory:

for line in f:
    result = re.search(SearchFunction, line)

Upvotes: 6

Related Questions