python parse large logs to extract certain patterns

Question

I have large system log (max 1GB) and I need to parse it for extracting certain things.

initially i wrote python code to reading file line by line (using with open) it took very very long time

I learned about mmap and using mmap it's taking around 5 mins .

(I have precompiled the regular expression to save time)

is there any better approach so that it takes less time ? (i am using python 2.7.3 32bit )

Answer 1

You might want to use awk for that kind of things, it's more or less what it's designed to do.