CODEWITHSUNDEEP
python
yoshco
yoshco

Reputation: 231

Add element to line

I have a log file with (space delimited) numbers. The problem is that the file is too big for autoit (1gig file, memory issues) and I've seen some python magic.
I want to

  1. Add an element after the 3rd
  2. Remove elements 4-6
  3. Do this as fast as possible but with respect to memory limits.

1 2 3 4 5 6 7 8 9
1 2 3 A 7 8 9

currently my regExp is 

StringRegExpReplace($line, "(\S+)\s(\S+)\s(\S+)\s(\S+)\s(\S+)\s(\S+)\s(\S+)\s(\S+)\s(\S+)\s", "$1 $2 $3 0 $7 $8 $9")

Upvotes: 0

Views: 156

Answers (2)

Chinmay Kanchi
Chinmay Kanchi

Reputation: 65923

Here's a Python solution anyway:

with open('filename', 'rb') as infile:
    with open('filename.out', 'wb') as outfile:        
        for j, line in enumerate(infile):
            if j == 0: #skip first line
                continue
            listLine = line.split(' ')
            listLine.insert(3, thingToInsert) #elements 4-6 are now elements 5-7
            listLine = (el for i, el in enumerate(listLine) if i>4 and i<=7) 
            outfile.write(' '.join(listLine))

This will be pretty quick, and won't require too much RAM, since it reads and writes the file line-by-line.

Upvotes: 2

Michael Berkowski
Michael Berkowski

Reputation: 270637

Assuming Windows, I'd probably grab a copy of gawk and do it with much less hassle than python.

c:\gawk '{print $1 " " $2 " " $3 " added " $7 " " $8 " " $9}' bigfile.log >> outputfile.log

Disclaimer: I don't have a Windows machine around to test this, but it would work on the Linux machine I'm currently using...

Upvotes: 2

Related Questions