CODEWITHSUNDEEP
pythonjsonpython-2.7
CEamonn
CEamonn

Reputation: 925

Json file only appends first hit of python script

I've a python script that runs through a directory, searches for a string and prints out the file and line when the string is read

for check_configs in glob.glob('/etc/ansible/logs/*.txt'):
    with open(check_configs) as f:
        for line in f:
            #find which line string occurs
            data = []
            if 'test' in line:
                data.append({'File':check_configs,'Line':line})
                print data
                #print file and line in json file
                with open('/etc/ansible/results/get_ip_check', 'w') as outfile:
                    json.dump(data, outfile, indent=4)      
            else:
                #if not in line continue to next line/file
                continue

For debugging purposes, I've included the line print data just to print what I get. This prints out every time the string occurs, 

[{'Line': 'test\n', 'File': '/my/test/dir/logs/log2.txt'}]

[{'Line': 'test other test config\n', 'File': '/my/test/dir/logs/log1.txt'}]

however the json file only prints out one occurrence of the string

[
    {
        "Line": "test\n", 
        "File": "/my/test/dir/logs/log1.txt"
    }
]

Am I missing something with the json loop?

Upvotes: 0

Views: 46

Answers (2)

Thierry Lathuille
Thierry Lathuille

Reputation: 24280

You reset data to the empty list [] in each loop. You have to move it out of the loop:

for check_configs in glob.glob('/etc/ansible/logs/*.txt'):
    with open(check_configs) as f:
        data = []   # We create the empty data once before the loop
        for line in f:
            #find which line string occurs
            if 'test' in line:
                data.append({'File':check_configs,'Line':line})
                print data
                #print file and line in json file

            else:
                #if not in line continue to next line/file
                continue

with open('/etc/ansible/results/get_ip_check', 'w') as outfile:
    json.dump(data, outfile, indent=4)

Upvotes: 3

Daniel Roseman
Daniel Roseman

Reputation: 599866

Because you open the file every time within the loop and just dump the value from the current iteration, overwriting whatever has gone before.

You should define data before the beginning of the loop, and move the open and dump to after the end of the loop, so that data is accumulated within the loop and you only write once.

(Note also, else: continue is pointless since that is what happens anyway; remove those lines.)

data = []
for check_configs in glob.glob('/etc/ansible/logs/*.txt'):
    with open(check_configs) as f:
        for line in f:
            if 'test' in line:
                data.append({'File':check_configs,'Line':line})

with open('/etc/ansible/results/get_ip_check', 'w') as outfile:
    json.dump(data, outfile, indent=4)

Upvotes: 3

Related Questions