CODEWITHSUNDEEP
pythoncsv
Malte Susen
Malte Susen

Reputation: 845

Saving loops without overwriting data

For a current project, I am running three loops for the JSON objects 'Text_Main','Text_Pro' and 'Text_Con'. The print() output in my terminal is correctly showing the consecutive results for all three iterations.

When saving the results in a CSV file, the script however starts overwriting the previous iterations and only saves the third iteration. Is there any way to have the loops appended to the previous loop, i.e. ending up with a CSV file that includes the results from all three iterations?

The corresponding code looks like this:

for i in ['Text_Main','Text_Pro','Text_Con']:
    common_words = get_top_n_trigram(df[i], 50)
    for word, freq in common_words:
        print(i, word, freq)

        with open('Glassdoor_A.csv', 'a', newline='') as file:
            writer = csv.writer(file)
            writer.writerow(["Section", "Word", "Frequency"])
            writer.writerows([i, word, freq] for word,freq in common_words)

Upvotes: 0

Views: 241

Answers (2)

Nico Griffioen
Nico Griffioen

Reputation: 5405

You have a nested for loop, so for each word, freq in common_words, you open a new CSV writer.

More correct would be:

# Open the file to write to:
with open('Glassdoor_A.csv', 'a', newline='') as file:
    writer = csv.writer(file)
    # Write headers
    writer.writerow(["Section", "Word", "Frequency"])
    # Loop over the JSON objects.
    for i in ['Text_Main','Text_Pro','Text_Con']:
        # Loop over the common words inside the JSON object
        common_words = get_top_n_trigram(df[i], 50)
        for word, freq in common_words:
            # Print and write row.
            print(i, word, freq)
            writer.writerow([i, word, freq])

Upvotes: 1

ludel
ludel

Reputation: 91

I think the problem is that you open the file on each iteration. And, you can use the file parameter in print function.

with open('Glassdoor_A.csv', 'a', , newline='') as file:
    for i in ['Text_Main','Text_Pro','Text_Con']:
        common_words = get_top_n_trigram(df[i], 50)
        for word, freq in common_words:
            print(i, word, freq, sep=',', file=file)

Upvotes: 0

Related Questions