Reputation: 41
I am writing a code where the final results(denoted by x) to be exported to csv file. I used a for loop to iterate over but stil it only exports the last line of the result. My full code below is:
import csv
import itertools
import requests
import json
import pandas as pd
domainfile=open('domainsinfo.csv',newline='',encoding='utf_8')
reader=csv.reader(domainfile)
w=[]
for row in reader:
w.extend(row)
domain = list(itertools.permutations(w,1))
print(domain)
def url_report(domain):
url = 'https://www.virustotal.com/vtapi/v2/url/report'
params = {'apikey': '', 'resource':domain}
response = requests.get(url, params=params)
return response
def pp_json(json_thing, sort=True, indents=4):
if type(json_thing) is str:
print(json.dumps(json.loads(json_thing), sort_keys=sort,
indent=indents))
else:
print(json.dumps(json_thing, sort_keys=sort,
indent=indents))
return None
for d in domain:
response = url_report(d)
json_response = response.json()
pretty_json = pp_json(json_response)
response_list=[]
for key in json_response['scans']:
if json_response['scans'][key] ['detected'] is True:
response_list.append(True)
else:
response_list.append(False)
x=any(response_list)
print(x)
for d in domain:
final_list=[]
final_list.append(x)
result=(final_list)
result_table = {'Domain': [d], 'Result':result}
df=pd.DataFrame(data=result_table)
print(df)
export_csv = df.to_csv (r'C:\csv', index=None, header=True)
print(pretty_json)
input()
Can someone explain why this does not work even if the for loop is present
Upvotes: 0
Views: 1475
Reputation: 6677
Like liakoyras mentioned, you are writing your dataframe in the CSV. There are many ways to overcome your problem. While he suggested a merge, I am giving you another alternative here using append. Read the documentation for more information.
Here is how you need to modify your code block, explanation in comments inside the code:
######## Declare a dataframe outside the loop
final = pd.DataFrame()
for d in domain:
........................
your other codes
........................
result_table = {'Domain': [d], 'Result':result}
df=pd.DataFrame(data=result_table)
###### keep appending df to final
###### the final df now gets updated in every loop
final = final.append(df)
#### now outside the loop, write final dataframe to your csv
final.to_csv('your/path/file.csv')
Upvotes: 2
Reputation: 1185
This happens because in each loop, your .csv is overwritten by your next one.
The df.to_csv exports the df in a csv file.
You could try taking this code out of the loop, or changing the file name dynamically (depending on what you want.
Upvotes: 2