Reputation: 4311
After reading data from database, I want to return data and write it to a csv file. How to use multi-processing to do it?
def get_data():
data = get_data_from_database() #a dataframe
data.to_csv('data.csv', index=False) #step 1: write to csv file
return data #step 2: return data
How to use multi-processing to do step 1 and step 2?
Upvotes: 0
Views: 286
Reputation: 133
You mean multi-threading? If yes, then if you want to write to the file somewhere else you should use locks and it can become a pain in the code!
But if you are not worried about that you can do something like this
import threading
def thread_write(data):
data.to_csv('data.csv', index=False) # step 1: write to csv file
def get_data():
data = get_data_from_database() #a dataframe
t = threading.Thread(target=thread_write, args=(data,)) #pass your function as target of thread, and it's input variables as a tuple to args
t.start()
return data # step 2: return data
Upvotes: 1
Reputation: 331
Try this;
def get_data():
data = []
data = get_data_from_database() #a dataframe
if len(data) >0:
for i in range (len(data)):
data[i].to_csv('data.csv', index=False) #step 1: write to csv file
return data #step 2: return data
Upvotes: 1