CODEWITHSUNDEEP
pythonmultiprocessingfile-writing
Chan
Chan

Reputation: 4311

python multi-processing on writing file and return value

After reading data from database, I want to return data and write it to a csv file. How to use multi-processing to do it?

def get_data():
    data = get_data_from_database() #a dataframe
    data.to_csv('data.csv', index=False) #step 1: write to csv file
    return data #step 2: return data

How to use multi-processing to do step 1 and step 2?

Upvotes: 0

Views: 286

Answers (2)

Hamid Sajjadi
Hamid Sajjadi

Reputation: 133

You mean multi-threading? If yes, then if you want to write to the file somewhere else you should use locks and it can become a pain in the code!

But if you are not worried about that you can do something like this

import threading

def thread_write(data):
    data.to_csv('data.csv', index=False)  # step 1: write to csv file

def get_data():
    data = get_data_from_database() #a dataframe
    t = threading.Thread(target=thread_write, args=(data,)) #pass your function as target of thread, and it's input variables as a tuple to args
    t.start()
    return data  # step 2: return data

Upvotes: 1

Hafsa Elif Özçiftci
Hafsa Elif Özçiftci

Reputation: 331

Try this; 

def get_data():
        data = []
        data = get_data_from_database() #a dataframe
        if len(data) >0:
           for i in range (len(data)):
           data[i].to_csv('data.csv', index=False) #step 1: write to csv file
        return data #step 2: return data

Upvotes: 1

Related Questions