CODEWITHSUNDEEP
pythonrq
0x2328
0x2328

Reputation: 1

Python RQ csv file processing

What is more preferable way to process .csv files using RQ? File possible can have few or few hundred thousands rows

def process_row():
    pass

# Is it better to push function per csv row as a job.

for row in csv:
    queue.enqueue(process_row, row)

# or process the whole csv file as one job ( loops over csv file in job process )

def some_func():
    for row in csv:
        process_row(row)

queue.enqueue(some_func, csv_path)

I was not able to find answer to that question in RQ docs.

Upvotes: 0

Views: 34

Answers (0)

Related Questions