CODEWITHSUNDEEP
pythonaws-lambda
user12913302
user12913302

Reputation:

How to use AWS lambda in python to run multiple functions at once

I currently have code that gets back school data from a database and saves it to a csv file:

schoolID = '12345'

def getSchool(schoolID):
    School = SchoolsDB.find_one({"_id": ObjectId(schoolID)})
    return School
school = getSchool(schoolID)


school.to_csv(schoolID + ".csv")

It currently takes in a schoolID and runs one school at a time. I have tried putting it in a for loop so that it runs one school after another automatically, but I want to be able to run all schools at the same time.

I want to be able to use lambda to run all the schools at the same time, instead of one at a time. Does anyone know how to do this?

Upvotes: 0

Views: 881

Answers (1)

Bob
Bob

Reputation: 425

From a purely Python POV:

It looks like you are using some form of MongoDB, rather than having the function take a single ID and executing each time, why not pass it an array and find them all in one go. 

def getSchool(list_of_school_ids):
    Schooldb.collection.find( { _id : { $in : list_of_school_ids} } )

school = getSchool(list_of_school_ids=["1234", "5678"])

Then just build a massive CSV where each row is your entry in SchoolsDB - I'm coming at this completely blind though.

If not, you could look at:

https://aws.amazon.com/blogs/compute/parallel-processing-in-python-with-aws-lambda/

But my gut tells me its overkill for your use case :)

Upvotes: 1

Related Questions