CODEWITHSUNDEEP
pythongoogle-app-enginemultiprocessing
user649542
user649542

Reputation: 113

Convert Python2.6 to Google App Engine compatible (multiprocessing)

I've made a little script in Python which uses multiprocessing. I've thought of running it on the Google App Engine as a cron-job, but unfortunately Google App Engine doesn't support multiprocessing. Can anyone help me convert this into Google App Engine compatible code (perhaps using Google App Engine tasks?)?

from multiprocessing import Pool
import MySQLdb
import urllib;
import urllib2;

def f(email_url):
    url = "http://my-domain.com/cron.php"
    values = { "email" : email_url[0], "url" : email_url[1] }
    data = urllib.urlencode(values)
    req = urllib2.Request(url, data)
    urllib2.urlopen(req)

if __name__ == '__main__':
    p = Pool()
    emails_urls = list()
    conn = MySQLdb.connect(host = "XXX.XXX.XXX.XXX", user = "USERNAME",
                           passwd = "PASSWORD", db = "MY-DATABASE")
    cursor = conn.cursor()
    cursor.execute ("SELECT email, url FROM data")
    rows = cursor.fetchall()
    for row in rows:
        emails_urls.append((row[0], row[1]))
    cursor.close()
    conn.close()

    p.map(f, emails_urls)

Upvotes: 0

Views: 258

Answers (1)

Daniel Kurka
Daniel Kurka

Reputation: 7985

Take a look at Task Queues.

ca can insert an amount of work into a task Queue (=> Thread) and set the number of jobs in a queue which are executed simultaniously.

Take a look here: http://code.google.com/intl/de-DE/appengine/docs/python/taskqueue/

Upvotes: 5

Related Questions