CODEWITHSUNDEEP
pythonexchange-serverexchangelib
pythoncoder
pythoncoder

Reputation: 51

Python: Function hangs in child process

I wrote this script which uses the exchangelib python API to log into my Microsoft exchange account and download sent messages that contain a particular subject.

I split it into two functions each running as a seperate process:

Function A fetches emails from my account and places the ones that match a specific subject in a Queue.

Function B takes these emails from the Queue and saves them to disk.

These functions work properly when run in a single process but when I decided to run function A in a child process while function B runs in main process, function A just hangs at the line that downloads the emails.

My code is below.

from exchangelib import Credentials, Account, Mailbox, HTMLBody, Message

import multiprocessing as mp
from multiprocessing import Process, Queue


def authenticate(user_id, passw):
    credentials = Credentials(user_id, passw)
    exchange = Account(account_id, credentials=credentials, autodiscover=True)
    return exchange

def fetch_bounces(exchange, output_q, progress):
    print('fetcher process started')
    while exchange.inbox.total_count != 0:
        exchange.inbox.refresh()
        inbox=exchange.inbox.all()
        for msg in inbox:
            if 'Undeliverable:' in msg.subject:
                output_q.put(msg.display_to)
        inbox.move_to_trash()
    progress.put('done')

def save_bounced(outputfile, output_q):
    bounces = []
    while output_q.empty() is False:
        lead = output_q.get()
        print('bounce: '+lead)
        bounces.append(lead)
    if len(bounces)>0:
        existing = []
        try:
            with open(outputfile, 'r+') as f:
                lines = f.readlines()
                existing = [line.strip().lower() for line in lines]
        except:
            pass
        with open(outputfile, 'a+') as f:
            for line in bounces:
                if line.strip().lower() in existing:
                    continue
                else:
                    f.write(line.strip().lower()+'\n')

if __name__ == '__main__':
    #credentials
    account_id =
    password = 


    #set Queues to store bounces and progress
    bounced = Queue()
    progress = Queue()

    #login to exhcnage
    exchange = authenticate(account_id, password)
    print('logged in successfully')

    #spawn bounce fetching process and start it
    f_process = Process(target=fetch_bounces, args=(exchange, bounced, progress))
    f_process.start()

    #define file path where bounces will be stored
    output_f = 'bounces.txt'

    #while new bounce emails are being fetch, remain in loop and also specify flag to know when all emails have been fetched

    complete = False
    while not complete:
        save_bounced(output_f, bounced)
        try:
            msg = progress.get(block=False)
            if msg == 'done':
                complete =  True
        except:
            pass

    #terminate fetcher process

    f_process.join()

    print('bounces completely removed')

Upvotes: 2

Views: 198

Answers (1)

Erik Cederstrand
Erik Cederstrand

Reputation: 10220

When you are using multiprocessing, all arguments must be serializable. But an Account instance cannot be safely serialized. You must create the Account instance within each process.

Upvotes: 1

Related Questions