CODEWITHSUNDEEP
pythonmacosprocessmultiprocessing
mrgloom
mrgloom

Reputation: 21632

What does "The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec()." error mean?

What does this error mean? Seems this happens when I use some C++ wrapped code under process (no errors when I use it in a plain for loop).

The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.

Example code:

def split_to_chunks(items, k):
    n = len(items) // k
    chunks = []
    for i in range(0, len(items), n):
        chunks.append(items[i:i + n])
    return chunks

def process_chunk(process_id, img_filepaths, output_dir):
    print('Enter process_id:', process_id)
    for img_filepath in img_filepaths:
        try:
           //Code section
        except Exception as e:
            print(f'Error: process_id: {process_id} : {img_filepath}', e)
    print('End process_id:', process_id)

img_filepaths = get_img_filepaths(images_dir)

#n_processes = multiprocessing.cpu_count() // 2
n_processes = 1
img_filepaths_chunks = split_to_chunks(img_filepaths, n_processes)

processes = []
for i in range(n_processes):
    process = multiprocessing.Process(
        target=process_chunk,
        args=(i, img_filepaths_chunks[i], output_dir)
    )
    processes.append(process)
for process in processes:
    process.start()
for i, process in enumerate(processes):
    process.join()
    if process.exitcode != 0:
        print(f'Error: process_id: {i} failed.')
        #sys.exit(1)

Upvotes: 2

Views: 3837

Answers (1)

mrgloom
mrgloom

Reputation: 21632

Seems using 'spawn' like suggested here https://turtlemonvh.github.io/python-multiprocessing-and-corefoundation-libraries.html is solving the problem.

import multiprocessing as mp
mp.set_start_method('spawn')

Upvotes: 3

Related Questions