CODEWITHSUNDEEP
pythonmultithreadingiomultiprocessing
rocksNwaves
rocksNwaves

Reputation: 6164

multithreading or multiprocessing for general file operations?

I've recently read a couple great SO questions/answers on the difference between multithreading and multiprocessing. I can't say that I understood everything that I read, but I think I get the gist. I thought a great way to learn more was to choose one or the other and use it for a job I need to do anyway.

Namely, I've got a bunch of folders each with a bunch of mp3 files. I want to resample all the files in all the folders to be at the same sampling rate and number of channels and then save them as .wav files. I'm thinking I could assign one folder per process until all the folders have been converted.

I've read that multi-threading often doesn't result in reduced time because threads are hard to run in parallel. This is why I am leaning towards this option.

Would multiprocessing give faster results for file IO operations than multi-threading?

Upvotes: 0

Views: 477

Answers (1)

Matthew
Matthew

Reputation: 184

It would depend on what dependencies there are in the problem you are trying to solve. If you can assign a task with little to no dependencies (shared data, shared variables, etc.) to run on its own then multi-processing would be great. As long as starting the processes doesn't take longer than it would be to just run it. (Starting new processes is expensive)

With that being said I think it would work well to assign one mp3 conversion task per process. Just make sure you have something to monitor which mp3 files have already been resampled and converted to prevent duplicate processes.

Upvotes: 2

Related Questions