How to limit number of parallel file copy using gsutil

Question

As per gsutil cp command documentation -

https://cloud.google.com/storage/docs/gsutil/commands/cp

we can use gsutil -m option for parallel multi-threaded/multi-processing file copy.

Since we are using gsutil inside a docker container and the POD size is very small which will not be helpful if more that 5 files are getting copied in parallel.

Is there any way we can limit the number of files to be copied in parallel using gsutil -m option ?

Answer 1

gsutil performs the specified operation using a combination of multi-threading and multi-processing. The number of threads and processors are determined by parallel_thread_count and parallel_process_count, respectively. These values are set in the .boto configuration file or specified in individual requests with the -o top-level flag.

Boto configuration file