CODEWITHSUNDEEP
pythonsubprocessprogress-barjupyter-notebooktqdm
R S
R S

Reputation: 11869

Show progress in Python from underlying C++ process

I have a C++ program that runs for a long time and performs lots (e.g. 1,000,000) of iterations. Typically I run it from Python (usually Jupyter Notebook). I would like to see the a progress from the C++ program. Is there a convenient way to do it? Perhaps to link it to a Pythonic progress bar library, e.g. tqdm?

Upvotes: 1

Views: 766

Answers (1)

gaborous
gaborous

Reputation: 16580

Disclaimer, I'm codeveloper of tqdm.

I see 3 solutions :

  • Either the cpp lib regularly calls back to python like after processing each row of a matrix (as pandas does) and then you can use a Python progress bar like tqdm, just like for any other common python loop. The loop won't be updated at each iteration but at each callback, so it's not really real-time but if the cpp lib is fast, you won't notice anything. See the submodule tqdm_pandas for example, it works exactly like that.

  • Either the cpp lib does all the work without any callback until the end (this maximizes performances, callbacks to Python are huge slowdowns), then you need to use a cpp progress bar inside your cpp lib, as you cannot use a python one (since it will never be called until the end). There is an official cpp port of tqdm in development, this might fit your needs.

  • Last case is if your cpp program is not a linked lib but rather a standalone program that can be run from command-line. In this case, tqdm has facilities to interface with such programs as long as your cpp program can output something. See the readme about it, it works already well for gzipping and other commin Unix commands.

Upvotes: 2

Related Questions