Python GPU Resource Utilization

Question

I have a Python script running inference on some deep learning models.

Is there any way I can find out the GPU resource utilization levels? For instance, utilization of shaders, float16 multipliers, etc.

I can't seem to find much documentation online regarding those GPU resources.

Thank you!

Answer 1

Yes, you can do this with pynvml and if you are using jupyterlab you could use nvdashboard. A recent medium post details the usage of these libraries: https://medium.com/rapids-ai/gpu-dashboards-in-jupyter-lab-757b17aae1d5

Answer 2

You can try to run your pyxthon-application in a GPU-Profiler like Renderdoc. It will analyse you your run. You will be able to get information about th eused resources, used buffers, output on the different renderstates etc.

Just check Renderdoc-website to learn more about it.