CODEWITHSUNDEEP
python
andrew
andrew

Reputation: 61

Program does not exit. How to find out what python is doing?

I have a python script which is working fine so far. However, my program does not exit properly. I can debug until and I'm returning to the end, but the programm keeps running.

main.main() does a lot of stuff: it downloads (http, ftp, sftp, ...) some csv files from a data provider, converts the data into a standardized file format and loads everyting into the database.

This works fine. However, the program does not exit. How can I find out, where the programm is "waiting"? There exist more than one provider - the script terminates correctly for all providers except for one (sftp download, I'm using paramiko)

if __name__ == "__main__":

    main.log = main.log2both
    filestoconvert = []
    #filestoconvert = glob.glob(r'C:\Data\Feed\ProviderName\download\*.csv')
    main.main(['ProviderName'], ['download', 'convert', 'load'], filestoconvert)

I'm happy for any thoughts and ideas!

Upvotes: 6

Views: 23130

Answers (5)

Eligio Mariño
Eligio Mariño

Reputation: 372

This question is 10 years old, but I post my solution for someone with a similar issue with a non-finishing Python script like mine.

In my case, the debugging process didn't help. All debugging outputs showed only one thread. But the suggestion by @JC Plessis that some work should be going on helped me find the cause.

I was using Selenium with the chrome driver, and I was finishing the selenium process after closing the only tab that was open with

driver.close()

But later, I changed the code to use a headless browser, and the Selenium driver wasn't closed after driver.close(), and the python script was stuck indefinitely. It results that the right way to shutdown the Selenium driver was actually.

driver.quit()

That solved the problem, and the script was finally finishing again.

Upvotes: 3

JC Plessis
JC Plessis

Reputation: 693

If your program does not terminate it most likely means you have a thread still working.

To list all the running threads you can use :

threading.enumerate()

This function lists all Thread that are currently running (see documentation)

If this is not enough you might need a bit of script along with the function (see documentation):

sys._current_frames()

So to print stacktrace of all alive threads you would do something like :

import sys, traceback, threading
thread_names = {t.ident: t.name for t in threading.enumerate()}
for thread_id, frame in sys._current_frames().iteritems():
    print("Thread %s:" % thread_names.get(thread_id, thread_id))
    traceback.print_stack(frame)
    print()

Good luck !

Upvotes: 16

Raf
Raf

Reputation: 146

You'd better use GDB, which allows to pinpoint hung processes, like jstack in Java

Upvotes: 2

rocksportrocker
rocksportrocker

Reputation: 7419

You can involve the python debugger for a script.py with

python -m pdb script.py

You find the pdb commands at http://docs.python.org/library/pdb.html#debugger-commands

Upvotes: 2

Cat Plus Plus
Cat Plus Plus

Reputation: 129764

You can use sys.settrace to pinpoint which function blocks. Then you can use pdb to step through it.

Upvotes: 1

Related Questions