CODEWITHSUNDEEP
pythondebugging
lllllllllllll
lllllllllllll

Reputation: 9110

Debug a Python program which seems paused for no reason

I am writing a Python program to analyze log files. So basically I have about 30000 medium-size log files and my Python script is designed to perform some simple (line-by-line) analysis of each log file. Roughly it takes less than 5 seconds to process one file.

So once I set up the processing, I just left it there and after about 14 hours when I came back, my Python script simply paused right after analyzing one log file; seems that it hasn't written into the file system for the analyzing output of this file, and that's it. No more proceeding.

I checked the memory usage, it seems fine (less than 1G), I also tried to write to the file system (touch test), it also works as normal. So my question is that, how should I proceed to debug the issue? Could anyone share some thoughts on that? I hope this is not too general. Thanks.

Upvotes: 0

Views: 345

Answers (2)

krizex
krizex

Reputation: 349

Try this tool https://github.com/khamidou/lptrace with command:

sudo python lptrace -p <process_id>

It will print every python function your program invokes and may help you understand where your program stucks or in an infinity loop.

If it does not output anything, that's proberbly your program get stucks, so try 

pstack <process_id>

to check the stack trace and find out where stucks. The output of pstack is c frames, but I believe somehow you can find something useful to solve your problem.

Upvotes: 1

A. STEFANI
A. STEFANI

Reputation: 6737

You may use Trace or track Python statement execution and/or The Python Debugger module.

Upvotes: 3

Related Questions