CODEWITHSUNDEEP
pythondebuggingpycharmpickle
user2986978
user2986978

Reputation: 163

PyCharm IDE is very slow on debug Python reading small pickle file

Debug takes sometimes 10 seconds. The problem starts after reading 30mega + 5 mega pickles, with with pdb debugging is fine. Even on lines of 'print'

any suggestions?

if __name__ == "__main__":
    print 'loading files...',
    with open(fname1, 'rb') as handle: items = pickle.load(handle)
    with open(fname2, 'rb') as handle: sentences_by_id= pickle.load(handle)
    print ' done!'

Upvotes: 13

Views: 12305

Answers (6)

Pinkponk
Pinkponk

Reputation: 21

What help for me was to change the "variable loading policy" to "on demand". This is done by clicking the small cog wheel in the debug menu:

Upvotes: 2

Roi
Roi

Reputation: 121

Try enabling Gevent, or add this to the start of the file:

np.set_printoptions(threshold=5)

Can't tell why it helped - I didn't change anything in my code, including usage of gevents, when the issue started.

Upvotes: 3

Mike
Mike

Reputation: 33

this happened to me (PyCharm 2019.3) and it was caused by invalid breakpoints that had been placed on files subsequently deleted, edited etc.  Delete all breakpoints and make sure you haven't got a breakpoint on Exception "raise".  Once I did that PyCharm debugging went back to normal speed

Upvotes: 0

Aniruddha Kalburgi
Aniruddha Kalburgi

Reputation: 393

I did some analysis on this, also changed my IDE to Elipse using PyDev plugin.

Still debugging python program was way too slower than the actual directly executing python program.

How much slower? It took my computationally intensive program more than 17 minutes and still no solution where when run directly, the same program returned output within 2 minutes.

I guess the problem is with Python Debugger, not with either of IDE's Eclipse or Jetbrains Pycharm. I can vouch for both as I have worked on both for many years.

It's worth checking if there's any explicit debugger speed setting option available for Python Debugger.

Upvotes: 0

user8837131
user8837131

Reputation: 141

I had a similar problem with a dict of lists of ndarrays. when the list was too long, the pycharm debugger would freeze, top would show cpu for the pycharm at 102% ( I guess several cores were active) and it would take a very long time to unfreeze. Running without breakpoints was fine. My solution was to convert all my lists on ndarrays to ndarrays, e.g

<pseudo python>
for k in dict_of_lists_of_ndarrays:
     dict_of_lists_of_ndarrays[k]=np.array(dict_of_lists_of_ndarrays[k])
</pseudo python>

now things are much improved. I have no idea what the issue was

Upvotes: 4

flipajs
flipajs

Reputation: 56

I found this thread on JetBrains support, which is suggesting to turn off "Show return values" option (you can find it in the settings of the Debug tool bar, the gear icon). It has solved it for me.

Upvotes: 0

Related Questions