CODEWITHSUNDEEP
pythonpandasgarbage-collection
Jose G
Jose G

Reputation: 343

Garbage collection too slow

I have a function foo() that creates a BIG list (and then converts it to a pandas dataframe)

When I time it, it runs a bit slow, but reasonable:

timeit.Timer('foo()', setup="from __main__ import foo").timeit(number=1)
6.69

However, that's not the time I get during execution, because the garbage collection is disabled. It runs 2x slower, if the garbage collector is enabled:

timeit.Timer('foo()', setup="from __main__ import foo \ngc.enable()").timeit(number=1)
20.31

When is it ok to disable the garbage collector? Should I do so from within foo()?

Upvotes: 0

Views: 326

Answers (1)

user3778137
user3778137

Reputation:

According to python documentation you can disable the collector if you are sure your program does not create reference cycles. If don't want automatic GC to happen during execution of foo. It makes sense to do something like this

def foo():
   gc.disable()
   // Your logic
   gc.enable()

Link: https://docs.python.org/2/library/gc.html

Upvotes: 1

Related Questions