CODEWITHSUNDEEP
pythonfor-loopoptimization
vlad.rad
vlad.rad

Reputation: 1083

Python - is packing everything in a for loop into a function a good idea?

This question is not about opinion, it's about understanding of some mechanisms of Python.

Let's take the following code (not real one, just a dummy code):

for i in range(1000):
   x = 10 + i*10
   y = some_calculations(x)
   z = send_data_to_db(y)

Does it make sense in terms of performance to pack the actions I do with x, y and z into one function? Like this:

def handle_everything(a):
   x = 10 + a*10
   y = some_calculations(x)
   z = send_data_to_db(y)
   return z 

for i in range(1000):
   z = handle_everything(i)

The reason I am asking is the following. In this article the author describes the mechanisms of importing a module in Python.

Based on this, I came to conclusion, that for functions that are going to be reused, Python doesn't make compilation again. And therefore I thought, that maybe it makes sense to pack all the actions inside of a loop into functions / one function. After I implemented this modification, my measurements showed, that the execution time wasn't affected much.

Please let me know, if understood something wrong.

Upvotes: 0

Views: 414

Answers (1)

Reedinationer
Reedinationer

Reputation: 5774

In general, when questioning the python ideals I find it best to open an interactive session and run:

>>>import this

Which prints (and comes standard with python)

"""
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
"""

Where I think the most applicable parts for your question are:

  • Simple is better than complex. Not using a function is probably simpler, in that you won't need to scroll back and forth through your program, and your code is a couple lines shorter.

  • Readability counts. This would depend on the way the rest of your script is set up. If your for loop will get a lot more added, then a function would probably make it a lot easier to read. Typically a main() function does nothing more than call functions I've defined in my scripts -> my main program flow is very readable.

You should note "Achieve the best execution time" is not mentioned. This should be (mostly) an afterthought once you have a working script. At that point you can time how long your program spends in each function and focus on optimizing the worst time losses! To optimize for speed before your script is done will not be worth the hassle in 95% of use cases. This stems from the fact you don't know where you are losing time! Say one part of your script takes 1 second to execute, and another takes 100 seconds. If you try to optimize without a working script you may try to speed up the thing that already only takes 1 second...Say you cut 50% time, well who cares about .5 seconds. The same 50% speed increase in the other section of your code would save you 50 seconds though!!! This is why you need to target your optimization on the slowest segments first, which you can't identify until your script is finished anyways...

Upvotes: 1

Related Questions