CODEWITHSUNDEEP
python
FrozenHeart
FrozenHeart

Reputation: 20756

How to avoid busy waiting for code that has no blocking calls?

What is the best way to avoid enormous CPU usage when writing while True loops which could do nothing in some circumstances? It seems like time.sleep(0) hurt CPU:

import time

msg = []

# in main thread I will add messages to msg list sometimes
msg.append('some_msg')

# in some other thread I will parse them like this:
while True:
  for msg in msgs:
    # do some operations
  time.sleep(0) # hurt CPU in case of len(msg) == 0

Yeah, I can use time.sleep(1) instead of it but is it really the most correct way to do it in Python?

Upvotes: 4

Views: 6079

Answers (2)

Daniel
Daniel

Reputation: 42778

You are using the wrong datatype. Lists are not made for inter-thread communication. Use queues. The get-method is blocking.

Upvotes: 0

anand
anand

Reputation: 1526

Its just a classical producer consumer problem, and you must be seeking a better implementation than you did.

In your case if you are using a positive value for sleep then you have already done saving CPU for other processes to also use it since high CPU usage is good (see Note at the bottom for clarity).

If you are looking for optimization and better performance then you need to tweak your code specific to your application needs, here are two suggestions.

  1. You might use signaling in threads (also call inter-thread communication) using wait and notify calls if you can convert msg to a thread-safe queue; this way you can overcome arbitrary waiting.

  2. If your processing on the msg items (that you perform in for loop) is IO hungry then you can try splitting in independent threads or you can upgrade your code to do async processing.

Note: If you have a high CPU usage then it means you are best using your computing resources, but it can lead to deprived reliability down the line and starvation of resources for other process which might be bad hence try putting priority on your threads/process as per your needs are.

Upvotes: 2

Related Questions