CODEWITHSUNDEEP
pythonpython-multithreading
setevoy
setevoy

Reputation: 4662

CPU usage per thread

I need to get CPU % for each process thread.

So, I create simple script:

import psutil
from psutil import Process
p = psutil.Process(4499)

treads_list = p.get_threads()

for i in treads_list:
    o = i[0]
    th = psutil.Process(o)
    cpu_perc = th.get_cpu_percent(interval=1)
    print('PID %s use %% CPU = %s' % (o, cpu_perc))

Here is how TOP looks like for this process:

 4942 teamcity  20   0 3288m 831m 3124 R 33.3 10.6  10303:37 java
32700 teamcity  20   0 3288m 831m 3124 S  5.9 10.6  18:49.99 java
 5824 teamcity  20   0 3288m 831m 3124 S  5.9 10.6   1:57.90 java
 4621 teamcity  20   0 3288m 831m 3124 S  3.0 10.6   1834:09 java
 4622 teamcity  20   0 3288m 831m 3124 S  2.6 10.6   1844:15 java

Threads use 2.6-5.9 % CPU, and parent PID - use 33.3.

But - here is script's result:

# ./psutil_threads.py
PID 10231 use % CPU = 60.9
PID 10681 use % CPU = 75.3
PID 11371 use % CPU = 69.9
PID 11860 use % CPU = 85.9
PID 12977 use % CPU = 56.0
PID 14114 use % CPU = 88.8

Looks like each thread 'eat' 56-88 % CPU...

What I'm missing here?

Upvotes: 7

Views: 12581

Answers (5)

Gabe
Gabe

Reputation: 897

This should give you what you need and match top (adapt to your use case):

import psutil

def get_threads_cpu_percent(p, interval=0.1):
   total_percent = p.cpu_percent(interval)
   total_time = sum(p.cpu_times())
   return [total_percent * ((t.system_time + t.user_time)/total_time) for t in p.threads()]

# Example usage for process with process id 8008:
proc = psutil.Process(8008)
print(get_threads_cpu_percent(proc))

Upvotes: 7

GalacticOwl
GalacticOwl

Reputation: 1

These other answers are a little bit misleading in that the solutions provided will not match top output. If you want to track the CPU usage that is displayed in a manner similarly to top you have to run a loop that will do the following...

If I have thread a and thread b under process P.

# Log the initial total time of process P: 
```
proc = psutil.Process(<your pid>)
initial_total_time = sum(proc.cpu_times())
```

# log the initial time of each thread
```

initial_thread_times = {'a': {'system': None, 'user': None}}

for thread in proc.threads():

    initial_thread_times[psutil.Process(thread.id).name()]['system'] = thread.system_time

    initial_thread_times[psutil.Process(thread.id).name()]['user'] = thread.user_time
```

# Determine total percent cpu usage over an interval
`total_cpu_percent = proc.cpu_percent(interval = 0.1)`
# grab the new total amount of time the process has used the cpu
`final_total_time = sum(proc.cpu_times())`
# grab the new system and user times for each thread
```
final_thread_times = {'a': {'system': None, 'user': None}}
for thread in proc.threads():
    final_thread_times[psutil.Process(thread.id).name()]['system'] = thread.system_time
    final_thread_times[psutil.Process(thread.id).name()]['user'] = thread.user_time
```
# calculate how much cpu each thread used by...
```
total_time_thread_a_used_cpu_over_time_interval = ((final_thread_times['a']['system']-initial_thread_times['a']['system']) + (final_thread_times['a']['user']-initial_thread_times['a']['user']))
total_time_process_used_cpu_over_interval = final_total_time - initial_total_time

percent_of_cpu_usage_utilized_by_thread_a = total_cpu_percent*(total_time_thread_a_used_cpu_over_time_interval/total_time_process_used_cpu_over_interval)

And if you iterate over this process you can dynamically report and calculate this like top

Upvotes: 0

Gotenks
Gotenks

Reputation: 377

I made improvements to Florent Thiery and Gabe solution, creating a little script you can use to monitor CPU usage (by thread) of any process.

python cpuusage.py <PID>

import psutil, sys, time, os

def clear():
    if os.name == "nt":
        _ = os.system("cls")
    else:
        _ = os.system("clear")

def get_threads_cpu_percent(p, interval=0.1):
   total_percent = p.cpu_percent(interval)
   total_time = sum(p.cpu_times())
   return [('%s %s %s' % (total_percent * ((t.system_time + t.user_time)/total_time), t.id, psutil.Process(t.id).name())) for t in p.threads()]

try:
    sys.argv[1]
except:
    sys.exit('Enter PID')

proc = psutil.Process(int(sys.argv[1]))

while True:
    clear()
    threads = get_threads_cpu_percent(proc)
    threads.sort(reverse=True)
    for line in threads:
       print(line)
    time.sleep(1)

Upvotes: 2

Florent Thiery
Florent Thiery

Reputation: 389

While Gabe's answer is great, note that newer psutil version requires the following updated syntax:

import psutil

def get_threads_cpu_percent(p, interval=0.1):
   total_percent = p.cpu_percent(interval)
   total_time = sum(p.cpu_times())
   return [total_percent * ((t.system_time + t.user_time)/total_time) for t in p.threads()]

# Example usage for process with process id 8008:
proc = psutil.Process(8008)
print(get_threads_cpu_percent(proc))

Upvotes: 4

dom0
dom0

Reputation: 7486

get_cpu_percent(interval=0.1)

Return a float representing the process CPU utilization as a percentage.

When interval is > 0.0 compares process times to system CPU times elapsed before and after the interval (blocking).

When interval is 0.0 or None compares process times to system CPU times elapsed since last call, returning immediately. In this case is recommended for accuracy that this function be called with at least 0.1 seconds between calls.

This sounds a lot like it will give you how much of the CPU time spent non-idle is returned (that is: amount of process CPU time per system CPU time), while top shows the amount of CPU time of the process in relation to real time. This seems realistic given your numbers.

To get the values top would show you, simply multiplying each threads' CPU usage by the CPU usage of the core the thread runs on should work. psutil.cpu_percent should help with that. Note that you need to divide percentages by 100.0 (to get a "percentage" between 0 and 1) before multiplying them.

Upvotes: 4

Related Questions