CODEWITHSUNDEEP
linuxperformanceprofiling
Dun Peal
Dun Peal

Reputation: 17739

Profiling resource usage - CPU, memory, hard-drive - of a long-running process on Linux?

We have a process that takes about 20 hours to run on our Linux box. We would like to make it faster, and as a first step need to identify bottlenecks. What is our best option to do so?

I am thinking of sampling the process's CPU, RAM, and disk usage every N seconds. So unless you have other suggestions, my specific questions would be:

  1. How much should N be?
  2. Which tool can provide accurate readings of these stats, with minimal interference or disruption from the fact that the tool itself is running?
  3. Any other tips, nuggets of wisdom, or references to other helpful documents would be appreciated, since this seems to be one of these tasks where you can make a lot of time-consuming mistakes and false-starts as a newbie.

Upvotes: 0

Views: 970

Answers (2)

Nachiket Kate
Nachiket Kate

Reputation: 8571

First of all, what you want and what you are asking is completely different.

Monitoring is required when you are running it for first time i.e. when you don't know its resource utilization (CPU, Memory,Disk etc.). You can follow below procedure to drill down the bottleneck,

  1. Monitor system resources (Generally 10-20 seconds interval should be fine with Munin, ganglia or other tool). In this you should be able to identify if your hw is bottleneck or not i.e are you running out of resources Ex. 100% cpu util, very low memory, high io etc.

If this your case then probably think about upgrading hw or tuning the existing.

  1. Then you tune your application/utility. Use profilers/loggers to find out which method, process is taking time. Try to tune that process. If you have single threaded codes then probably use parallelism. If DB etc. are involved try to tune your queries, DB params.

Then again run test with monitoring to drill down more :)

Upvotes: 2

berthni
berthni

Reputation: 192

I think a graph representation should be helpful for solving your problem and i advice you Munin.

It's a resource monitoring tool with a web interface. By default it monitors disk IO, memory, cpu, load average, network usage... It's light and easy to install. It's also easy to develop your own plugins and set alert thresholds.

http://munin-monitoring.org/

Here is an example of what you can get from Munin : http://demo.munin-monitoring.org/munin-monitoring.org/demo.munin-monitoring.org/

Upvotes: 0

Related Questions