CODEWITHSUNDEEP
linuxoptimizationprofiling
Will
Will

Reputation: 75625

Profiling very long running tasks

How can you profile a very long running script that is spawning lots of other processes?

We have a job that takes a long time to run - 11 or more hours, sometimes more than 17 - so it runs on a Amazon EC2 instance.

(It is doing cufflinks DNA alignment and stuff.)

The job is executing lots of processes, scripts and utilities and such.

How can we profile it and determine which component parts of the job take the longest time?

A simple CPU utilisation per process per second would probably suffice. How can we obtain it?

Upvotes: 3

Views: 109

Answers (1)

Cédric Julien
Cédric Julien

Reputation: 80761

There are many solutions to your question :

  • munin is a great monitoring tool that can scan almost everything in your system and make nice graph about it :). It is very easy to install and use it.

  • atop could be a simple solution, it can scan cpu, memory, disk regulary and you can store all those informations into files (the -W option), then you'll have to anaylze those file to detect the bottleneck.

  • sar, that can scan more than everything on your system, but a little more hard to interpret (you'll have to make the graph yourself with RRDtool for example)

Upvotes: 3

Related Questions