CODEWITHSUNDEEP
performancedebuggingoptimizationsasenterprise-guide
Elton Rodrigues
Elton Rodrigues

Reputation: 21

Why real time is much higher than "user" and "system" CPU TIME combined?

We have a batch process that executes every day. This week, a job that usually does not past 18 minutes of execution time (real time, as you can see), now is taking more than 45 minutes to finish.

enter image description here

Fullstimmer option is already active, but we don't know why only the real time was increased.

In old documentation there are Fullstimmer stats that could help identify the problem but they do not appear in batch log. (The stats are those down below: Page Faults, Context Switches, Block Operation and so on, as you can see)

enter image description here

It might be an I/O issue. Does anyone know how we can identify if it is really an I/O problem or if it could be some other issue (network, for example)?

To be more specific, this is one of the queries that have increased in time dramatically. As you can see, it is reading from a data base (SQL Server, VAULT schema) and work and writing in work directory.

enter image description here

Number of observations its almost the same:

enter image description here

We asked customer about any change in network traffic, and they said still the same.

Thanks in advance.

Upvotes: 0

Views: 1091

Answers (1)

Dirk Horsten
Dirk Horsten

Reputation: 3845

For a process to complete, much more needs to be done than the actual calculations on the CPU.

  • Your data has te be read and your results have to be written.
  • You might have to wait for other processes to finish first, and if your process includes multiple steps, writing to and reading from disk each time, you will have to wait for the CPU each time too.

In our situation, if real time is much larger than cpu time, we usually see much trafic to our Network File System (nfs).

As a programmer, you might notice that storing intermediate results in WORK is more efficient then on remote libraries.

You might safe much time by creating intermediate results as views instead of tables, IF you only use them once. That is not only possible in SQL, but also in data steps like this

data MY_RESULT / view=MY_RESULT;
    set MY_DATA;
    where transaction_date between '1jan2022'd and 30jun2022'd;
run;

Upvotes: 2

Related Questions