CODEWITHSUNDEEP
pythonscikit-learncpu-usage
Damon
Damon

Reputation: 63

How to limit the CPU usage when applying sklearn.Linear_model.LinearRegression?

I need to limit the CPU usage for the following command since it's using 100% of the CPU.

    from sklearn.linear_model import LinearRegression
    model = LinearRegression(fit_intercept = True, n_jobs = 1)
    model.fit(df_x0, df_y0)
    model.predict(df_x1)

I have set n_jobs == 1, and I did not use multiprocessing, but still it kept CPU fully occupied for all kernels and df_y0. ndim == 1, I learned that the n_jobs would not be effective if so.

Can anyone tell me why it's using 100% of the CPU, and how to solve it in python?

Python 3.7, Linux.

Upvotes: 6

Views: 5421

Answers (1)

Mario
Mario

Reputation: 1966

With n_jobs=1 it uses 100% of the CPU of one of the cores. Each process is run in a different core and each process takes the 100% usage of a given core. In Linux with 4 cores It can be clearly seen the CPU usage:

  • (100%,~5%, ~5%, ~5%) when it runs n_jobs=1(if you specify n_jobs to 1, only one core is used).

  • (100%, 100%, 100%, 100%) when running with n_jobs=-1 (if you specify n_jobs to -1, it will use all cores).

also, you can check @kenlukas answer based on his test with scikit-learn 0.20.3 under Linux

Update: To fulfill all scenarios which question Unintended multithreading in Python (scikit-learn) please check out the answers

In case you want to set the number of threads dynamically, and not globally via an environment variable, like:

import mkl
mkl.set_num_threads(2)

Upvotes: 4

Related Questions