CODEWITHSUNDEEP
pythonmachine-learningscikit-learnneural-networkactivation-function
Harry Salmon
Harry Salmon

Reputation: 189

Is the sklearn MLPRegressor linear activation producing a linear model?

Can anyone help me understand the linear activation option in the sklearn.neural_network.MLPRegressor? From the documentation:

‘identity’, no-op activation, useful to implement linear bottleneck, returns f(x) = x

However a neural network with all linear connections reduces to a linear sum of weights and inputs (a linear model). And there is no option to use different activation functions for different layers of the model. What else is going on with this model which I'm missing or is it just a way to enable your NN to act as a linear model during grid search?

Upvotes: 1

Views: 430

Answers (1)

Laassairi Abdellah
Laassairi Abdellah

Reputation: 805

You're correct, if you take a look at the code here you'll see that they're using the same activation for all the hidden layers. Using the Identity activation would indeed make multiple hidden layers useless, and it'd just act as a plain linear regressor.

Upvotes: 1

Related Questions