CODEWITHSUNDEEP
machine-learningterminologyactivation-function
Gowdhaman008
Gowdhaman008

Reputation: 1323

Difference between Cost Function and Activation funtion?

I would like to understand the difference between Cost function and Activation function in a machine learning problems.

Can you please help me understand the difference?

Upvotes: 1

Views: 1334

Answers (1)

Skander HR
Skander HR

Reputation: 620

  • A cost function is a measure of error between what value your model predicts and what the value actually is. For example, say we wish to predict the value yi for data point xi. Let fθ(xi) represent the prediction or output of some arbitrary model for the point xi with parameters θ. The cost function is the sum of (yi−fθ(xi))2 (this is only an example it could be the absolute value over the square). Training the hypothetical model we stated above would be the process of finding the θ that minimizes this sum.

  • An activation function transforms the shape/representation of the in the model. A simple example could be max(0,x), a function which outputs 0 if the input x is negative or x if the input x is positive. This function is known as the “Rectified Linear Unit” (ReLU) activation function. These representations are essential for making high-dimensional data linearly separable, which is one of the many uses of a neural network. The choice of these functions depends on your case if you need a custom model also the kind of layer (hidden / output) or the model architecture.

Upvotes: 3

Related Questions