CODEWITHSUNDEEP
machine-learningheuristicshyperparameters
aa1
aa1

Reputation: 803

Difference between hyperparameter and heuristic in machine learning?

What is the difference between hyperparameter and heuristic in the context of machine learning. If you are not learning the parameter and instead deciding it in advance, doesn't that essentially act as a heuristic?

If anyone could provide a clear definition of each and a comparison that would be great! I hope I am not comparing apples to oranges in this case.

Upvotes: 0

Views: 236

Answers (1)

Hristo Vrigazov
Hristo Vrigazov

Reputation: 1407

Interesting thought! The most clear difference between a heuristic and a hyperparameter is that the heuristic is a function which is helpful in search problems, it is used to quickly weigh in different solutions, while a hyperparameter is essentially "hardcoded" in the model, thus does not help the search at all.

However, if you define a heuristic function that just returns the constant hyperparameter, then we can really say that this heuristic function is indeed the same as a hyperparameter. We usually don't view it this way though, since such a heuristic function (that always returns a constant) does not help the search of optimal parameters.

Upvotes: 1

Related Questions