CODEWITHSUNDEEP
machine-learning
Mani
Mani

Reputation: 135

What is meant by model architecture and training parameters.?

I am working on some machine learning methods such as Support Vector Machine, K-Nearest Neighbor and Isolation Forest in Python but I have a question about it that what is meant by model architecture and training parameters about these methods?

Upvotes: 0

Views: 372

Answers (2)

tzinie
tzinie

Reputation: 772

I would suggest you use the term Model Architecture when you are referring to Neural Networks. You can think of Model Architecture like Lego Blocks. There are different sizes (number of units in each hidden layer) and colours (different layers: LSTM, Convolutional, Linear/Dense etc.), which can be used to build any simple to complex architecture that you wish.

The Machine Learning models you mentioned: SVM, k-NN, Isolation Forest are supervised algorithms that allow you to perform classification or regression. There is no architecture here. Nevertheless, each algorithm takes a different approach to tackle the problem. For example, SVM aims to find a hyperplane in an N-dimensional space (N = number of features) that distinctly classifies the data points, while k-NN classifies a datapoint by a plurality vote of its k neighbours.

There are 2 kinds of parameters: model parameters and hyper-parameters. The former are learned during the training phase, the latter must be set before the training phase. If the quality of your data is good and your problem is not very complex then you will learn during the training phase the correct model parameters thanks to an optimization algorithm. However, hyper-parameters are problem-specific and differ between algorithms. For example, in SVM one of the most important hyper-parameters is C, which is the penalty parameter of the error term. It controls the trade off between smooth decision boundary and classifying the training points correctly.

Generally, it is not a good practice to choose the default hyper-parameters, therefore, I would suggest you perform hyper-parameter tuning. Here is a link to get you started.

Upvotes: 1

Tdoggo
Tdoggo

Reputation: 419

Good question. Model architecture is just like building architecture: which blocks did you put together to make the thing? For instance, did you make a neural network with 5 layers or 2 layers? How many neurons in each layer?

Training parameters are like slight tweaks to how the machine learns. For instance, how many branches should your tree have? And how many times can each branch split?

For a neural network, you could specify how long you want it to train.

Upvotes: 0

Related Questions