How do i take a trained neural network and implement in another system?

Question

I have trained a feedforward neural network in Matlab. Now I have to implement this neural network in C language (or simulate the model in Matlab using mathematical equations, without using direct functions). How do I do that? I know that I have to take the weights and bias and activation function. What else is required?

Answer 1

After some days of searching, I have found the following webpage to be very useful http://ufldl.stanford.edu/tutorial/supervised/MultiLayerNeuralNetworks/

The picture below shows a simple feedforward neural network. Picture taken from the above website.

In this figure, the circles denote the inputs to the network. The circles labeled “+1” are called bias units, and correspond to the intercept term. The leftmost layer of the network is called the input layer, and the rightmost layer the output layer (which, in this example, has only one node). The middle layer of nodes is called the hidden layer, because its values are not observed in the training set. In this example, the neural network has 3 input units (not counting the bias unit), 3 hidden units, and 1 output unit.

The mathematical equations representing this feedforward network are

This neural network has parameters (W,b)=(W^(l),b^(l),W⁽²⁾,b⁽²⁾), where we write W_ij^(l) to denote the parameter (or weight) associated with the connection between unit j in layer l, and unit i in layer l+1. (Note the order of the indices.) Also, b_i^(l) is the bias associated with unit i in layer l+1.

So, from the trained model, as Mido mentioned in his answer, we have to take the input weight matrix which is W⁽¹⁾, the layer weight matrix which is W⁽²⁾, biases, hidden layer transfer function and output layer transfer function. After this, use the above equations to estimate the output h_W,b(x). A popular transfer function used for a regression problem is tan-sigmoid transfer function in the hidden layer and linear transfer function in the output layer.

Those who use Matlab, these links are highly useful

try to simulate neural network in Matlab by myself

Neural network in MATLAB

Programming a Basic Neural Network from scratch in MATLAB

Answer 2

There is no point in representing it as a mathematical function because it won't save you any computations.

Indeed all you need is the weights, biases, activation and your architecture. I'm assuming it is a simple feedforward network as you said, you need to implement some kind of matrix multiplication and addition in C. Also, you'll need to implement the activation function. After that, you're ready to go. Your feed forward NN is ready to be implemented. If the C code will not be used for training, it won't be necessary to implement the backpropagation algorithm in C.

A feedforward layer would be implemented as follows:

Output = Activation_function(Input * weights + bias)

Where,

Input: (1 x number_of_input_parameters_for_this_layer)

Weights: (number_of_input_parameters_for_this_layer x number_of_neurons_for_this_layer)

Bias: (1 x number_of_neurons_for_this_layer)

Output: (1 x number_of_neurons_for_this_layer)

The output of a layer is the input to the next layer.