CODEWITHSUNDEEP
machine-learningneural-network
nbecker
nbecker

Reputation: 1729

neural net with external nonlinear function on output

I am interested in trying NN in a perhaps unusual setting.

The input to the NN is a vector. The output is also a vector. However, the training data and error is not computed directly on this output vector, but is a (nonlinear) function of this output vector. So at each epoch, I need to activate the NN, find an output vector, apply this to my (external) nonlinear function to compute a new output vector. However, this new output vector is of length 1 and the error is computed based on just this single output.

Some questions:

  1. Is this something that NN might usefully do?
  2. Is this a structure that is well-known already?
  3. Any ideas how to approach this?

Upvotes: 3

Views: 55

Answers (1)

Fred Foo
Fred Foo

Reputation: 363787

  1. In principle, yes.

  2. Yes, this is what a softmax unit does. It takes the activations at the output layer and computes a single value from them, which is then used to compute the error.

  3. You need to know the partial derivative of your multivariate function (let's call it f). From there, you can use the chain rule to compute the derivative of the error in the parameters of f and backpropagate the error derivative.

Upvotes: 3

Related Questions