CODEWITHSUNDEEP
c#unit-testingneural-network
Gabor
Gabor

Reputation: 179

Unit Testing Backpropagation

I'm writing a class library for Backpropagation. I was not following the principles of TDD, but still I want to write some unit test for it. My problem is that I don't know how should I test it. My only guess is to compute the deltas for the weights manually to have something to compare the results to, but that seems not right, or does it?

So: I would compute the deltas in Excel in one scenario, and than test that scenario. Would that be a valid test?

Thanks.

Upvotes: 0

Views: 114

Answers (2)

Houshalter
Houshalter

Reputation: 2798

Testing backpropagation algorithms is really important because it's really common for them to have subtle bugs. E.g. the gradients are off, but close enough the network can still sort of learn.

You just use the finite differences method. For each parameter (or a random parameter or whatever) you add a very small value to it. See how much the error changes divided by the amount you changed the parameter, and see if that matches your gradient.

Upvotes: 3

ultrajohn
ultrajohn

Reputation: 2597

You can start by solving a simple test problem with easily verifiable results such as the XOR problem as described in this document.

Upvotes: 1

Related Questions