CODEWITHSUNDEEP
pythonneural-networkpybrainfeed-forward
Shinjitsu
Shinjitsu

Reputation: 639

Pybrain Feedforward neural network training error completely stuck

Hey guys I need a bit of help with my pybrain code. Everything loads fine, but after it trains the first time the training error doesn't go down. In fact, it just stays stuck there at exactly 13.3484055174. I've been checking my code many times and comparing it with other examples, but I consistently get the same problem. I've also already tried changing the number of hidden units, learningrate, momentum, weightdecay to no avail. I've checked the parameters and it starts off from [-1 to 1] then blows up into ~240-250. I was wondering if anyone can see why it's not working. I'm sure it's a really simple 1-liner that I'm missing.

I'm working on the kaggle 0-9 digit classification dataset. I've already gotten the randomforest to work but I really want to make this neural network work too. Any help would get greatly appreciated.

#learn digit classification with a nerual network

import pybrain
from pybrain.datasets import *
from pybrain.tools.shortcuts import buildNetwork
from pybrain.supervised.trainers import BackpropTrainer
from pybrain.structure.modules   import SoftmaxLayer
from pybrain.utilities           import percentError
import numpy

print "Importing training and test data"
data = numpy.genfromtxt('trainR.csv', delimiter = ',')
data = data[1:]
traindata = data[:(len(data)/2)]
testdata = data[(len(data)/2)+1:]

print "Importing actual data"
actualdata = numpy.genfromtxt('trainR.csv', delimiter = ',')

print "Adding samples to dataset and setting up neural network"
ds = ClassificationDataSet(784, 10, nb_classes = 10)
for x in traindata:
    ds.addSample(tuple(x[1:]),tuple(x[0:1]))
ds._convertToOneOfMany( bounds=[0,1] )
net = buildNetwork(784, 100, 10, bias=True, outclass=SoftmaxLayer)

print "Training the neural network"
trainer = BackpropTrainer(net, dataset=ds, momentum = 0.1,
                    verbose = True, weightdecay = 0.01)
for i in range(3):
    # train the network for 1 epoch
    trainer.trainEpochs( 1 )

    # evaluate the result on the training and test data
    trnresult = percentError( trainer.testOnClassData(), [x[0] for x in traindata] )

    # print the result
    print "epoch: " + str(trainer.totalepochs) + "  train error: " + str(trnresult)

print ""
print "Predicting with the neural network"
answerlist = []
for row in testdata:
    answer = numpy.argmax(net.activate(row[1:]))
    answerlist.append(answer)
tstresult = percentError(answerlist, [x[0] for x in testdata])
print "Test error: " + str(tstresult)

Upvotes: 2

Views: 2957

Answers (1)

david_adler
david_adler

Reputation: 10902

try changing

 ds = ClassificationDataSet(784, 10, nb_classes = 10)

to

ds = ClassificationDataSet(784, 1, nb_classes = 10)

i think ClassificationDataSet second argument is the dimensions of the targets rather than the number of classes this is given by nb_classes. It depends on how your data is organized. Best thing is to enter each target as an integer for each class and then use _convertToOneOfMany()

It would be useful if you provided your first sample

Upvotes: 3

Related Questions