CODEWITHSUNDEEP
rmachine-learningneural-network
Mohammed Khalifa
Mohammed Khalifa

Reputation: 21

Neuralnet package in r with simple structure taking very long time , what is the issue here?

I have a problem with neuralnet function from neuralnet package in R. I designed a simple structure with 82 feature as input and only 1 hidden layer with 10 neurons and output is 20 class and I left this line which represent neuralnet function to run above 4 hours and didn't finish ! This is the code : 

nn=neuralnet(f, data = train, hidden = 10, err.fct = "sse",threshold = 1, 
             learningrate=.05,rep = 1, linear.output = FALSE)

Upvotes: 2

Views: 7989

Answers (2)

agenis
agenis

Reputation: 8377

You can easily predict the computation time and complexity of your code before running it on the full data with the GuessCompx package.

Create fake data with the same characteristics as yours, and 20-class Y vector and a wrapper function:

train = data.frame(matrix(rnorm(300000*82, 3), ncol=82))
train['Y'] = as.character(round(runif(300000, 1,20)))
nn_test = function(data) {
  nn=neuralnet(formula=Y~., data=data, hidden = 10, err.fct = "sse",threshold = 1, 
            learningrate=.05,rep = 1, linear.output = FALSE)
}

And then do the audit:

library(GuessCompx) # get it by running: install.packages("GuessCompx")
library(neuralnet)
CompEst(train, nn_test)
#### $`TIME COMPLEXITY RESULTS`$best.model
#### [1] "NLOGN"
#### $`TIME COMPLEXITY RESULTS`$computation.time.on.full.dataset
#### [1] "1M 4.86S"
#### $`MEMORY COMPLEXITY RESULTS`$best.model
#### [1] "LINEAR"
#### $`MEMORY COMPLEXITY RESULTS`$memory.usage.on.full.dataset
#### [1] "55535 Mb"
#### $`MEMORY COMPLEXITY RESULTS`$system.memory.limit
#### [1] "16282 Mb"

See that the computation time is not a problem, but the memory usage and limitations might be impacting your computer, causing the long delay? The only nn output object takes more than 4Gb to be stored!

Upvotes: 1

lejlot
lejlot

Reputation: 66805

Training of the neural network can be arbitrary long, what affects this time?

  • Complexity of the network (not a problem here as your network is quite small)
  • Size of the training data - even few thousands of samples can take quite a while, furthermore number of features also significantly increase computation time
  • Training algorithm and its hyperparameters - in particular for SGD based solutions - too small learning rate (or to big as it causes the oscilation)
  • Type of stopping criterion - there are many ways of checking whether to stop training a NN, some more expensive (validation score) than others (amplitude of gradient/number of epochs).

In your particular example your training takes at most 100,000 steps and you use rprop+ learning. Thus the most probable problem is the size of the training data. You can try to set stepmax to some much smaller value to see how much time it needs and how good is the model.

In general - neural networks are hard and slow to train, you have to deal with it or switch to other models.

Upvotes: 2

Related Questions