CODEWITHSUNDEEP
rdeep-learningh2o
Ingolifs
Ingolifs

Reputation: 311

R h2o: examining the nodes on a deeplearning neuralnet model

I would like to be able to examine the node structure on my neural networks. Specifically, I use L1 and L2 regularisation and would like to know what proportion of my weights have atrophied to zero. Does my trained neuralnet use every single node, or can I get away with using much fewer hidden nodes? That sort of thing.

Here's a toy problem:

library(h2o)
h2o.init()

x <-seq(-10,10,by=0.0002)
y <- dnorm(x,sd=2)*sin(2*x)     # The function the neuralnet will attempt to fit
plot(x,y,type="l")

df <- data.frame(x=x,y=y)
df2 <- as.h2o(df)
model <- h2o.deeplearning(df2,
                x=1,
                y=2,
                hidden=c(200,100,50,10,5),  # way more hidden nodes than it needs
                l1=0.0000001,           # regularisation to reduce the number of unnecessary nodes
                l2=0.0000001,
                activation="Tanh",
                export_weights_and_biases=TRUE)

P <- as.data.frame(h2o.predict(model,df2))

lines(x,P$predict,col=2)
legend("topleft",legend=c("Training data","nn output"),col=c(1,2),lwd=1)

training vs ground truth

Is there a function within h2o that will give me the information on what all the weights are?

(I've tried h2o.weights(), it only appears to give me the first layer)

Failing that, given that the model is a S4 object, what are the ways of inspecting an S4 object?

Bonus question: Is there any ability within h2o for visualising the node structure?

Upvotes: 1

Views: 43

Answers (1)

Erin LeDell
Erin LeDell

Reputation: 8819

The h2o.weights() function returns the first layer weights by default, as an H2OFrame. You can get an arbitrary layer by changing the matrix_id argument.

Some examples:

> h2o.weights(model)
           x
1  0.3520632
2  0.5535296
3 -0.4793063
4  0.3828013
5 -0.3253765
6  0.7234487

[200 rows x 1 column]


> h2o.weights(model, matrix_id = 5)
          C1        C2         C3         C4         C5         C6          C7          C8         C9         C10
1  0.7233770 0.7793967 -1.4486042 -0.8187707  0.8667952  1.0290394  0.26213858  0.02487412  0.3342296  0.39383927
2  0.4528885 0.2434976  0.5963052  0.9640941 -0.4480562 -0.1976745 -0.63002998  0.17428128 -0.9241131  0.13199258
3 -0.5477357 0.4918589 -0.7991062 -0.6445754  0.3618000  0.1324274  0.60856968 -0.35876906 -0.0655041  0.21673690
4 -0.3147219 0.2541574 -0.5886489 -0.9993418  0.3042635  0.4107490 -0.08639368 -1.11863077  0.8755589 -0.06117416
5 -0.7028044 0.4625969 -0.3838535 -0.6484048 -0.6975272  0.2663548 -0.17953268  0.14127040 -0.6482394 -0.04426440



> hidden <- c(200,100,50,10,5)
> for (i in 1:(length(hidden) + 1)) {
+   print(dim(h2o.weights(model, matrix_id = i)))
+ }
[1] 200   1
[1] 100 200
[1]  50 100
[1] 10 50
[1]  5 10
[1] 1 5

Upvotes: 1

Related Questions