h2o.deeplearning predict error in R

Question

DNN was performed using the h2o.deeplearning function.

Finally, I used the h2o.predict function to perform test data prediction.

But when I try to display the actual values ​​and the predicted values ​​in a visual way, I got an error. Here is my code:

library("h2o")
h2o.init(nthreads = -1, max_mem_size = "5G")

credit<-read.csv("http://freakonometrics.free.fr/german_credit.csv", header=TRUE)
F=c(1,2,4,5,7,8,9,10,11,12,13,15,16,17,18,19,20,21)
for(i in F) credit[,i]=as.factor(credit[,i])
str(credit)

library(caret)
set.seed(1000)
intrain<-createDataPartition(y=credit$Creditability, p=0.7, list=FALSE) 
train<-credit[intrain, ]
test<-credit[-intrain, ]


deep_train<-as.h2o(train,destination_frame = "deep_train")
deep_test<-as.h2o(test,destination_frame = "deep_test")


h2o.str(deep_train)
h2o.str(deep_test)

x<-names(train[,-1])
y<-"Creditability"

deep_model<-h2o.deeplearning(x=x, y=y,
                             training_frame = deep_train,
                             activation = "RectifierWithDropout",
                             hidden=c(30,40,50),
                             epochs = 10,
                             input_dropout_ratio = 0.2,
                             hidden_dropout_ratios = c(0.5,0.5,0.5),
                             l1=1e-5 ,l2= 0,
                             rho = 0.99, epsilon = 1e-08,
                             loss = "CrossEntropy",
                             variable_importances = TRUE)



pred<-h2o.predict(deep_model, newdata=deep_test)

confusionMatrix(pred$predict, test$Creditability)
Error in unique.default(x, nmax = nmax) : 
  invalid type/length (environment/0) in vector allocation

How to visualize the predict table??

Answer 1

The pred object is an H2OFrame.

> class(pred)
[1] "H2OFrame"
> head(pred)
  predict        p0        p1
1       1 0.1776320 0.8223680
2       1 0.1959193 0.8040807
3       1 0.2143592 0.7856408
4       1 0.1561238 0.8438762
5       1 0.1461881 0.8538119
6       0 0.2978314 0.7021686

The confusionMatrix() function is from the caret package, which does not know what to do with an H2OFrame object -- this is the cause of the error. The caret::confusionMatrix() function expects the first argument to be vector in R of class, "factor".

If you want to use the caret::ConfusionMatrix() function, then you just need to convert the pred object to the right format (which requires bringing it from H2O Cluster memory into R memory and then converting it to a factor).

> confusionMatrix(as.factor(as.data.frame(pred$predict)[,1]), test$Creditability)

Confusion Matrix and Statistics

          Reference
Prediction   0   1
         0  13  10
         1  77 200

               Accuracy : 0.71            
                 95% CI : (0.6551, 0.7607)
    No Information Rate : 0.7             
    P-Value [Acc > NIR] : 0.3793          

                  Kappa : 0.123           
 Mcnemar's Test P-Value : 1.484e-12       

            Sensitivity : 0.14444         
            Specificity : 0.95238         
         Pos Pred Value : 0.56522         
         Neg Pred Value : 0.72202         
             Prevalence : 0.30000         
         Detection Rate : 0.04333         
   Detection Prevalence : 0.07667         
      Balanced Accuracy : 0.54841         

       'Positive' Class : 0               

Alternatively, you can use the h2o.confusionMatrix() function on the deep_model object directly.