CODEWITHSUNDEEP
pythonh2opredictgam
Geir Inge
Geir Inge

Reputation: 189

H2O GAM - weighted: prediction does not work anymore

If I train a weighted H2O GAM regression model, I can not predict with it. Weighted regression is done using the parameter weights_column

I am running python=3.6.13, h2o=3.32.1.3, pandas=0.25.3, numpy=1.19.5, sklearn=0.24.2. Java Version: openjdk version "14.0.2".

Prediction works with :

I have registrered this as a bug on http://jira.h2o.ai, but I would still be interested if anyone has a way to make it work, without downgrading h2o.

import numpy as np
import pandas as pd
from sklearn.datasets import load_boston
import h2o
from h2o.estimators.gam import H2OGeneralizedAdditiveEstimator

h2o.no_progress()
h2o.init()

np.random.seed(42)
boston = load_boston()
y = pd.Series(boston["target"], name="y")
X = pd.DataFrame(boston["data"], columns=boston["feature_names"])  # shape: (506, 13)
myweight = pd.Series(np.random.random_sample((len(y),)), name="myweight2")

predictors = ['CRIM', 'AGE']
gam_columns = ['CRIM']

params = {
    "family": "gaussian",
    "gam_columns": gam_columns,
    'bs': len(gam_columns) * [0],
}

df0 = pd.concat([y, X, myweight], axis=1)
df = h2o.H2OFrame(python_obj=df0)

model = H2OGeneralizedAdditiveEstimator(**params)
model.train(
    x=predictors,
    y="y",
    weights_column="myweight2",
    training_frame=df,
)

print('df.shape', df.shape)
y_pred = model.predict(df)
print('y_pred:', y_pred.as_data_frame()["predict"].values[0:5])

I get this output. It complains about myweight2:

Checking whether there is an H2O instance running at http://localhost:54321 . connected.
--------------------------  ------------------------------------------

df.shape (506, 15)
Traceback (most recent call last):
  File "/Users/g009655/tmp7/h2otest/test_gam_predict.py", line 37, in <module>
    y_pred = model.predict(df)
  File "/Users/g009655/Library/Caches/pypoetry/virtualenvs/h2otest-S7Xak4Mg-py3.6/lib/python3.6/site-packages/h2o/model/model_base.py", line 237, in predict
    j.poll()
  File "/Users/g009655/Library/Caches/pypoetry/virtualenvs/h2otest-S7Xak4Mg-py3.6/lib/python3.6/site-packages/h2o/job.py", line 80, in poll
    "\n{}".format(self.job_key, self.exception, self.job["stacktrace"]))
OSError: Job with key $03017f00000132d4ffffffff$_9242dd1b28497090cf9ccad52bd54b9f failed with an exception: java.lang.AssertionError:  null vec: $04ff0f000000ffffffff$_b0f0839f8f1a041e8bf5254b552e4dd3; 

name: myweight2

stacktrace: 
java.lang.AssertionError:  null vec: $04ff0f000000ffffffff$_b0f0839f8f1a041e8bf5254b552e4dd3; 

name: myweight2

    at water.fvec.Frame.<init>(Frame.java:161)
    at hex.gam.GAMModel.cleanUpInputFrame(GAMModel.java:505)
    at hex.gam.GAMModel.adaptTestForTrain(GAMModel.java:492)
    at hex.Model.score(Model.java:1697)
    at water.api.ModelMetricsHandler$1.compute2(ModelMetricsHandler.java:422)
    at water.H2O$H2OCountedCompleter.compute(H2O.java:1637)
    at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
    at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
    at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
    at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
    at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

Closing connection _sid_ad95 at exit
H2O session _sid_ad95 closed.

Process finished with exit code 1

Upvotes: 1

Views: 163

Answers (2)

Marcel Bischoff
Marcel Bischoff

Reputation: 116

I get the same error but I found a workaround. For me reloading (from a pandas.DataFrame in my case) the trainings H2OFrame works. Seems like in training it get somehow corrupted...

In your case, try:

df = h2o.H2OFrame(python_obj=df0)
y_pred = model.predict(df)

Upvotes: 0

Erin LeDell
Erin LeDell

Reputation: 8819

Thanks for the bug report. Here's a link to the Jira ticket, for reference.

Upvotes: 2

Related Questions