How to add checkpoints to Random Forest Model In PySpark Environment?

Question

I'm using RandomForest.trainClassifier.

Supported Parameter are:

• input
• numClasses
• categoricalFeaturesInfo
• numTrees
• featureSubsetStrategy
• impurity
• maxDepth
• maxBins
• seed

But extra parameters like useNodeIdCache, checkpointDir, and checkpointInterval are available in Scala.

How can be these features be implemented in PySpark?

Answer 1

pyspark.mllib is in the maintenance mode and is longer developed (access to these features won't be added). If you full featured API use pyspark.ml which provides:

• setCacheNodeIds
• setCheckpointInterval