CODEWITHSUNDEEP
machine-learningdata-miningwekadecision-treeclassification
stackoverflowuser2010
stackoverflowuser2010

Reputation: 40909

How to deal with missing attribute values in C4.5 (J48) decision tree?

What's the best way to handle missing feature attribute values with Weka's C4.5 (J48) decision tree? The problem of missing values occurs during both training and classification.

  1. If values are missing from training instances, am I correct in assuming that I place a '?' value for the feature?

  2. Suppose that I am able to successfully build the decision tree and then create my own tree code in C++ or Java from Weka's tree structure. During classification time, if I am trying to classify a new instance, what value do I put for features that have missing values? How would I descend the tree past a decision node for which I have an unknown value?

Would using Naive Bayes be better for handling missing values? I would just assign a very small non-zero probability for them, right?

Upvotes: 6

Views: 8464

Answers (2)

nair.ashvin
nair.ashvin

Reputation: 801

An alternative approach is to leave the missing value as the '?', and not use it for the information gain calculation. No node should have an unknown value during classification because you ignored it during the information gain step. For classifying, I believe you simply consider the missing value unknown and do not delete it during classification on that specific attribute.

Upvotes: 2

greeness
greeness

Reputation: 16104

From Pedro Domingos' ML course in University of Washington:

Here are three approaches what Pedro suggests for missing value of A:

  • Assign most common value of A among other examples sorted to node n
  • Assign most common value of A among other examples with same target value
  • Assign probability p_i to each possible value v_i of A; Assign fraction p_i of example to each descendant in tree.

The slides and video is now viewable at here.

Upvotes: 9

Related Questions