R - Cannot change NAs in Data Frame to numeric

Question

I have a data frame of values called "games" with several columns of numerics. The original csv file had some missing values, which became NAs when I read them in. I'm trying to replace these NAs with the row median (already stored as a column of the data frame). I can't get the original NA to coerce from a character to a numeric.

I first found the indices of the missing values.

ng <- which(is.na(games), arr.ind = TRUE)

Then I tried replacing the NAs with a value from the column "linemedian".

games[ng] <- games[ng[,1], "linemedian"]
games[ng]
[1] " -3.25" "  9.98" " -9.1"  " -9.1"  " 14.0"  " -3.25" "  9.98" " -3.25" "  9.98" "  2.30" " 13.75" "-24.00" "  3.71" " 15.94" " 14.25" " -9.83" " 13.75" " -4.88"

Replacing the NAs with just any number also did not work.

games[is.na(games)] <- 0
[1] "  0.0"  "  0.0"  "  0"    "  0"    "  0"    "  0.0"  "  0.0"  "  0.0"  "  0.0"  "  0.00" "  0.00" "  0.00" "  0"    "  0"    "  0.00" "  0.00" "  0.00" "  0.00"

I thought that removing the whitespace might change the outcome but it did not.

games[ng] <- as.numeric(trimws(games[ng[,1], "linemedian"]))
[1] "-3.25" "9.98"  "-9.1"  "-9.1"  "14"    "-3.25" "9.98"  "-3.25" "9.98"  "2.3"   "13.75" "-24"   "3.71"  "15.94" "14.25" "-9.83" "13.75" "-4.88"

Other attempts that did not work:

games[ng] <- type.convert(games[ng]) # using type.convert()

games[, -c(1,2)] <- as.numeric(games[, -c(1,2)]) # first two columns are metadata
Error: (list) object cannot be coerced to type 'double'

games[, -c(1,2)] <- as.numeric(unlist(games[, -c(1,2)]))    

games[ng] <- as.numeric(as.character(trimws(games[ng[,1], "linemedian"])))

# New Addition from Answer
games[, sapply(games, is.numeric)][ng] <- games[, sapply(games, is.numeric)][ng[,1], "linemedian"]

I know for sure that the value I'm assigning to games[ng] is a numeric.

games[ng[,1], "linemedian"]
[1]  -3.25   9.98  -9.10  -9.10  14.00  -3.25   9.98  -3.25   9.98   2.30  13.75 -24.00   3.71  15.94  14.25  -9.83  13.75  -4.88
typeof(games[ng[,1], "linemedian"])
[1] "double"

Everywhere I look on the Stack Overflow boards, the obvious answer should be games[is.na(games)] <- VALUE. But that isn't working. Anybody have some idea?

Here's the full code if you want to replicate:

## Download Raw Files

download.file("http://www.thepredictiontracker.com/ncaa2016.csv",
          "data/ncaa2016.csv")

download.file("http://www.thepredictiontracker.com/ncaapredictions.csv",
          "data/ncaapredictions.csv")

## Create Training and Prediction Data Sets

games <- read.csv("data/ncaa2016.csv", header = TRUE, stringsAsFactors = FALSE, 
              colClasses=c(rep("character",2),rep("numeric",72)))

preds <- read.csv("data/ncaapredictions.csv", header = TRUE, stringsAsFactors = TRUE)
colnames(preds)[colnames(preds) == "linebillings"] <- "linebill"
colnames(preds)[colnames(preds) == "linebillings2"] <- "linebill2"
colnames(preds)[colnames(preds) == "home"] <- "Home"
colnames(preds)[colnames(preds) == "road"] <- "Road"

## Remove Columns with too many missing values

rm <- unique(c(names(games[, sapply(games, function(z) sum(is.na(z))) > 50]), # Games and predictions
           names(preds[, sapply(preds, function(z) sum(is.na(z))) > 10]))) # with missing data

games <- games[, !(names(games) %in% rm)] # Remove games with no prediction data 

preds <- preds[, !(names(preds) %in% rm)] # Remove predictions with no game data 

## Replace NAs with Prediction Median
ng <- which(is.na(games), arr.ind = TRUE)
games[ng] <- games[ng[,1], "linemedian"]

Also, I can't post the entire dput() output, but here's a bit of a the data set just to show the structure.

dput(head(games[1:6]))

structure(list(Home = c("Alabama", "Arizona", "Arkansas", "Arkansas St.", 
"Auburn", "Boston College"), Road = c("USC", "BYU", "Louisiana Tech", 
"Toledo", "Clemson", "Georgia Tech"), line = c("12", "-2", "24.5", 
"4", "-8.5", "-3"), linesag = c(12.19, 0.97, 24.26, -2.07, -4.78, 
-2.74), linepayne = c(12, -0.81, 12.53, -0.86, -10.72, -3.87), 
linemassey = c(19.15, -2.1, 21.07, -8.68, -5.45, -6.76)), .Names = c("Home", 
"Road", "line", "linesag", "linepayne", "linemassey"), row.names = c(NA, 
6L), class = "data.frame")

Lastly, I'm running R Version 3.2.1 on x86_64-w64-mingw32.

Answer 1

Without a test case this will be untested. It appears you are getting a global replacement but because some of your columns are character, you get coercion to all character values coerced from 0. I might have tried restricting the process to just numeric columns:

games[ , sapply(games, is.numeric) ][ ng ] <- 
                        games[ , sapply(games, is.numeric)][ng[,1], "linemedian"]

After modifying your almost reproducible code I've concluded that your original code was successful but the output of your checking was the problem area>

 str( games[ , sapply(games, is.numeric)][ng[,1], "linemedian"] )
#num [1:23] -3.25 9.98 -9.1 -9.1 14 -3.25 9.98 -3.25 9.98 2.3 ...

 games[ ng ] <- 
                         games[ , sapply(games, is.numeric)][ng[,1], "linemedian"]
games[ ng[1:2,] ]
[1] " -3.25" "  9.98"

> ng[1:2,]
     row col
[1,] 619   3
[2,] 678   3

> str(games)
'data.frame':   720 obs. of  58 variables:
 $ Home         : chr  "Alabama" "Arizona" "Arkansas" "Arkansas St." ...
 $ Road         : chr  "USC" "BYU" "Louisiana Tech" "Toledo" ...
 $ line         : num  12 -2 24.5 4 -8.5 -3 8.5 37 -10.5 5 ...
 $ linesag      : num  12.19 0.97 24.26 -2.07 -4.78 ...
 $ linepayne    : num  12 -0.81 12.53 -0.86 -10.72 ...
deleted

 > games[ c(619,678)  , 3]
#[1] -3.25  9.98
> games[ matrix(c(619,678,3,3), ncol=2)]
[1] " -3.25" "  9.98"

So the third column remained numeric after the assignment, but for reasons I don't understand the output of the print function for matrix-indexed-extract looked like it was character when it was in fact numeric.