In R, improving efficiency / speed parsing a long XML document

Question

I have my hands on a bit of sportVU data (coordinate player tracking of basketball players) in XML format, and am trying to find an efficient way to parse this into R dataframes. For reference, here is the type of data I am working with:

xml_str = '
             
             
             
             
             
             
             
             
             
             
             
             
             
             
             
          '

This is a subset of the actual data, where 1 game of basketball corresponds with roughly (25 per second) * (10 players) * (48 * 60 seconds in game) = 720,000 moment nodes per game. My goal is to parse this into an organized dataframe - the first 4 columns are the first 4 attributes of the moment nodes (game-clock, time, game-event-id, shot-clock), and then there are several columns of the data that arise from the locations attribute, which is storing coordinate data for 10 different players plus the ball, all in one string (each location attribute has 55 numbers, 5 for each of the 10 players and 5 for the ball).

Here is my code, which is working okay-ish but needs to be sped up quite a bit for efficiency for what I'm doing. I use the XML package, and simple functions within, but am okay with switching to xml2 or doing anything else needed to speed this up.

sportvu = xmlParse(xml_str)

# quick load the data into a dataframe
sportvu_df <- data.frame(
  game_clock = sapply(sportvu["//sequences/moment/@game-clock"], as, "numeric"),
  time = sapply(sportvu["//sequences/moment/@time"], as, "character"),
  game_event_id = sapply(sportvu["//sequences/moment/@game-event-id"], as, "integer"),
  shot_clock = sapply(sportvu["//sequences/moment/@shot-clock"], as, "numeric"), 
  locations = sapply(sportvu["//sequences/moment/@locations"], as, "character")
)

# convert locations into what we want, many more columns 
locations = as.character(sportvu_df$locations)
locations = strsplit(gsub(";", ",", locations), ",")
reps = sapply(locations, length) / 5
locations = as.data.frame(matrix(unlist(locations), ncol = 5, byrow = TRUE), stringsAsFactors = FALSE)
colnames(locations) = c("team_id", "global.player.id", "x_loc", "y_loc", "radius")

locations$global.player.id = as.integer(locations$global.player.id); locations$team_id = as.integer(locations$team_id); locations$x_loc = as.numeric(locations$x_loc); locations$y_loc = as.numeric(locations$y_loc); locations$radius = as.numeric(locations$radius)

# connect locations back with sportvu_df
sportvu_df = sportvu_df[!(names(sportvu_df) %in% "locations")]
sportvu_df = sportvu_df[rep(row.names(sportvu_df), reps), ]
sportvu_df = cbind(sportvu_df, locations)
sportvu_df$order = seq(1,nrow(sportvu_df),by=1)

Any thoughts are appreciated. Thanks!

Dave2e · Accepted Answer

There is no need to wrap the type conversions into an sapply function. They are vectorized thus a direct call should be enough:

sportvu_df <- data.frame(
  game_clock = as.numeric(sportvu["//sequences/moment/@game-clock"]),
  time = as.character(sportvu["//sequences/moment/@time"]),
  game_event_id = as.integer(sportvu["//sequences/moment/@game-event-id"]),
  shot_clock = as.numeric(sportvu["//sequences/moment/@shot-clock"]), 
  locations = as.character(sportvu["//sequences/moment/@locations"])
)

By eliminating the 5 sapply (i.e. loops) this should speed up your routine.

In R, improving efficiency / speed parsing a long XML document

Answers (1)

Related Questions