CODEWITHSUNDEEP
rmergematchnumericapproximate
celow
celow

Reputation: 85

Join two data frames by nearest match

I have two large datasets with the only shared feature being a numerical timestamp. I'd like to merge the data frames by this timestamp, but the frequency of data collection doesn't match exactly so I need to allow it to merge with the nearest possible match.

As a simplified example, here's a small data set with a value column, some event, and an ID:

a<-c("150", "164", "175", "183", "195", "200", "205","213")
b<-c("start1","end1","start2", "end2", "start1", "end1", "start2", "end2")
c<-c("A","A","A", "A", "B", "B", "B", "B")

(data<-data.table(value = a, event = b, ID = c))

And I'd like to be able to merge this "data" with this numerical series ("times") by the value column:

(times<-data.frame(value = c(seq(from = 150, to = 213, by = 3))))

So that they merge by the nearest approximate match in the value column to produce this final data frame:

agoal<-c(seq(from = 150, to = 213, by = 3))
bgoal<-c("start1","","","","","end1","", "",
     "start2", "", "", "end2", "", "", "",
     "start1", "", "end1", "start2", "", "", "end2")
cgoal<-c("A","","","","","A","", "",
         "A", "", "", "A", "", "", "",
         "B", "", "B", "B", "", "", "B")

(goal<-data.frame(value = agoal, event = bgoal, ID = cgoal))

Is there a way to do this, especially for a very large dataset (so it doesn't crash R)?

Upvotes: 1

Views: 728

Answers (2)

celow
celow

Reputation: 85

To join by nearest matches without filling in the gaps with approximate matches, fuzzyjoin worked well!

(end<-fuzzyjoin::difference_left_join(times, data, by = "value", max_dist = 1, distance_col= "distance"))

Upvotes: 1

Peace Wang
Peace Wang

Reputation: 2419

data.table provides a roll join solution.

library(data.table)
setkey(data,value)
setkey(times,value)
data[times,roll = "nearest"]
#    value  event ID
# 1:   150 start1  A
# 2:   153 start1  A
# 3:   156 start1  A
# 4:   159   end1  A
# 5:   162   end1  A
# 6:   165   end1  A
# 7:   168   end1  A
# 8:   171 start2  A
# 9:   174 start2  A
#10:   177 start2  A
#11:   180   end2  A
#12:   183   end2  A
#13:   186   end2  A
#14:   189   end2  A
#15:   192 start1  B
#16:   195 start1  B
#17:   198   end1  B
#18:   201   end1  B
#19:   204 start2  B
#20:   207 start2  B
#21:   210   end2  B
#22:   213   end2  B

data:

a<-c("150", "164", "175", "183", "195", "200", "205","213")
b<-c("start1","end1","start2", "end2", "start1", "end1", "start2", "end2")
c<-c("A","A","A", "A", "B", "B", "B", "B")

data<-data.table(value = as.numeric(a), event = b, ID = c)

times<-data.table(value = c(seq(from = 150, to = 213, by = 3)))

Upvotes: 2

Related Questions