S Novogoratz
S Novogoratz

Reputation: 378

Combine 2 sequential, but irregular, time series datasets with overlapping rows, eliminating duplicate rows

Using R, I'm trying to combine 2 sequential, but irregular, time series datasets using the same fields but with overlapping rows; i.e., some of the same transactions appear in both datasets and I want to eliminate the overlapping rows.

Because the time intervals are irregular, I may have valid identical rows in each dataset. With my example datasets, I want to combine rows 1 through 12 from Dataset 1 with rows 6 through 11 from Dataset 2, to get the desired result. In this example, it's clear that rows 1 through 5 of Dataset 2 are the same as rows 8 through 12 of Dataset 1. I've tried using the unique() function, but it also eliminates identical valid rows. Any ideas on how to solve this dilemma?

Dataset 1

1  2019-02-19 15:17:14 25886    1                           
2  2019-02-19 15:17:14 25886    1                           
3  2019-02-19 15:17:15 25885    1                           
4  2019-02-19 15:17:16 25886    2                           
5  2019-02-19 15:17:16 25886    1                           
6  2019-02-19 15:17:16 25886    2                           
7  2019-02-19 15:17:16 25886    1                           
8  2019-02-19 15:17:18 25885    4                           
9  2019-02-19 15:17:19 25885    1  
10 2019-02-19 15:17:19 25885    1                            
11 2019-02-19 15:17:20 25885    2                           
12 2019-02-19 15:17:21 25885    1                           

Dataset 2

1  2019-02-19 15:17:18 25885    4                           
2  2019-02-19 15:17:19 25885    1  
3  2019-02-19 15:17:19 25885    1                          
4  2019-02-19 15:17:20 25885    2                           
5  2019-02-19 15:17:21 25885    1                           
6  2019-02-19 15:17:23 25886    2                           
7  2019-02-19 15:17:23 25886    3                           
8  2019-02-19 15:17:23 25886    3                           
9  2019-02-19 15:17:23 25886    1                           
10 2019-02-19 15:17:23 25886    1                           
11 2019-02-19 15:17:23 25886    2 

My desired result is:

1  2019-02-19 15:17:14 25886    1                           
2  2019-02-19 15:17:14 25886    1                           
3  2019-02-19 15:17:15 25885    1                           
4  2019-02-19 15:17:16 25886    2                           
5  2019-02-19 15:17:16 25886    1                           
6  2019-02-19 15:17:16 25886    2                           
7  2019-02-19 15:17:16 25886    1                           
8  2019-02-19 15:17:18 25885    4                           
9  2019-02-19 15:17:19 25885    1   
10 2019-02-19 15:17:19 25885    1                             
11 2019-02-19 15:17:20 25885    2                           
12 2019-02-19 15:17:21 25885    1                      
13 2019-02-19 15:17:23 25886    2                           
14 2019-02-19 15:17:23 25886    3                           
15 2019-02-19 15:17:23 25886    3                           
16 2019-02-19 15:17:23 25886    1                           
17 2019-02-19 15:17:23 25886    1                           
18 2019-02-19 15:17:23 25886    2 

Here's Dataset 1

structure(list(time = structure(c(1550589434, 1550589434, 1550589435, 
1550589436, 1550589436, 1550589436, 1550589436, 1550589438, 1550589439, 
1550589439, 1550589440, 1550589441), class = c("POSIXct", "POSIXt"
), tzone = "UTC"), price = c(25886, 25886, 25885, 25886, 25886, 
25886, 25886, 25885, 25885, 25885, 25885, 25885), size = c(1, 
1, 1, 2, 1, 2, 1, 4, 1, 1, 2, 1)), row.names = c("1", "2", "3", 
"4", "5", "6", "7", "8", "9", "10", "11", "12"), class = "data.frame")

Here's Dataset 2

structure(list(time = structure(c(1550589438, 1550589439, 1550589439, 
1550589440, 1550589441, 1550589443, 1550589443, 1550589443, 1550589443, 
1550589443, 1550589443), class = c("POSIXct", "POSIXt"), tzone = "UTC"), 
price = c(25885, 25885, 25885, 25885, 25885, 25886, 25886, 
25886, 25886, 25886, 25886), size = c(4, 1, 1, 2, 1, 2, 3, 
3, 1, 1, 2)), row.names = c("1", "2", "3", "4", "5", "6", 
"7", "8", "9", "10", "11"), class = "data.frame")

Upvotes: 0

Views: 62

Answers (1)

arg0naut91
arg0naut91

Reputation: 14774

One idea would be:

library(dplyr)

df2 %>%
  anti_join(df1) %>%
  bind_rows(df1)

Upvotes: 1

Related Questions