CODEWITHSUNDEEP
rdataframefor-loopsplit
RobertP.
RobertP.

Reputation: 285

R - split dataset by row position and save in different files

I have a huge dataset in which several mini dataset were merged. I want to split them in different dataframes and save them. The mini datasets are identified by a variable name (which always include the string "-gram") on a given row.

I have been trying to construct a for loop, but with no luck.

grams <- read.delim("grams.tsv", header=FALSE) #read dataset
index <- which(grepl("-gram", grams$V1), arr.ind=TRUE) # identify the row positions where each mini dataset starts
index[10] <- nrow(grams) # add the total number of rows as last variable of the vector 

start <- c() # initialize vector
end <- c() # initialize vector
for (i in 1:length(index)-1) for ( k in 2:length(index)) {
    start[i] <- index[i] # add value to the vector start
    if (k != 10) { end[k-1] <- index[k]-1 } else { end[k-1] <- index[k] } # add value to the vector end    
    gram <- grams[start[i]:end[i],] #subset the dataset grams so that the split mini dataset has start and end that correspond to the index in the vector
    write.csv(gram, file=paste0("grams_", i, ".csv"), row.names=FALSE) # save dataset
}

I get an error when I try to subset the dataset:

Error in start[i]:end[i] : argument of length 0

...and I do not understand why! Can anyone help me?

Thanks!

Upvotes: 1

Views: 141

Answers (2)

akrun
akrun

Reputation: 887118

An option with group_split and endsWith

library(dplyr)
library(stringr)
dat %>%
      group_split(grp = cumsum(endsWith(V1, '-gram')), keep = FALSE)

Upvotes: 1

r2evans
r2evans

Reputation: 160447

You can cumsum and split:

dat <- data.frame(V1 = c("foo", "bar", "quux-gram", "bar-gram", "something", "nothing"),
                  V2 = 1:6, stringsAsFactors = FALSE)
dat
#          V1 V2
# 1       foo  1
# 2       bar  2
# 3 quux-gram  3
# 4  bar-gram  4
# 5 something  5
# 6   nothing  6
grepl("-gram$", dat$V1)
# [1] FALSE FALSE  TRUE  TRUE FALSE FALSE
cumsum(grepl("-gram$", dat$V1))
# [1] 0 0 1 2 2 2

spl_dat <- split(dat, cumsum(grepl("-gram$", dat$V1)))
spl_dat
# $`0`
#    V1 V2
# 1 foo  1
# 2 bar  2
# $`1`
#          V1 V2
# 3 quux-gram  3
# $`2`
#          V1 V2
# 4  bar-gram  4
# 5 something  5
# 6   nothing  6

With that, you can write them to files with:

ign <- Map(write.csv, spl_dat, sprintf("gram-%03d.csv", seq_along(spl_dat)),
           list(row.names=FALSE))

Upvotes: 2

Related Questions