CODEWITHSUNDEEP
rarulesmarket-basket-analysis
Babatunde Awosanya
Babatunde Awosanya

Reputation: 81

Association analysis with duplicate transactions using arules package in R

I want to create a transaction object in basket format which I can call anytime for my analyses. The data contains comma separated items with 1001 transactions. The first 10 transactions look like this:

hering,corned_b,olives,ham,turkey,bourbon,ice_crea
baguette,soda,hering,cracker,heineken,olives,corned_b
avocado,cracker,artichok,heineken,ham,turkey,sardines
olives,bourbon,coke,turkey,ice_crea,ham,peppers
hering,corned_b,apples,olives,steak,avocado,turkey
sardines,heineken,chicken,coke,ice_crea,peppers,ham
olives,bourbon,coke,turkey,ice_crea,heineken,apples
corned_b,peppers,bourbon,cracker,chicken,ice_crea,baguette
soda,olives,bourbon,cracker,heineken,peppers,baguette
corned_b,peppers,bourbon,cracker,chicken,bordeaux,hering
...

I observed that there are duplicated transactions in the data and removed them but each time I tried to read the transactions, I get:

Error in asMethod(object) : can not coerce list with transactions with duplicated items

Here is my code:

data <- read.csv("AssociationsItemList.txt",header=F)
data <-  data[!duplicated(data),]
pop <- NULL
for(i in 1:length(data)){
pop <- paste(pop, data[i],sep="\n")
}
write(pop, file = "Trans", sep = ",")
transdata <- read.transactions("Trans", format = "basket", sep=",")

I'm sure there's something little yet important I've missed. Kindly offer your assistance.

Upvotes: 8

Views: 16528

Answers (3)

Andre
Andre

Reputation: 370

I just used the 'unique' function to remove duplicates. My data was a little different since I had a dataframe (data was too large for a CSV) and I had 2 columns: product_id and transaction_id. I know it's not your specific question, but I had to do this to create the transaction dataset and apply association rules.

data # > 1 Million Transactions 
data <- unique(data[ , 1:2 ] )
trans <- as(split(data[,"product_id"], data[,"trans_id"]),"transactions")
rules <- apriori(trans, parameter = list(supp = 0.001, conf = 0.2))

Upvotes: 0

xialu
xialu

Reputation: 194

Vincent Zoonekynd is right, the problem is caused by duplicated items in a transaction. Here I can explain why arules require transactions without duplicated items.

  • The data of transactions is store internally as a ngCMatrix Object. Relevant source code:

    setClass("itemMatrix",
  representation(
    data        = "ngCMatrix", 
...
setClass("transactions",
  contains = "itemMatrix",
...

  • ngCMatrix is an sparse matrix defined at Matrix package. It's description from official document:

    The nsparseMatrix class is a virtual class of sparse “pattern” matrices, i.e., binary matrices conceptually with TRUE/FALSE entries. Only the positions of the elements that are TRUE are stored

It seems ngCMatirx stored status of an element by an binary indicator. Which means the transactions object in arules can only store exist/not exist for a transaction object and can not record quantity. So...

Upvotes: 2

Vincent Zoonekynd
Vincent Zoonekynd

Reputation: 32351

The problem is not with duplicated transactions (the same row appearing twice) but duplicated items (the same item appearing twice, in the same transaction -- e.g., "olives" on line 4).

read.transactions has an rm.duplicates argument to remove those duplicates.

read.transactions("Trans", format = "basket", sep=",", rm.duplicates=TRUE)

Upvotes: 18

Related Questions