Reputation: 69
I have two RDD's, one looks like this:
increase
rose
die
bear
contracted
own
eyes
lights
making
Then I count the first RDD:
(float,2)
(agree,20)
(healing,2)
(shot,45)
(guide,24)
(opening,11)
(urging,9)
(practises,1)
(surge,9)
(maintained,2)
I have another RDD, which is a dictionary of different forms of verbs, like this
abash,abash,abashed,abashed,abashes,abashing
abate,abate,abated,abated,abates,abating
abide,abide,abode,abode,abides,abiding
absorb,absorb,absorbed,absorbed,absorbs,absorbing
accept,accept,accepted,accepted,accepts,accepting
accompany,accompany,accompanied,accompanied,accompanies,accompanying
ache,ache,ached,ached,aches,aching
achieve,achieve,achieved,achieved,achieves,achieving
Now, I need to count the words in the first RDD and merge the words that belong to the same word but have different forms according to the second RDD. E.g. (work, 100), (works,50), (working,150) -> (work, 300) I tried counting the first RDD and then figuring out which of the elements in the first RDD belongs to which in the second RDD and counting, but this part doesn't know how to do it through an RDD operation
Upvotes: 0
Views: 70