xRandom
xRandom

Reputation: 69

How do you count both filters through RDD operations?

I have two RDD's, one looks like this:

increase
rose
die
bear
contracted
own
eyes
lights
making

Then I count the first RDD:

(float,2)
(agree,20)
(healing,2)
(shot,45)
(guide,24)
(opening,11)
(urging,9)
(practises,1)
(surge,9)
(maintained,2)

I have another RDD, which is a dictionary of different forms of verbs, like this

abash,abash,abashed,abashed,abashes,abashing
abate,abate,abated,abated,abates,abating
abide,abide,abode,abode,abides,abiding
absorb,absorb,absorbed,absorbed,absorbs,absorbing
accept,accept,accepted,accepted,accepts,accepting
accompany,accompany,accompanied,accompanied,accompanies,accompanying
ache,ache,ached,ached,aches,aching
achieve,achieve,achieved,achieved,achieves,achieving

Now, I need to count the words in the first RDD and merge the words that belong to the same word but have different forms according to the second RDD. E.g. (work, 100), (works,50), (working,150) -> (work, 300) I tried counting the first RDD and then figuring out which of the elements in the first RDD belongs to which in the second RDD and counting, but this part doesn't know how to do it through an RDD operation

Upvotes: 0

Views: 70

Answers (1)

OldWolfs
OldWolfs

Reputation: 626

Is this a homework or something. Same question (that targets same task) is asked and answered here.

Upvotes: 1

Related Questions