CODEWITHSUNDEEP
pythonapache-sparkpyspark
user3803714
user3803714

Reputation: 5389

Saving PairedRDD as a text file

users_grpd = pairs.groupByKey()

users_grpd_flattened = meds_grpd.map(
    lambda keyValue: (keyValue[0], ' '.join(map(str, keyValue[1]))))

users_grpd_flattened.saveAsTextFile('pairedrddresults.txt')

Output: 

(u'3300975212', '120818 120519 120850 120521')

(u'3200272220', '120036 105037')

(u'13101231222', '2024574 12024')

I was wondering if there is a way to save this pairedrdd as a text file where the leading u and quotes are omitted?

Upvotes: 2

Views: 349

Answers (1)

zero323
zero323

Reputation: 330393

If you need a specific format you can map to strings directly:

users_grpd_flattened = (pairs.groupByKey().
    map(lambda (k, vals): "{0}, {1}".format(k, ' '.join(str(x) for x in vals))))

If parentheses are required just replace format string with "({0}, {1})".

Upvotes: 1

Related Questions