CODEWITHSUNDEEP
pysparkapache-spark-sqlaws-glue
Miroslav Petrovic
Miroslav Petrovic

Reputation: 99

Problem when converting DataFrame to DynamicFrame

I have a code that converts DataFrame to DynamicFrame and I get this weird error when trying to execute return statement, any clues what's going on?

Error: {AttributeError}'str' object has no attribute '_jvm'

# record is DynamicFrame
def extractCustomFields(record, ctx):
    rec = record.toDF()
    rec = rec.withColumn("lastname", rec["customfields"][0].value)
    rec.show()
return DynamicFrame.fromDF(rec, ctx, "recordTransform")

Upvotes: 0

Views: 2938

Answers (1)

Robert Kossendey
Robert Kossendey

Reputation: 7028

fromDF() expects the GlueContext as second argument. You need to pass that:

return DynamicFrame.fromDF(rec, ctx, "recordTransform")

Upvotes: 1

Related Questions