CODEWITHSUNDEEP
pysparkaws-glue
user16974
user16974

Reputation: 1

AnalysisException: cannot resolve '`item1`' given input columns: [];

I used AWS Glue Transform-custom code, and wrote the following code.

def MyTransform (glueContext, dfc) -> DynamicFrameCollection:

    from pyspark.sql.functions import col, lit
    selected = dfc.select(list(dfc.keys())[0]).toDF()
    selected = selected.withColumn('item2',lit(col('item1')))
    results = DynamicFrame.fromDF(selected, glueContext, "results")
    return DynamicFrameCollection({"results": results}, glueContext)

I got the following error.

AnalysisException: cannot resolve '`item1`' given input columns: [];

Upvotes: 0

Views: 1768

Answers (1)

pltc
pltc

Reputation: 6082

You don't need to wrap lit(col('item1')). lit is used to create a literal static value, while col is to refer to a specific column. Depends on what exactly you're trying to achieve, but this change will at least fix your error

selected = selected.withColumn('item2', col('item1'))

Upvotes: 1

Related Questions