CODEWITHSUNDEEP
javaapache-sparkdataframe
volity
volity

Reputation: 93

Why I cannot use callUDF method?

I use spark 1.6.1, and code in java. When I use callUDF(), it shows 

The method callUDF(String, Column) is undefined for the type PhaseOne

and callUdf() does not work. My codes are as follows: 

sqlContext.udf().register("stringToLong", new UDF1<String, Long>() {

        @Override
        public Long call(String arg0) throws Exception {
            // TODO Auto-generated method stub
            IPTypeConvert itc = new IPTypeConvert();
            return itc.stringtoLong(arg0);
        }
    }, DataTypes.LongType);
    DataFrame interDF = initInterDF.withColumn("interIPInt", callUDF("stringToLong", initInterDF.col("interIP")));

Upvotes: 3

Views: 1072

Answers (1)

T. Gawęda
T. Gawęda

Reputation: 16076

You must add at the beginning:

import static org.apache.spark.sql.functions.callUDF;

And then use it:

sqlContext.udf().register("stringToLong", new UDF1<String, Long>() {

        @Override
        public Long call(String arg0) throws Exception {
            // TODO Auto-generated method stub
            IPTypeConvert itc = new IPTypeConvert();
            return itc.stringtoLong(arg0);
        }
    }, DataTypes.LongType);
DataFrame interDF = initInterDF.withColumn("interIPInt", callUDF("stringToLong", initInterDF.col("interIP")));

Upvotes: 3

Related Questions