CODEWITHSUNDEEP
hadoopapache-spark
Bobo
Bobo

Reputation: 971

unable to submit spark python script

I'm using the following script to submit a python script

#!/usr/bin/python

from pyspark.mllib.classification import LogisticRegressionWithSGD
from pyspark.mllib.regression import LabeledPoint
from numpy import array
from pyspark import SparkContext as sc, SparkConf

data = sc.textFile("hdfs:/dataset/parkinsons.data")

got this error:

data = sc.textFile("hdfs:/dataset/parkinsons.data")
TypeError: unbound method textFile() must be called with SparkContext instance as first argument (got str instance instead)

Upvotes: 0

Views: 1216

Answers (1)

artemdevel
artemdevel

Reputation: 641

You must create a SparkContext at first, for example:

from pyspark import SparkContext

sc = SparkContext(appName="TestApp")
data = sc.textFile("hdfs:/dataset/parkinsons.data")

Upvotes: 1

Related Questions