CODEWITHSUNDEEP
eclipsescalaapache-spark
Sridhar
Sridhar

Reputation: 61

Error: Could not find or load main class with Spark in Eclipse

Facing issue while running a spark application from Eclipse (scala). However I'm able to run Scala from eclipse without any issue; the issue seems to appearing only with spark app.

Error: Could not find or load main class com.sidSparkScala.RatingsCounter*

package com.sundogsoftware.spark

import org.apache.spark._
import org.apache.spark.SparkContext._
import org.apache.log4j._

/** 
  * Count up how many of each star rating exists in the MovieLens 
  * 100K data set.
  */
object RatingsCounter {

  /** Our main function where the action happens */
  def main(args: Array[String]) {

    // Set the log level to only print errors
    Logger.getLogger("org").setLevel(Level.ERROR)

    // Create a SparkContext using every core of the local machine, named RatingsCounter
    val sc = new SparkContext("local[*]", "RatingsCounter")

    // Load up each line of the ratings data into an RDD
    val lines = sc.textFile("../ml-100k/u.data")

    // Convert each line to a string, split it out by tabs, and extract the third field.
    // (The file format is userID, movieID, rating, timestamp)
    val ratings = lines.map(x => x.toString().split("\t")(2))

    // Count up how many times each value (rating) occurs
    val results = ratings.countByValue()

    // Sort the resulting map of (rating, count) tuples
    val sortedResults = results.toSeq.sortBy(_._1)

    // Print each result on its own line.
    sortedResults.foreach(println)

  }

}

Upvotes: 1

Views: 3450

Answers (3)

Ted Corleone
Ted Corleone

Reputation: 883

You must change the package name to yours, not the courses provided. I guess you import this scala file from external, based the code above, the package is com.sundogsoftware.spark , just change it to yours would be fine.

Upvotes: 0

Waqas Bader Shah
Waqas Bader Shah

Reputation: 31

It seems like the package name you originally created is this com.sidSparkScala.RatingsCounter and the package name mentioned in the script is com.sundogsoftware.spark (at the top). All you have to do is replace the package com.sundogsoftware.spark with com.sidSparkScala.RatingsCounter.

Upvotes: 2

magic
magic

Reputation: 72

Your class is not compiled properly, otherwise eclipse could find this class. Please check, is there any compilation error.

Upvotes: 0

Related Questions