CODEWITHSUNDEEP
snappydata
Jason
Jason

Reputation: 2076

org.apache.spark.sql.catalyst.TableIdentifier cannot be resolved error in SnappySQLJob

I have an compile-time error trying to write a SnappySQLJob. Am I missing a dependency?

The error message is:

The type org.apache.spark.sql.catalyst.TableIdentifier cannot be resolved. It is indirectly referenced from required .class files

@Override
    public Object runJob(Object sparkContext, Config jobConfig) {
        SnappyContext snappyContext = (SnappyContext)sparkContext;

        String fileResource = "data.csv";

        DataFrame dataFrame = snappyContext.read()
                .format("com.databricks.spark.csv")
                .option("header", "true")
                .option("inferSchema", "true")
                .load(fileResource);

        // Compile-Time error is on this line
        dataFrame.write().insertInto("example_table_col");


        return null;
    }

Here is my pom.xml dependencies:

    <dependency>
        <groupId>io.snappydata</groupId>
        <artifactId>snappy-core_2.10</artifactId>
        <version>0.2.1-PREVIEW</version>
    </dependency>
    <dependency>
        <groupId>io.snappydata</groupId>
        <artifactId>snappy-tools_2.10</artifactId>
        <version>0.2.1-PREVIEW</version>
        <exclusions>
            <exclusion>
                <artifactId>jdk.tools</artifactId>
                <groupId>jdk.tools</groupId>
            </exclusion>
            <exclusion>
                <artifactId>logback-classic</artifactId>
                <groupId>ch.qos.logback</groupId>
            </exclusion>
        </exclusions>
    </dependency>

Upvotes: 1

Views: 475

Answers (1)

Sumedh
Sumedh

Reputation: 383

This old release seems to be missing spark-catalyst dependency. I will suggest upgrading to 0.5 release version instead (snappy-tools is now called snappy-cluster) and the snappydata cluster should also be upgraded to 0.5

For the 0.2.1 release, below should correct the problem:

<dependency>
  <groupId>io.snappydata</groupId>
  <artifactId>snappy-spark-catalyst_2.10</artifactId>
  <version>1.6.0-BETA</version>
</dependency>
<dependency>
  <groupId>io.snappydata</groupId>
  <artifactId>snappy-spark-sql_2.10</artifactId>
  <version>1.6.0-BETA</version>
</dependency>

Upvotes: 1

Related Questions