CODEWITHSUNDEEP
cassandraapache-spark-sqldatastaxcassandra-3.0spark-cassandra-connector
BdEngineer
BdEngineer

Reputation: 3179

java.util.NoSuchElementException: Columns not found in table

My cassandra table columns in lower case like below

CREATE TABLE model_family_by_id(
  model_family_id int PRIMARY KEY,
  model_family text,
  create_date   date,
  last_update_date date,
  model_family_name text
);

my dataframe schema is like this

     root
               |-- MODEL_FAMILY_ID: decimal(38,10) (nullable = true)
               |-- MODEL_FAMILY: string (nullable = true)
               |-- CREATE_DATE: timestamp (nullable = true)
               |-- LAST_UPDATE_DATE: timestamp (nullable = true)
               |-- MODEL_FAMILY_NAME: string (nullable = true)

So while insert into cassandra I am getting below error

tabException in thread "main" java.util.NoSuchElementException: Columns not found in table sample_cbd.model_family_by_id: MODEL_FAMILY_ID, MODEL_FAMILY, CREATE_DATE, LAST_UPDATE_DATE, MODEL_FAMILY_NAME
    at com.datastax.spark.connector.SomeColumns.selectFrom(ColumnSelector.scala:44)

Upvotes: 1

Views: 1207

Answers (1)

Alex Ott
Alex Ott

Reputation: 87069

If I correctly understand the source code, the Spark Connector wraps the columns in to the double quotes, so they become case-sensitive, and don't match to the names in the CQL definition.

You need to change schema of your DataFrame - either run the withColumnRenamed on it for every column, or use select with alias for every column.

Upvotes: 1

Related Questions