CODEWITHSUNDEEP
pythonapache-sparkpysparkcassandraspark-cassandra-connector
Tri
Tri

Reputation: 37

Pyspark Cassandra - Create correct User Defined Type (UDT) for spark dataframe column with type "struct"

My spark dataframe has the following schema: 

root
 |-- a: string (nullable = true)
 |-- b: long (nullable = true)
 |-- c: array (nullable = true)
 |    |-- element: struct (containsNull = true)
 |    |    |-- c1: string (nullable = true)
 |    |    |-- c2: double (nullable = true)
 |    |    |-- c3: long (nullable = true)

I'm trying to create a UDT in Cassandra keyspace to cater for column c. This new UDT will then be used to create Cassandra table to store my dataframe.

However, I keep encountering syntax error with my CQL. My CQL is:

"CREATE TYPE IF NOT EXISTS keyspace.my_udt(list<element frozen<c1 text, c2 double, c3 bigint>>);"

Error message is: 

SyntaxException: <Error from server: code=2000 [Syntax error in CQL query] message="line 1:50 no viable alternative at input '<' (... NOT EXISTS keyspace .my_udt(list[<]...)">

I'm not familiar with CQL at all. Could someone help me please?

Upvotes: 1

Views: 456

Answers (2)

undefined_variable
undefined_variable

Reputation: 6218

CREATE TYPE IF NOT EXISTS keyspace.my_udt(c1 text, c2 double, c3 bigint);

CREATE TABLE example (id uuid,elements list<FROZEN <my_udt>>,primary key(id));

UDT in Cassandra

Upvotes: 0

Alex Ott
Alex Ott

Reputation: 87069

You need to have following definition for user-defined type (see documentation):

CREATE TYPE IF NOT EXISTS my_udt(c1 text, c2 double, c3 bigint);

And the table should be:

CREATE TABLE my_table(a text, b bigint, c frozen<list<my_udt>>, primary key(a));

Upvotes: 1

Related Questions