CODEWITHSUNDEEP
dynamiccassandra
Sridhar
Sridhar

Reputation: 21

How to handle Dynamic columns in Cassandra

I am loading JSON data to Cassandra table through Python script. But few Json files have more columns than usual. Currently i have created table with 100 columns and able to insert all. But there are chances that few json files will have more than 100 columns. How to handle this? Is there any way we can create dynamic columns if Json has more columns than table?

Upvotes: 1

Views: 932

Answers (1)

Alex Ott
Alex Ott

Reputation: 87164

If you're using CQL, then you need to define all columns before inserting the data. Theoretically you can use ALTER TABLE add ... to add new columns, but usually it's not recommended to do programmatically as it may cause schema disagreement, and other problems.

You may workaround this problem as following:

  1. Store the JSON as text in addition to primary key and the most used columns, and then parse data when you read it (it could be even done automatically in Java driver 3.x by using extra codecs;
  2. Store data in the maps (best to use frozen map if you won't update individual values) with key as text, and value corresponding to actual value type - int, text, etc. For example:
create table test (
  pk1 ..,
  pk2 ..,
  pkN ..,
  imap frozen<map<text, int>,
  tmap frozen<map<text, text>,
  ...
  primary key(pk1, pk2, ...)
);

and then in your code, separate columns by types, and insert into corresponding map.

Upvotes: 1

Related Questions