CODEWITHSUNDEEP
cassandrabigdatacqldata-migration
kjk
kjk

Reputation: 126

How to Copy/Move data between columnfamilies with same structure in cassandra

I'm looking for a method to copy/move data in a column family to another with same structure, in same keyspace.

I had already tried the COPY command to import as csv & export the same to destination column family.

Since the data is large, I'm getting timeouts while using COPY.

One method is to write an application to copy data using a client api and add to the other column family.

Is there any tool to copy/move data between different column family? Or any other way?

Upvotes: 2

Views: 1063

Answers (2)

user3810849
user3810849

Reputation: 1

If you want to get rid of the timeouts of cqlsh. You could actually start cqlsh with the option --request-timeout=3600, which changes the timeout from the default 10 seconds to an hour.

Upvotes: 0

RussS
RussS

Reputation: 16576

I would recommend using Spark for this kind of bulk migration. It's also a useful tool for general maintenance of C*.

https://github.com/datastax/spark-cassandra-connector

With spark the command

sc.cassandraTable("ks1","table").saveToCassandra("ks2","table")

you would move your tables.

If you aren't interested in Spark I think a custom java program or Brian Hess's Bulkloader tool would be useful

https://github.com/brianmhess/cassandra-loader

Upvotes: 3

Related Questions