CODEWITHSUNDEEP
migrationrecordaerospike
surajs1n
surajs1n

Reputation: 1583

Migrating records to another set on Aerospike

Current Setup:
I have a single namespace running on the Aerospike cluster. The namespace has few sets in it.

My Use-case:
I want to copy all the records from one set (has ~100 records) to another new set (keeping the schema same) under the same namespace.

My Finding:
I did some deep dive and found out a few solutions using aql:

  1. List down all the records from the first set and insert them one-by-one into a new set.
    Pros: Simple to implement.
    Cons: Time taking, and Prone to manual error.
  2. Using asbackup/asrestore command.
    Pros: It is immune to manual error.
    Cons: It doesn't allow to change the name of the set during restoration, which I can't afford. Aerospike's FAQ link does provide some workaround, but again it is risky.

Help Needed:
Is there any efficient way to migrate data from one to another set, with less effort and validation? I did think of writing some java code that would scan the entire set and write those records into another set, but again that was falling under the first categories I explained earlier.

Thanks!

Upvotes: 3

Views: 1092

Answers (1)

pgupta
pgupta

Reputation: 5415

A record in Aerospike is stored using the hash of your key and your set name. Set name is "stored" with that record in Aerospike purely as a metadata on that record. So you can scan an entire namespace and return records belonging to that set and in the scan callback, write each of them back as new records (due to the different set name). You will have to know "your key" for each record that comes back from the scan. By default Aerospike only stores the 20byte hash digest as the key for the record. So unless you stored it explicitly in the record either with send key true or in a bin, I don't see how you would identify "your key". Storing "your key" in a bin is easiest. You may have to first update all your 100 records and add a bin that has "your key" in it. Then in scan callback, where records come in no particular order, you will be able to compose a new Key with "your key" and "new set name". You will have to write your own java code for it. (If you have "your key" in the original records - its easy to do.)

I have not tested this .. but something along these lines would work assuming original records had your key in the "mykey" bin.

client.scanAll(null, "test", "set1", new ScanCallback(){
                public void scanCallback (Key key, Record record) throws AerospikeException {   

                    String mykey = (String)(record.getValue("mykey"));

                    String bin1data = record.getString("bin1");  
                    //Alternate way to get string

                    Key reckey = new Key("test", "set2", mykey);
                    client.put(null, reckey, new Bin("bin1", bin1data));
                }
});

Upvotes: 3

Related Questions