CODEWITHSUNDEEP
cassandra
Konstantin Milyutin
Konstantin Milyutin

Reputation: 12366

Cassandra replicates to a node different from the one in nodetool getendpoints

I have a table in Cassandra:

CREATE TABLE imdb.movies_by_actor (
    actor text,
    movie_id uuid,
    character text,
    movie_title text,
    salary int,
    PRIMARY KEY (actor, movie_id)
) WITH CLUSTERING ORDER BY (movie_id ASC)


 actor     | movie_id                             | character | movie_title | salary
-----------+--------------------------------------+-----------+-------------+--------
 Tom Hanks | 767b7a89-868c-46ce-8fa6-f6184dfb6d69 |       Dad |     Seattle |  25000
 Tom Hanks | a9a64b89-a19d-46e9-b5ee-991ac4939891 |   Officer |  Green mile |  20000

Then I find out which nodes are responsible for the 'Tom Hanks' partition:

select token(actor) from movies_by_actor ;

 system.token(actor)
----------------------
 -4258050846863339499
 -4258050846863339499

root@2f5aa8d649e2:/# nodetool getendpoints imdb movies_by_actor -4258050846863339499
172.13.0.6
172.13.0.3
172.13.0.4

Then I shut the node corresponding to 172.13.0.6 down:

docker stop cassandra6
root@2f5aa8d649e2:/# ping 172.13.0.6
PING 172.13.0.6 (172.13.0.6): 56 data bytes
92 bytes from 2f5aa8d649e2 (172.13.0.2): Destination Host Unreachable

When I try to update the row and look at tracing info, it looks like data are sent to 172.13.0.2, 172.13.0.4, 172.13.0.5:

cqlsh:imdb> update movies_by_actor set salary = 26000 where actor = 'Tom Hanks' and movie_id = 767b7a89-868c-46ce-8fa6-f6184dfb6d69;

Tracing session: f44dbd70-4228-11e7-89c9-cf534e0135c6

 activity                                                                                                                                                       | timestamp                  | source     | source_elapsed | client
----------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------+------------+----------------+-----------
                                                                                                                                             Execute CQL3 query | 2017-05-26 15:35:32.295000 | 172.13.0.2 |              0 | 127.0.0.1
 Parsing update movies_by_actor set salary = 26000 where actor = 'Tom Hanks' and movie_id = 767b7a89-868c-46ce-8fa6-f6184dfb6d69; [Native-Transport-Requests-1] | 2017-05-26 15:35:32.295000 | 172.13.0.2 |            303 | 127.0.0.1
                                                                                                              Preparing statement [Native-Transport-Requests-1] | 2017-05-26 15:35:32.295000 | 172.13.0.2 |            646 | 127.0.0.1
                                                                                                Determining replicas for mutation [Native-Transport-Requests-1] | 2017-05-26 15:35:32.296000 | 172.13.0.2 |           1181 | 127.0.0.1
                                                                                                                       Appending to commitlog [MutationStage-3] | 2017-05-26 15:35:32.296000 | 172.13.0.2 |           1420 | 127.0.0.1
                                                                                                           Adding to movies_by_actor memtable [MutationStage-3] | 2017-05-26 15:35:32.296000 | 172.13.0.2 |           1557 | 127.0.0.1
                                                                          Sending MUTATION message to /172.13.0.4 [MessagingService-Outgoing-/172.13.0.4-Small] | 2017-05-26 15:35:32.296000 | 172.13.0.2 |           1567 | 127.0.0.1
                                                                          Sending MUTATION message to /172.13.0.5 [MessagingService-Outgoing-/172.13.0.5-Small] | 2017-05-26 15:35:32.296000 | 172.13.0.2 |           1583 | 127.0.0.1
                                                                             MUTATION message received from /172.13.0.2 [MessagingService-Incoming-/172.13.0.2] | 2017-05-26 15:35:32.297000 | 172.13.0.4 |             27 | 127.0.0.1
                                                                             MUTATION message received from /172.13.0.2 [MessagingService-Incoming-/172.13.0.2] | 2017-05-26 15:35:32.297000 | 172.13.0.5 |             23 | 127.0.0.1
                                                                                                                       Appending to commitlog [MutationStage-1] | 2017-05-26 15:35:32.297000 | 172.13.0.4 |            332 | 127.0.0.1
                                                                                                           Adding to movies_by_actor memtable [MutationStage-1] | 2017-05-26 15:35:32.297000 | 172.13.0.4 |            577 | 127.0.0.1
                                                                                                            Enqueuing response to /172.13.0.2 [MutationStage-1] | 2017-05-26 15:35:32.298000 | 172.13.0.4 |            884 | 127.0.0.1
                                                                                                                       Appending to commitlog [MutationStage-2] | 2017-05-26 15:35:32.298000 | 172.13.0.5 |           1526 | 127.0.0.1
                                                                  Sending REQUEST_RESPONSE message to /172.13.0.2 [MessagingService-Outgoing-/172.13.0.2-Small] | 2017-05-26 15:35:32.298000 | 172.13.0.4 |           1122 | 127.0.0.1
                                                                                                           Adding to movies_by_actor memtable [MutationStage-2] | 2017-05-26 15:35:32.299000 | 172.13.0.5 |           1854 | 127.0.0.1
                                                                                                            Enqueuing response to /172.13.0.2 [MutationStage-2] | 2017-05-26 15:35:32.299000 | 172.13.0.5 |           2187 | 127.0.0.1
                                                                  Sending REQUEST_RESPONSE message to /172.13.0.2 [MessagingService-Outgoing-/172.13.0.2-Small] | 2017-05-26 15:35:32.299000 | 172.13.0.5 |           2423 | 127.0.0.1
                                                                     REQUEST_RESPONSE message received from /172.13.0.4 [MessagingService-Incoming-/172.13.0.4] | 2017-05-26 15:35:32.300000 | 172.13.0.2 |             56 | 127.0.0.1
                                                                     REQUEST_RESPONSE message received from /172.13.0.5 [MessagingService-Incoming-/172.13.0.5] | 2017-05-26 15:35:32.300000 | 172.13.0.2 |             15 | 127.0.0.1
                                                                                                  Processing response from /172.13.0.5 [RequestResponseStage-5] | 2017-05-26 15:35:32.300000 | 172.13.0.2 |            273 | 127.0.0.1
                                                                                                  Processing response from /172.13.0.4 [RequestResponseStage-4] | 2017-05-26 15:35:32.300000 | 172.13.0.2 |            774 | 127.0.0.1
                                                                                                                                               Request complete | 2017-05-26 15:35:32.296887 | 172.13.0.2 |           1887 | 127.0.0.1

Selecting with consistency level ALL also works although 172.13.0.6 is down. Could someone explain it please?

Upvotes: 1

Views: 116

Answers (1)

Ashraful Islam
Ashraful Islam

Reputation: 12840

The command nodetool getendpoints received partition key value as parameter not the token

But there is a issue with nodetool getendpoints when the parameter value contain space.

You could use the script from the answer : https://stackoverflow.com/a/43155224/2320144

Or

You could run a nodetool ring to list out the token ranges for each node, and see which nodes are responsible for that range.

Source : https://stackoverflow.com/a/30515201/2320144

Upvotes: 2

Related Questions