CODEWITHSUNDEEP
databasereplicationclickhouse
Tomáš Repík
Tomáš Repík

Reputation: 155

Unable to setup replication in ClickHouse using Zookeeper

I've spent past two days trying to setup replication in ClickHouse, but what ever configuration I try I end up with the same behavior.

I'm able to create a ReplicatedMergeTree table on the first node and insert data to it. Then I create a replica on the second node. The data gets replicated and I can see it querying the second node. But when I insert data to the second node the weird behavior starts. Data is not copied to the first node and it gets the following error:

2017.11.14 11:16:43.464565 [ 30 ] <Error> DB::StorageReplicatedMergeTree::queueTask()::<lambda(DB::StorageReplicatedMergeTree::LogEntryPtr&)>: Code: 33, e.displayText() = DB::Exception: Cannot read all data, e.what() = DB::Exception,

It is very similar to this issue on GitHub.

When I restart the first node it is able to load the new data inserted to the second node and seems to be working. However inserting some more data brings the same error again.

The most recent setup I tried:

Following the tutorial, I have a three node Zookeeper cluster with the following config:

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/opt/zoo2/data
clientPort=12181
server.1=10.201.1.4:2888:3888
server.2=0.0.0.0:12888:13888
server.3=10.201.1.4:22888:23888

The zookeeper config for ClickHouse loooks like this:

<?xml version="1.0"?>
<yandex>
    <zookeeper>
        <node>
            <host>10.201.1.4</host>
            <port>2181</port>
        </node>
        <node>
            <host>10.201.1.4</host>
            <port>12181</port>
        </node>
        <node>
            <host>10.201.1.4</host>
            <port>22181</port>
        </node>
    </zookeeper>
</yandex>

I create all tables like this:

CREATE TABLE t_r (
  id UInt32,
  d Date
) ENGINE = ReplicatedMergeTree('/clickhouse/tables/t_r', '03', d, (d, id), 8192);

The only difference accross all replicas is the replica id '03' which is set accordingly.

Thanks for any advice!

Upvotes: 2

Views: 4893

Answers (2)

Tom&#225;š Rep&#237;k
Tom&#225;š Rep&#237;k

Reputation: 155

Actually I figured out the issue by myself. Thanks to @egorlitvinenko I went through all the configs again and I noticed that for all three nodes I had set up the same interserver_http_port. It would not be problem if all the nodes were running on separate machines, but in my test scenario they run side by side hosted on the same OS.

Upvotes: 3

egorlitvinenko
egorlitvinenko

Reputation: 2776

ReplicatedMergeTree('/clickhouse/tables/t_r', '03', d, (d, id), 8192);

You should configure zookeeper unique id for each replicas. Currently you use '03', it is not correct. In tutorial, by {replica} means macros, which configured in clickhouse config file on each nodes.

See - https://clickhouse.yandex/docs/en/table_engines/replication.html#replicatedmergetree

p.s. For futher help, please, provide config of all nodes.

Upvotes: 0

Related Questions