CODEWITHSUNDEEP
apache-zookeeperspring-xd
Will Milligan
Will Milligan

Reputation: 3

Is Zookeeper Essential For XD Single Node?

This question relates to the XD 1.1.2.RELEASE. I'm brand new to XD, so please forgive if this is a dumb question.

The documentation says only that XD does not ship with Zookeeper, and I took this to mean that it would only be needed once I'm ready to go multi-node.

That being said, I'm getting some undesired behavior in trying to deploy a stream: 1. Any time I add a stream, it is not persistent across XD restarts. 2. The stream setup is timing out. In debug mode, it's getting stuck in ModuleDeploymentWriter in spring-dirt. If I'm reading that class correctly, it seems like it's writing a file to be read by Zookeeper and then timing out when a response is never received.

At first I thought it was my custom Cassandra XD sink connection that was timing out, but it doesn't seem like that code is ever reached in the first place.

Any help appreciated!

Upvotes: 0

Views: 314

Answers (2)

Ilayaperumal Gopinathan
Ilayaperumal Gopinathan

Reputation: 4179

The documentation says only that XD does not ship with Zookeeper, and I took this to mean that it would only be needed once I'm ready to go multi-node.

That's right. But, XD single-node uses embedded ZooKeeper.

  1. Any time I add a stream, it is not persistent across XD restarts.

Are you using singlenode or distributed mode? If you are using singlenode, everytime XD restarts it uses different ZooKeeper instance. Hence, in singlenode, you can't get the streams persisted between server restart. You can still override this functionality by providing external ZK configuration for single-node by setting zk.client.connect in servers.yml.

  1. The stream setup is timing out.

Some log messages or stack trace could help figure out what's going on here. If you are using single-node, the ModuleDeploymentWriter should write the deployment to the container that's running on the same JVM.

In case if you are running on the debug mode like @dturanski mentioned, you can try increasing deployment timeout property xd.admin.deploymentTimeout in servers.yml. The default is 30s.

Upvotes: 1

dturanski
dturanski

Reputation: 1723

You don't need external Zookeeper to run in singlenode since it starts an embedded server. This keeps state in memory, so you lose state between restarts. However, you can configure it to connect to an external Zookeeper ensemble.

If "debug modue" refers to running SingleNodeApplication with a debugger, then yes it could time out if you set a breakpoint.

Upvotes: 0

Related Questions