CODEWITHSUNDEEP
javaapache-kafkaapache-storm
user2611300
user2611300

Reputation: 133

Apache Storm not receiving messages from kafka

I am trying to integrate Apache Storm with Kafka. The connection seems to be established fine but not receiving any messages. But the messages seems to have been sent to Kafka servers also while the index file for the respective topic in Kafka server shows some data is present. Is there a way to debug this more on Storm end..? I am using a customer decoder in Storm for the messages. The implementation for Storm is:

TopologyBuilder builder = new TopologyBuilder();
Broker brokerForPartition0 = new Broker("xxxxx");
GlobalPartitionInformation partitionInfo = new GlobalPartitionInformation();
partitionInfo.addPartition(0, brokerForPartition0);
StaticHosts hosts = new StaticHosts(partitionInfo);
SpoutConfig spoutConfig = new SpoutConfig(hosts, TOPIC, "/"+TOPIC, clientId);
spoutConfig.scheme = new MyLogScheme();
builder.setSpout("spout", new KafkaSpout(spoutConfig));
builder.setBolt("printer", new PrinterBolt());
Config conf = new Config();
conf.setDebug(true);
LocalCluster cluster = new LocalCluster();
cluster.submitTopology("test", conf, builder.createTopology());

Upvotes: 2

Views: 893

Answers (2)

syed jameer
syed jameer

Reputation: 454

Check the topic name in the main Storm Topology. That both kafka and storm topics are same. Check the brokers in kafka topic

Upvotes: 0

redTiger
redTiger

Reputation: 322

You are using your own scheme "MyLogScheme". The issue might be in myLogScheme. Try using the default scheme which is StringScheme. Hopefully that will let you see some messages.

Upvotes: 1

Related Questions