CODEWITHSUNDEEP
scalaapache-kafkaapache-flinkflink-streaming
Adrian P
Adrian P

Reputation: 19

Flink with Kafka Consumer doesn't work

I want to benchmark Spark vs Flink, for this purpose I am making several tests. However Flink doesn't work with Kafka, meanwhile with Spark works perfect.

The code is very simple: 

val env: StreamExecutionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment

val properties = new Properties()
properties.setProperty("bootstrap.servers", "localhost:9092")
properties.setProperty("group.id", "myGroup")
println("topic: "+args(0))
val stream = env.addSource(new FlinkKafkaConsumer09[String](args(0), new SimpleStringSchema(), properties))
stream.print

env.execute()

I use kafka 0.9.0.0 with the same topics (in consumer[Flink] and producer[Kafka console]), but when I send my jar to the cluster, nothing happens:

Cluster Flink

What it could be happening?

Upvotes: 0

Views: 2667

Answers (2)

Sudhanshu Lenka
Sudhanshu Lenka

Reputation: 123

Your stream.print will not print in console on flink .It will write to flink0.9/logs/recentlog. Other-wise you can add your own logger for confirming output.

Upvotes: 2

Chesnay Schepler
Chesnay Schepler

Reputation: 1280

For this particular case (a Source chained into a Sink) the Webinterface will never report Bytes/Records sent/received. Note that this will change in the somewhat near future.

Please check whether the job-/taskmanager logs do not contain any output.

Upvotes: 0

Related Questions