CODEWITHSUNDEEP
javaapache-beambeam
olaf
olaf

Reputation: 347

Proper way to call asynchronous api via beam in java?

I want to call api for each message that I consume from kafka to do some stateful calculation

pipeline.apply("Read message",
        KafkaIO.<MyMessage>read()
            .withTopic("MyTopic")
            .withCluster("MyCluster")
            .withoutMetadata())
    .apply("Filter input", Filter.by(message -> Objects.nonNull(message.getValue()))
    .apply("call api for each message", // What is the most efficient way to do this, is each api call blocking?)

But I don't want to block on each api call

If you call api within each apply in beam is it asychronous or it is blocking and try to exhaust all thread available?

An example of blocking and non blocking api call in apply would be appreciated.

Upvotes: 1

Views: 93

Answers (0)

Related Questions