CODEWITHSUNDEEP
exceptionapache-kafkaapache-nifi
CMPE
CMPE

Reputation: 1963

Kafka: How to fix TimeoutException error?

I am using Kafka With Nifi and when I am ingesting a large file (over 100MB). I am getting a TimeOutException error. Kafka does not crash. Reading about it, i need to increase the following property 'request.timeout.ms' that is set by default to 30ms. What should this property be set for with bigger files that takes longer time to be ingested? How can I calculate it? i am using Confluent Kafka 5.3.1 in a production environment.

Thank you

Upvotes: 0

Views: 278

Answers (1)

maxime G
maxime G

Reputation: 1771

With publishKafka and ConsumeKafka processors you can add dynamic properties to add kafka configuration to your consumer or producer. so you can add a property like this :
key : request.timeout.ms
value : the value you want

you can also configure back pressure in nifi connection to secure your environement.

you should check nifi app-logs, maybe your kafka is down.

Upvotes: 1

Related Questions