CODEWITHSUNDEEP
apache-kafkaapache-kafka-connectkafka-producer-api
Navaneetha Krishnan Ramagopal
Navaneetha Krishnan Ramagopal

Reputation: 23

Kafka Connect 5.5.0 - Unable to reset max.request.size

In confluent-5.5.0 - I am unable to change the max.request.size , which always defaults to max.request.size = 1048576 in the ProducerConfig.

The following are the parameters I have already tried with noluck:

confluent-5.5.0/etc/kafka/producer.properties

  max.request.size=15728640
  producer.max.request.size=15728640

confluent-5.5.0/etc/kafka/server.properties

  message.max.bytes=15728640
  replica.fetch.max.bytes=15728640
  max.request.size=15728640
  fetch.message.max.bytes=15728640

/data/confluent-5.5.0/etc/kafka/consumer.properties

  max.partition.fetch.bytes=15728640

confluent-5.5.0/etc/kafka-rest/kafka-rest.properties

  max.request.size=15728640

NOTE : None of these values is getting updated in the connect.log

  1. I have stop/started confluent-5.5.0 , even destroyed the previous images and restarted.

Am i missing something ?

The following i have also tried after the information from comment :

/data/confluent-5.5.0/etc/kafka/connect-standalone.properties

producer.override.max.request.size=15728640 consumer.override.max.partition.fetch.bytes=15728640

/data/confluent-5.5.0/etc/kafka/connect-distributed.properties

producer.override.max.request.size=15728640 consumer.override.max.partition.fetch.bytes=15728640

Still in the max.request.size has not got changed.

( Solved )Based on the inputs :

I have added the above configuration in the connect or configuration. And also changed the policy from none to ALL. Which applied the configuration changes properly.

Upvotes: 2

Views: 5300

Answers (2)

Zak_Stack
Zak_Stack

Reputation: 137

the solution is to set configuration in kafka connect proprties file : add the following in distributed or standalone connect properties file

producer.max.request.size=157286400
consumer.max.request.size=157286400
max.request.size=157286400

and it will work !!!

Upvotes: 1

OneCricketeer
OneCricketeer

Reputation: 192033

Those files are not used by Connect.

  • server is for the Apache Kafka Broker only
  • consumer|producer are for the kafka-console utilities
  • kafka-rest is for the Confluent REST Proxy only

You need to use connect-distributed.properties or connect-standalone.properties and notice that you need to additionally set the property correctly using prefixes.

Upvotes: 5

Related Questions