unable to send large files to Kafka broker | Error message: The request included a message larger than the max message size the server will accept

Question

I am unable to send large file to the broker, I have edited the configuration files and inside my client program but it didn't work.

• Error Message

  The request included a message larger than the max message size the server will accept.

• Server properties

  message.max.bytes=900000000
  replica.fetch.max.bytes=900000000
  

• Producer

  max.request.size=100000000 
  

• Consumer

  max.partition.fetch.bytes=100000000
  fetch.message.max.bytes=100000000
  

• Client app

  props.put("max.request.size",96214400);
  props.put("message.max.bytes",96214400);
  

Answer 1

Moving comment to answer...

The message.max.bytes is a cluster-wide setting and you need to make sure that the corresponding max.message.bytes configuration is also set at topic-level. As the topic-level configuration defaults to 1048588 you are seeing this error.

Answer 2

kafka-topics.bat --create --zookeeper 127.0.0.1:2181 --replication-factor 1 --partitions 1 --topic test --config max.message.bytes=9000000