CODEWITHSUNDEEP
rabbitmqmessage-queuemessagebrokercowboy
llama
llama

Reputation: 1651

RabbitMQ message size problems

We have been using RabbitMQ as a broker. It sits between our client written in Go with NGINX and the API server. We have used it successfully until now. The message size that we are using has increased to ~20mb. Sending this large message consistently causes the following error:

=ERROR REPORT==== 20-Oct-2018::11:17:59 === Error in process <0.21106.0> on node 'rabbit@pc11213' with exit value: {[{reason,{badmatch,{more,<<8001224 bytes>>,{http_req,#Port<0.99287>,ranch_tcp,keepalive,<0.21106.0>,<<4 bytes>>,'HTTP/1.1',{{127,0,0,1},58904},<<9 bytes>>,undefined,15672,<<39 bytes>>,undefined,<<0 bytes>>,[],[{exchange,<<12 bytes>>},{vhost,<<3 bytes>>}],[{<<4 bytes>>,<<15 bytes>>},{<<10 bytes>>,<<18 bytes>>},{<<14 bytes>>,<<8 bytes>>},{<<13 bytes>>,<<26 bytes>>},{<<12 bytes>>,<<16 bytes>>},{<<15 bytes>>,<<4 bytes>>}],[{<<14 bytes>>,20432940},{<<17 bytes>>,[<<8 bytes>>]},{<<6 bytes>>,undefined},{<<12 bytes>>,{<<11 bytes>>,<<4 bytes>>,[]}},{<<17 bytes>>,undefined},{<<13 bytes>>,undefined},{<<19 bytes>>,undefined...

We are using the HTTP management endpoint and plugins:

=INFO REPORT==== 19-Oct-2018::14:28:44 === Server startup complete; 6 plugins started.
  * rabbitmq_management
  * rabbitmq_web_dispatch
  * rabbitmq_management_agent
  * cowboy
  * cowlib
  * amqp_client

We tried changing the rabbitMQ/ cowboy server config to no avail e.g.

[{rabbitmq_management,
   [{listener, [{port,        15672},
                {cowboy_opts, [{ max_request_line_length, 16000 },{max_keepalive, 1000}]}
               ]}
   ]
}].

So our feeling is we either need to move away from rabbitMQ or use some other protocol with a different plugin.

We feel rabbitMQ should not be limited on message size. We would rather not try to reference a payload in the message. Any suggestions as to how we can increase the tolerated message size?

Edit: We tried further modifications to cowboy settings:

{rabbitmq_management,
     [{cors_allow_origins,[]},
      {cors_max_age,1800},
      {http_log_dir,none},
      {listener,
          [{port,15672},
           {cowboy_opts,
               [{compress,true},
                {max_request_line_length,160000},
                {max_keepalive,1000},
                {idle_timeout,120000},
                {inactivity_timeout,120000},
                {request_timeout,120000}]}]},
      {load_definitions,none},
      {management_db_cache_multiplier,5},
      {process_stats_gc_timeout,300000},
      {stats_event_max_backlog,250}]},

Which produced the same error:

2018/10/24 16:31:06 Size of file listing:20431558 2018/10/24 16:31:06 Extracted pid:20.# 2018/10/24 16:31:07 Post http://derived:#@localhost:#/api/exchanges/%2F/derived-exchange/publish: net/http: HTTP/1.x transport connection broken: write tcp 127.#->127.#: write: connection reset by peer

Upvotes: 1

Views: 4751

Answers (1)

Giri Chintala
Giri Chintala

Reputation: 130

it looks error is misleading. ideally rabbitmq shouldn't limit on message size. But HTTP API may have restrictions.

RabbitMQ (in fact any queueing solution) is not ideal for exchanging large size messages. Instead store large size message in a Object storage (AWS s3) and publish stored objects meta data to queue.

Other options ...

  1. Try using AMQP to exchange large size messages with RabbitMQ.

  2. Try increasing cowboy_opts --> request_timeout (milli seconds).

           {cowboy_opts, [{compress, true},
                      %% 120 seconds
                      {idle_timeout,      120000},
                      {inactivity_timeout,120000},
                      {request_timeout,   120000}]}
      ]}

Upvotes: 1

Related Questions