CODEWITHSUNDEEP
mqttmosquittomessagebroker
Tom Freudenberg
Tom Freudenberg

Reputation: 1367

Mosquitto with persitence as MQTT Forwarding Broker

We want to use Mosquitto MQTT as Message Broker on a number of OpenWRT gateways to forward "local" anonymous MQTT publications to a central RabbitMQ/MQTT cluster with authorization.

Our Problem:

Mosquitto does not reliable forward queued messages when restarted (persistance does not work)

Running mosquitto version: 1.4.15

current config

This is the last tested config:

port 1883
persistence true
persistence_file /mosquitto.db
persistence_location /etc/mosquitto

autosave_interval 1
autosave_on_changes true

allow_anonymous true

connection iotcluster
address ip:1883

notifications false
keepalive_interval 300
restart_timeout 30
start_type automatic

clientid rabbitmqtt
username user
password password

topic mqtt out 2
try_private true

As long as all systems running and online - everything works as expected - any message published locally via:

mosquitto_pub -h localhost -p 1883 -t mqtt -m "Test-Bridge-Online"

will be forwarded and published to the RabbitMQTT.

connection lost

When we produce a connection lost (e.g. disconnect cable) and re-establish that connection, the messages received in the meantime won't be automatically forward from Mosquitto to RabbitMQTT

BUT - when sending a new publish message in addtion - Mosquitto will send out also the queued messages ???

Mosquitto restart

If we restart the Mosquitto during the connection lost, the queued messages are all lost - NO PERSISTINACE options worked so long.

Please help

We tried different options of autosave_interval, QoS 0/1/2 ond other option combination - but in anyway - on restart Mosquitto - all messages are lost - NO PERSISTANCE in any way

Upvotes: 2

Views: 1921

Answers (2)

user681365
user681365

Reputation: 353

I have set up a complex network of mosquitto brokers bridged together. Persistence works like a charm on mosquitto if set up correctly. Note that on the eclipse version (free open source) the queues and retain messages are limited by the memory of the host. This is the persistent data we are talking about. So when turned on this data is persisted on disk at regular intervals in case the broker crashes or is restarted. And that works really well. You can limit this by conf to avoid crashing mosquitto by exceeding the memory of the host or limit by client connections.

The pro version of mosquitto from Cedalo from version 2.9.1 allows you to also persist active queues on disk to avoid the host memory limitations. This allows very larges queues of data on devices with limited memory. This is a feature we requested and their work is really awesome. We use mosquitto for very large amount of messages in an industrial context.

Upvotes: 0

Tom Freudenberg
Tom Freudenberg

Reputation: 1367

After running an update to mosquitto version 1.6.10 still no success.

I could isolate the main problem - the database did not save when seetings:

persistence true
persistence_file /mosquitto.db
persistence_location /etc/mosquitto

autosave_interval 1
autosave_on_changes true

Not sure why but could not manage to invoke a save of database by the above settings.

The database save on SIG-EXIT but not when pushing new message.

So I changed the settings to:

persistence true
persistence_file /mosquitto.db
persistence_location /etc/mosquitto

autosave_interval 300
autosave_on_changes false

and invoke a request to save the database after pushing the message:

# publish a new message
mosquitto_pub -h localhost -p 1883 -t mqtt -m "Test Offline 100" -q 1
# send signal to save database
killall -SIGUSR1 mosquitto

When using this - all kind of messages where queued in an state of the connection and get delivered.

That's the final configuration:

user mosquitto
port 1883

allow_anonymous true
queue_qos0_messages true

persistence true
persistence_file mosquitto.db
persistence_location /etc/mosquitto/
autosave_interval 300
autosave_on_changes false

connection ConName
address remote-ip:1883
bridge_protocol_version mqttv31
clientid ClientName
username user
password password
cleansession false
try_private false
retain_available false

start_type automatic
restart_timeout 60
keepalive_interval 120

notifications true
notifications_local_only true
notification_topic mqtt

topic mqtt out 1

Upvotes: 1

Related Questions