Reputation: 1367
We want to use Mosquitto MQTT as Message Broker on a number of OpenWRT gateways to forward "local" anonymous MQTT publications to a central RabbitMQ/MQTT cluster with authorization.
Mosquitto does not reliable forward queued messages when restarted (persistance does not work)
Running mosquitto version: 1.4.15
This is the last tested config:
port 1883
persistence true
persistence_file /mosquitto.db
persistence_location /etc/mosquitto
autosave_interval 1
autosave_on_changes true
allow_anonymous true
connection iotcluster
address ip:1883
notifications false
keepalive_interval 300
restart_timeout 30
start_type automatic
clientid rabbitmqtt
username user
password password
topic mqtt out 2
try_private true
As long as all systems running and online - everything works as expected - any message published locally via:
mosquitto_pub -h localhost -p 1883 -t mqtt -m "Test-Bridge-Online"
will be forwarded and published to the RabbitMQTT.
When we produce a connection lost (e.g. disconnect cable) and re-establish that connection, the messages received in the meantime won't be automatically forward from Mosquitto to RabbitMQTT
BUT - when sending a new publish message in addtion - Mosquitto will send out also the queued messages ???
If we restart the Mosquitto during the connection lost, the queued messages are all lost - NO PERSISTINACE options worked so long.
We tried different options of autosave_interval
, QoS 0/1/2
ond other option combination
- but in anyway - on restart Mosquitto - all messages are lost - NO PERSISTANCE in any way
Upvotes: 2
Views: 1921
Reputation: 353
I have set up a complex network of mosquitto brokers bridged together. Persistence works like a charm on mosquitto if set up correctly. Note that on the eclipse version (free open source) the queues and retain messages are limited by the memory of the host. This is the persistent data we are talking about. So when turned on this data is persisted on disk at regular intervals in case the broker crashes or is restarted. And that works really well. You can limit this by conf to avoid crashing mosquitto by exceeding the memory of the host or limit by client connections.
The pro version of mosquitto from Cedalo from version 2.9.1 allows you to also persist active queues on disk to avoid the host memory limitations. This allows very larges queues of data on devices with limited memory. This is a feature we requested and their work is really awesome. We use mosquitto for very large amount of messages in an industrial context.
Upvotes: 0
Reputation: 1367
After running an update to mosquitto version 1.6.10 still no success.
I could isolate the main problem - the database did not save when seetings:
persistence true
persistence_file /mosquitto.db
persistence_location /etc/mosquitto
autosave_interval 1
autosave_on_changes true
Not sure why but could not manage to invoke a save of database by the above settings.
The database save on SIG-EXIT but not when pushing new message.
So I changed the settings to:
persistence true
persistence_file /mosquitto.db
persistence_location /etc/mosquitto
autosave_interval 300
autosave_on_changes false
and invoke a request to save the database after pushing the message:
# publish a new message
mosquitto_pub -h localhost -p 1883 -t mqtt -m "Test Offline 100" -q 1
# send signal to save database
killall -SIGUSR1 mosquitto
When using this - all kind of messages where queued in an state of the connection and get delivered.
That's the final configuration:
user mosquitto
port 1883
allow_anonymous true
queue_qos0_messages true
persistence true
persistence_file mosquitto.db
persistence_location /etc/mosquitto/
autosave_interval 300
autosave_on_changes false
connection ConName
address remote-ip:1883
bridge_protocol_version mqttv31
clientid ClientName
username user
password password
cleansession false
try_private false
retain_available false
start_type automatic
restart_timeout 60
keepalive_interval 120
notifications true
notifications_local_only true
notification_topic mqtt
topic mqtt out 1
Upvotes: 1