Mosquitto Bridge and Broker setup questions

Mosquitto MQTT Broker
1

Hi all, I have not found too much in the forums about Mosquitto bridge setup. Which might mean its really easy and no one has any issues :wink:

However I have been going through trying to set up a Broker and Bridge so two independent home assistant instances can share a small number of topics. Unfortunately Iā€™m finding its not working for me yet.

Tiny bit of background for WHY use a bridge. I need to reduce and tightly control bandwidth use. Every bit and byte counts. The connection will be over mobile data with a low cost IOT sim card. So Iā€™m looking to regulate the bit count carefully.

I have two Mosquitto test installs up and running. one on a Raspberry pi the other on a VM. MQTT Explorer connects perfect in each case.

A topic has been created on each eg homeassistant/sensor/test
homeaassistant/sensor/test2 and some small value added as payload.

On the bridge side:

The mosquitto.conf has a valid username and password (tested in MQTT explorer and the same values for both bridge and broker)
The IP address is for a VPN connection. (Tcailscale. And I can see the port 1883 is used. I cant see packets but its noted as a used port by both installs)

Customize section is set like this:

active: true
folder: mosquitto
logins:
  - username: fred
    password: "<pass>"
certfile: fullchain.pem
keyfile: privkey.pem
require_certificate: false

I have the password in quotes " " since it has a special character in it that may get interpreted strange.

mosquitto.conf has this:

connection bridge-1
address 100.127.132.116:1883
topic # out 0
topic # in 0 
remote_username fred
remote_password "<pass>"
allow_anonymus true

log_type debug

I have confirmed the mosquitto.conf file is being read. Logs tell me and I can change the log level to debug etc.

on the Broker side I have a the same Customize section as above.
Again the mosquitto.conf file is being read as I can see in the logs and I can change the log level.
The conf file on the Broker side just has:

 log_type debug

The acl.conf file has :

acl_file /share/mosquitto/accesscontrollist

and accesscontrollist has :

user mqtt
topic reaadwrite #

user addons
topic readwrite #

user fred
topic topic readwrite #

user homeassistant
topic readwrite #

I do not see the type of connection log records for fails or success I would like to see. I dont know what they should look like exactly but I would expect to see something referencing one instance being connected to by the other or attempting and failing.

I do not see the topics from on appearing on the other either. Using MQTT explorer to watch this.

What am I missing ?

Thanks for anyone with the patience to look over this.

2

Hello,

I would suggest trying to simplify the setup to try and isolate the problem. What youā€™ve posted looks fine in principle (although there are mistakes like allow_anonymus which I presume arenā€™t in your real setup), but the details matter of course.

I would try setting up a bridge to test.mosquitto.org in the first instance. That removes one half of the problem.

You could use this as your config:

listener 1883
allow_anonymous true

connection tmo
address test.mosquitto.org:1883
topic DerfMan/# both

log_type all

Running mosquitto with that config should give you log results similar to this:

Bridge local.dev.tmo doing local SUBSCRIBE on topic #
Connecting bridge tmo (test.mosquitto.org:1883)
Bridge dev.tmo sending CONNECT
mosquitto version 2.1.0 running
Received CONNACK on connection local.dev.tmo.
Bridge local.dev.tmo sending UNSUBSCRIBE (Mid: 2, Topic: DerfMan/#)
Received PUBACK from local.dev.tmo (Mid: 1, RC:0)
Received UNSUBACK from local.dev.tmo

You could then try subscribing to your local instance and t.m.o and observing messages:

mosquitto_sub -t '#' -v
mosquitto_sub -h test.mosquitto.org -t 'DerfMan/#' -v

And:

mosquitto_pub -t test -m "should appear on the local subscriber only"
mosquitto_pub -t DerfMan/one -m "should appear on both subscribers"
mosquitto_pub -h test.mosquitto.org -t DerfMan/two -m "should appear on both subscribers"

If that all works, try reconfiguring your bridge to connect to your remote broker - but have the remote broker running without authentical / acl_file. You now know what the log should look like, so you can compare what you get in this situation, and likewise rerun the subscriber/publisher tests to check it works as you expect.

Next step would be to add authentication to both sides and recheck, then finally the ACLs.

If you run through that and find a problem you donā€™t know how to fix, then give a reply and we can sort it out.

Cheers,

Roger

3

That is something to chew on for sure, thanks.
Specifically for simplifying, I have intended to remove the VPN in my tests and just run two virtual machines with the Home assistant OS + MQTT. I like the test.mosquitto.org idea too. I had not explored that thought before.
Regarding the ā€œallow_anonymous trueā€, yes, just in my test setup and only since I am pulling my hair out trying to understand the blocker.
Thanks for the replay.
I will try to be that guy who comes back with the ā€œeurekaā€ answer.

4

First odd update.
I did a fresh install. modified the mosquitto.conf and restart the mosquitto.
I clicked though to see the logs and noticed that before the reboot it had picked up the mosquitto.conf and was giving the log you suggested would come up.
ā€œBridge local.dev.tmo doing local SUBSCRIBE on topic #
Connecting bridge tmo (test.mosquitto.org:1883) ā€¦ etcā€

However. The service restart happened and then I get this error
Error: Address already in use
Apparently this means that mosquitto is already running and occupying the port or another service is engaged on it.
This error is followed by a ā€œService legacy-services: stoppingā€ and a few more log entries suggesting things are stopping. no more log activity
I rebooted the vm completely and tried again. Same error.
Iā€™ll try a few more fresh installs and see if I can make it behave slightly differently.

EDIT:
It feels like the /share/mosquitto/mosquitto.conf is read very quickly before the stop cycles are started in a restart.
So the listener 1883 is started and iit all kicks off only to be killed by the stop / restarat signal.
On the start cycle the /share/mosquitto/mosquitto.conf file is read before the etc version of the filemosquitto.conf. The result seems to be that the listener 1883 line is loaded from my share/mosquitto/mosquitto.conf locks the port and stops the etc version of the file from doing its thing !! Its just a theory.
To put a little confirmation on this I first removed the listener 1883 line and checked the logs. No error
Then I put the listener 1883 line back and got the error ( I also got the short lived good log lines)
then I changed it to listener 2883. This gave the same log result as removing the line.

5

I just went through a whole write up of an error I got, and then managed to delete it all by mistake while editing it. !! not exactly sure how.

Short version.
There looks to be a bug in the implementation in some form.
Using your suggested mosquitto.conf file I got the expected log line very briefly during the inital stop part of a restart. It seems the conf file is loaded briefly during the stop.
However on start its also loaded before the etc stored version of the mosquitto.conf file and this is possibly causing a conflict for some reason.
I get an error ā€œError: Address already in useā€
Apparently this suggests that the port 1883 is already being used. It causes a stop of some things and no connection happens !!
I tried removing and modifying and then putting the ā€œlistener 1883ā€ line back. Its not happy with that line. !!

6

I just went through a whole write up of an error I got, and then managed to delete it all by mistake while editing it. !! not exactly sure how.

Short version.
There looks to be a bug in the implementation in some form.
Using your suggested mosquitto.conf file I got the expected log lines very briefly during the initial stop part of a restart. It seems the custom mosquitto.conf file is loaded briefly during the stop.
However on start its also loaded but before the etc stored version of the mosquitto.conf file and this is possibly causing a conflict for some reason.
I get an error ā€œError: Address already in useā€
Apparently this suggests that the port 1883 is already being used. It causes a stop of some things and no connection happens !!
I tried removing and modifying and then putting the ā€œlistener 1883ā€ line back. Its not happy with that line. !!

EDIT. The forum bot has sensed my activity and decided Iā€™m strange and a possible spam bot. So my post is not getting seen by others until a human reads it. I feel mixed emotions about being noticed by a bot and then flagged for ignoring. life will continue.
So Iā€™ll update here anyway.

Home assistant provides aa few GUI options for setting variables. Simplifying the things that 90% of users need for IOT purposes.
One of the settings is the ā€œNormal MQTTā€ listener port which logically defaults to 1883. I dont know where its saved to but Iā€™m currently thinking its in the Mosquito MQTT Docker container and out of arms reach. (Iā€™m using the Home assistant OS install)

With this in mind I decided to try mixing and matching the custom mosquitto.conf file settings with the available GUI options for settings.
Remove the"listener 1883" line in the Mosquitto.conf file (confirm its in the GUI setting with 1883)
Add the subscribe topic topic ā€œDerfMan/# bothā€ in the GUI option (actually the GUI has one space for subscribe and one for publish. So I used both. I also left that line in the mosquitto.conf)
The publish topic topic ā€œDerfMan/testā€ in the GUI option ( it will not allow wild cards)

The result was a tangle I think.

Then I tried again but simplifying things. Ignore/clear the GUI settings for publish and subscribeā€¦ Remvre the listener 1833 line from your suggested .conf file. I added a topic from the test.mosquitto.org MQTT server into the .conf file to subscribe to. Something with regularly changing data. /Testing/Traffic/Paddy/House/# (thank you whoever you are)
I now see that new data in my local MQTT server.
I then used the home assistant GUI tool to publish a new value for DerfMan and it appeared on test.mosquitto.org

I call that a eureka moment.
Whats different !! I could not tell you at this point.
I will do a fresh install and build to replicate and confirm. (Vm snapshots are great) Then build up all the other bits and see what, if anything, breaks.

7

Instead of starting from scratch I decided I would clone the Home Assistant VM and get the two talking to each over over the LAN.
It worked easily. Just changed the address value on one and added the ā€œremote_usernameā€ and ā€œremote_passwordā€ values to the .conf

Then I added taailscale. No big cigar moment there unfortunatluy.
I probably need to figure out adding a rout to get traffic out over the VPN. Cant be sure yet.