# 10. Bridging with MQTT brokers
###### tags: `corsounipd2022`
> based on http://www.steves-internet-guide.com/mosquitto-bridge-configuration/
:::info
The code of this section is [here ](https://www.dropbox.com/sh/u73b82rr9crtzwt/AAALzsmqKDWiGLGu4wqW5eqga?dl=0)
:::
An MQTT broker can be executed as a Docker container. This fact provides flexibility when deploying these services, especially in edge devices, like Raspberry Pi or similar.
Searching for MQTT, you'll get:
```
$ docker search mqtt
NAME DESCRIPTION STARS OFFICIAL AUTOMATED
eclipse-mosquitto Eclipse Mosquitto is an open source message … 899 [OK]
ncarlier/mqtt MQTT Docker image based on Debian. 34 [OK]
efrecon/mqtt-client Minimal MQTT clients based on mosquitto 10 [OK]
dersimn/mqtt-admin Docker image for https://github.com/hobbyqua… 5 [OK]
...
```
The first one, [eclipse-mosquitto](https://hub.docker.com/_/eclipse-mosquitto), is the official image for the Mosquitto MQTT broker, probably the most widely used MQTT broker.
The third one, [efrecon/mqtt-client](https://hub.docker.com/r/efrecon/mqtt-client) is also a helpful image that allows having a handy client to test deployments.
## Starting an MQTT broker as a container
The `docker-compose.yml` file below indicates how to start up an MQTT broker locally:
```yaml=
version: '3'
networks:
pubsub-net:
driver: bridge
services:
mosquitto1:
image: eclipse-mosquitto
container_name: broker1
ports:
- 1883:1883
volumes:
- ./mosquitto:/mosquitto/
networks:
- pubsub-net
```
Executing `docker compose up` you will get something like:
```
$ docker compose up
[+] Running 1/0
⠿ Container broker1 Created 0.0s
Attaching to broker1
broker1 | 1644776311: mosquitto version 2.0.14 starting
broker1 | 1644776311: Config loaded from /mosquitto/config/mosquitto.conf.
broker1 | 1644776311: Opening ipv4 listen socket on port 1883.
broker1 | 1644776311: Opening ipv6 listen socket on port 1883.
broker1 | 1644776311: mosquitto version 2.0.14 running
```
giving:
```
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5f94617ba10f eclipse-mosquitto:latest "/docker-entrypoint.…" About a minute ago Up About a minute 0.0.0.0:1883->1883/tcp broker1
```
this is obtained with a `./mosquitto/config/mosquitto.conf` file as simple as this:
```
allow_anonymous true
listener 1883
```
We can do a quick test by opening two terminals and executing:
```
$ docker run -it --rm --network=host efrecon/mqtt-client sub -t 'test/#'
```
or
```
docker run -it --rm --net=container:broker1 efrecon/mqtt-client sub -t 'test/#'
```
in one of them (_check the use of `--network=...`_), and:
```
$ docker run -it --rm --network=host efrecon/mqtt-client pub -t "test/pietro" -m "proceditrasmissione"
```
in the other. We basically created a subscriber and a publisher containers that are talking through our broker.
---
## Mosquitto MQTT Bridge -Usage and Configuration
An MQTT bridge lets you connect two MQTT brokers together. They are generally used for sharing messages between systems.
A common usage is to connect and **edge** MQTT brokers to a central or remote MQTT network. A Mosquitto broker (server) can be configured to work as an MQTT bridge.
Generally the local edge broker will only bridge a subset of the local MQTT traffic.
You only need to configure one of the brokers to act as the bridge, the other will act as a normal broker.
### How an MQTT Bridge Works
When you configure the Mosquitto broker to act as a bridge it becomes an MQTT client to the remote broker and subscribes and publishes to topics on the other broker just like a normal MQTT client.
### Configuring the Broker as a Bridge
You will need to configure
* The Address and port of the remote broker or brokers. – Address keyword
* A client name. – Connection keyword
* What topics the broker will publish, and which topics it will subscribe to. -topic keyword
* Remapping if needed.
Note: You can configure multiple remote broker addresses and the broker will switch to another broker if the current broker connection fails.
### Topic Bridging
What topics are being bridged, and if they are being remapped is controlled by the topic entry. Wildcards can be used in the topic entry.
The general format is
```
topic 'topic pattern' direction QOS 'local prefix'/'remote prefix'
```
`direction` can be:
* out = publish from the broker
* in = receive from remote broker
* both = publish and receive
The `mosquitto.conf` below shows how to bridge all topics with no remapping.
```shell=
# connection <name>
connection bridge-01
address 192.168.1.184:1883
topic # out 0
topic # in 0
try_private false
```
:::info
try_private [ true | false ]
If try_private is set to true, the bridge will attempt to indicate to the remote broker that it is a bridge not an ordinary client. If successful, this means that loop detection will be more effective and that retained messages will be propagated correctly. Not all brokers support this feature so it may be necessary to set try_private to false if your bridge does not connect properly.
:::
Note: If you aren’t using topic prefixes then you don’t need to add them.
### More examples
Supponsing the configuration below, will see some example
**Example 1:**
Configuration file topics
```
topic house/# out
```
Result:
* client 1 publishes on `house/sensor1` or `house/anything` and the message is sent to broker 2
* client 1 publishes on `room/sensor1` and the message is not sent to broker 2
* client 2 publishes to any topic and nothing is sent to broker B1.
**Example 2:**
Configuration file topics
```
topic house/# in
```
Result:
* client1 publishes to any topic and nothing is sent to broker B2.
* client 2 publishes on `house/sensor1` or `house/anything` and the message is sent to broker B1
* client 2 publishes on `room/sensor1` and the message is not sent to broker B1
**Example 3:**
```
topic house/sensor1 both 0 “” “”
```
or
```
topic house/sensor1 both 0
````
:::info
The entries:
`topic house/sensor1 in 0 “” “”`
`topic house/sensor1 out 0 “” “”`
Are equivalent to the single entry:
`topic house/sensor1 both 0 “” “”`
and also
`topic house/sensor1 both 0`
:::
Result:
* client 1 publishes on house/sensor1 and the message is sent to broker B2
* client 2 publishes on house/sensor1 and the message is sent to broker B1
* client 1 publishes on house1/sensor1 and the message is not sent to broker B2
* client 2 publishes on house1/sensor1 and the message is not sent to broker B1
### Example with Remapping
Remapping is obtained using the local and remote prefixes.
For example, when the bridge on broker1 subscribes to a topic on broker2 it uses the form:
`remote_prefix + topic name` e.g. `b2/house/sensor1`
When the bridge on broker1 subscribes to a topic on broker1 it uses the form:
`local_prefix + topic name` e.g. `b1/house/sensor1`
The `mosquitto.conf` below shows a remapping configuration for broker 1:
```shell=
# conneciton <name>
connection bridge-01
address 192.168.1.184:1883
topic # out 0 "" b1/
topic # in 0 b2/ ""
```
With the above configuration then:
* When Client1 publishes on any topic that message will be sent to broker 2 with the topic prefix b1/. Example incoming client message to B1 on topic `house/sensor` is published to broker 2 on topic `b1/house/sensor`.
* When a client publishes a message to broker2 then messages are sent from broker2 to broker 1 but with the b2/ prefix.
Broker2 is functioning as a normal broker (not a bridge) and doesn’t need any extra configuration.
## A complete example with flespi
In this case we will configure the local brocker running as a container to bridge with a cloud broker, more specifically, [flespi](https://flespi.com).
### First example
We have to modify the `mosquitto.conf` file as indicated below. This is the most basic configuration where everything is forwarded between the two brokers.
```shell=
allow_anonymous true
listener 1883
connection bridge-flespi
address mqtt.flespi.io:1883
remote_username 6bR9Nubyfb02SuxCI7x0pS3hCRgqrt0nsFn5Xmr0ogS4Agw6dRXtxQyZvEfMV3Yc
topic # both 0
try_private false
```
Executing `docker compose up` now, we obtain a sligthly differente result:
```
$ docker compose up
[+] Running 1/0
⠿ Container broker1 Created 0.0s
Attaching to broker1
broker1 | 1644831547: mosquitto version 2.0.14 starting
broker1 | 1644831547: Config loaded from /mosquitto/config/mosquitto.conf.
broker1 | 1644831547: Opening ipv4 listen socket on port 1883.
broker1 | 1644831547: Opening ipv6 listen socket on port 1883.
broker1 | 1644831547: Connecting bridge bridge-flespi (mqtt.flespi.io:1883)
broker1 | 1644831547: mosquitto version 2.0.14 running
```
We can do a quick test by opening two terminals and executing:
```
$ docker run -it --rm --network=host efrecon/mqtt-client sub -t 'test/#'
```
in one of them, and:
```
$ docker run -it --rm --network=host efrecon/mqtt-client pub -h mqtt.flespi.io -u 6bR9Nubyfb02SuxCI7x0pS3hCRgqrt0nsFn5Xmr0ogS4Agw6dRXtxQyZvEfMV3Yc -t "test/pietro" -m "proceditrasmissione"
```
in the other.
The simmetrical scenario is as follows:
```
$ docker run -it --rm --network=host efrecon/mqtt-client pub -t 'test/pietro' -m 'the other way'
```
and:
```
$ docker run -it --rm --network=host efrecon/mqtt-client sub -h mqtt.flespi.io -u 6bR9Nubyfb02SuxCI7x0pS3hCRgqrt0nsFn5Xmr0ogS4Agw6dRXtxQyZvEfMV3Yc -t "test/#"
```
### Second example
We have to modify the `mosquitto.conf` file as indicated below. In this case we are testing a scenario where a remapping of the topics is preformed:
```shell=
allow_anonymous true
listener 1883
connection bridge-flespi
address mqtt.flespi.io:1883
remote_username 6bR9Nubyfb02SuxCI7x0pS3hCRgqrt0nsFn5Xmr0ogS4Agw6dRXtxQyZvEfMV3Yc
topic # out 0 "" b1/
topic # in 0 b2/ ""
try_private false
```
We can test the first rule by opening two terminals and executing:
```
$ docker run -it --rm --network=host efrecon/mqtt-client sub -t 'test/#'
```
in one of them, and:
```
$ docker run -it --rm --network=host efrecon/mqtt-client pub -h mqtt.flespi.io -u 6bR9Nubyfb02SuxCI7x0pS3hCRgqrt0nsFn5Xmr0ogS4Agw6dRXtxQyZvEfMV3Yc -t "test/pietro" -m "proceditrasmissione"
```
in the other. Now, nothing will appear... we have to modify the local subscriber as follows:
```
$ docker run -it --rm --network=host efrecon/mqtt-client sub -t 'b2/test/#'
```
to get the data.
The simmetrical scenario is as follows:
```
$ docker run -it --rm --network=host efrecon/mqtt-client pub -t 'test/pietro' -m 'the other way'
```
and:
```
$ docker run -it --rm --network=host efrecon/mqtt-client sub -h mqtt.flespi.io -u 6bR9Nubyfb02SuxCI7x0pS3hCRgqrt0nsFn5Xmr0ogS4Agw6dRXtxQyZvEfMV3Yc -t "b1/test/#"
```
### Bridging to Multiple Brokers- Round Robin
It is possible to bridge to another broker in the event of a broker failure.
To do so you need to set multiple `address:por` entries in the address field i.e
```
address 192.168.1.168:1883, 192.168.1.185:1883
```
will use broker 192.168.1.185 if the connection to the first broker (192.168.1.168) fails.
Here is the description taken from the online manual:
:::info
round_robin [ true | false ]
If the bridge has more than one address given in the address/addresses configuration, the round_robin option defines the behaviour of the bridge on a failure of the bridge connection. If round_robin is false, the default value, then the first address is treated as the main bridge connection. If the connection fails, the other secondary addresses will be attempted in turn. Whilst connected to a secondary bridge, the bridge will periodically attempt to reconnect to the main bridge until successful.
If round_robin is true, then all addresses are treated as equals. If a connection fails, the next address will be tried and if successful will remain connected until it fails.
:::
**Looping**
Publishing and subscribing to the same topic on the broker could result in a loop.
However if you try this you find that a broker detects that another broker sent the message, and doesn’t send the message back to it even though the broker has subscribed to that topic.
This behaviour is not 100% clear, since no reference for broker detection can be found in the specs.
:::danger
Exercise:
Modify the `docker-compose.yml` file to start 2 brokers locally that perform **full** bridging among them
:::
Sign in with Wallet
Connect another wallet