Your Own MQTT Broker

Like many folks, I too started out using the public MQTT broker at test.mosquitto.org. It’s a great way to get started – simple, easy to get working, and FREE! But it does not take long to realize it is unsecured. Anyone can listen in on your topics and there are no logon credentials required or offered as an option.

So I got to searching for a better broker…

One with security. And all the options available with the MQTT standard. Things like:

• Security Authentication (passwords,certificates)
• Simultaneous websockets and mqtt (tcp) listeners
• Persistent Messages

But I did not want to pay for the service. The obvious solution was to host your own broker, either on a host server, or on your local network with broadband access via a router.

Since my host server does not permit continuously running scripts or programs, was limited to a local network solution. But with a broadband connection, it would be on-line and accessible anywhere.

After some research, the most promising options included:

• PC Broker with Windows OS
• PC Broker with Linux OS
• Flash Driver Linux Distro Broker
• Raspberry Pi2 Broker
• Android Device Broker
• Embedded micro-controller

I have read numerous comments about poor (slow) performance using a Raspberry Pi, and since I do not own one, that option was ruled out. For the same reason, I thought about hosting a broker on the trusty ESP8266 but decided against it, at least for now.

And while it would be great to use an old Android phone as an MTTQ broker, the path to get there was a bit murkier than using a Linux hosted server.  It can be done, but few have gone this path. That is, there is little in the way of guidance so this would require significant development.

Windows? Maybe with Windows Server OS running. But that’s not what I got. No.

So looking around at my inventory, I decided to use an unused Window 7 netbook. But, rather than overwriting the hard-drive, a USB flash drive installation was done.

Linux running an MQTT broker when booting to the flash drive.

Windows 7 with the flash drive removed.

While there were a few challenges along the way, it turned out to be a great solution. It has been running continuously now for over a week – flawlessly.

Here is how to set it up…

Linux Installation

Looking for a small, clean Linux distribution, I selected Puppy Linux.  The choice was easy to make, since it had already been setup and running. This post provides step-by-step instructions to configure your flash drive. Follow all the instructions as you will need the development environment to build the MQTT application.

Building the MQTT Broker Application

First thing needed is a copy of the Mosquitto 1.4.7 broker. You can get it here. Then, with the flash drive in a Windows PC, copy the unzipped contents of the folder org.eclipse.mosquitto-1.4.7 to the flash drive in a new folder in the path:

<flash drive>/MyPrograms/mosquitto

You can now install the flash drive in your target computer and reboot. It should start in Puppy Linux.

Before we can build the application, a couple of steps are needed.

• Install Mosquitto Package
• Install libwebsockets
• Edit build configuration file

Let’s go…

Installing the Mosquitto Package

1. From the Desktop, click on the blue “install” icon.
2. Click on the “Install Applications” tab.
3. Click on the Puppy Package Manager icon.
4. Enter “mqtt” into the search box and hit the “Enter” key.
5. Click on the mosquitto_0.15 Package.
6. Click “Install” (Upper right of windows).

Installing the libwebsockets Library

1. From the Desktop, click on the blue “install” icon.
2. Click on the “Install Applications” tab.
3. Click on the Puppy Package Manager icon.
4. Enter “libwebsockets” into the search box and hit the “Enter” key.
5. Click on the libwebsockets3_1.2.2.1 Package.
6. Click on the libwebsockets-dev_1.2.2.1 Package.
7. Click “Install” (Upper right of windows).

Edit build configuration file

1. From the Desktop, click on the green “edit” icon.
2. Click Open, then under “Places”, click “File System”.
3. Under “Name”, click “mnt”. Then click open.
4. Under “Name”, click “home”. Then click open.
5. Under “Name”, click “MyPrograms”. Then click open.
6. Under “Name”, click “mosquitto”. Then click open.
7. Under “Name”, click “config.mk”. Then click open.
8. Scroll down to “WITH_WEBSOCKETS:=no and change to “yes”
9. Save the file and exit.

Building the broker application

We are now ready to build the application. This is really simple. First, open the console window by clicking on the “console” icon from the desktop. Now switch to the directory that contains the mosquitto source code by entering:

cd /mnt/home/myprograms/mosquitto/src

now build the application by entering:

make

Configuring the MQTT Broker Application

Edit mosquitto run-time configuration file

1. From the Desktop, click on the green “edit” icon.
2. Click Open, then under “Places”, click “File System”.
3. Under “Name”, click “mnt”. Then click open.
4. Under “Name”, click “home”. Then click open.
5. Under “Name”, click “MyPrograms”. Then click open.
6. Under “Name”, click “mosquitto”. Then click open.
7. Under “Name”, click “mosquitto.conf”. Then click open.
8. From the edit menu, select”Save as” and save this file to the src folder. The full file path should now be /mnt/home/myprograms/mosquitto/src/mosquitto.conf.
9. Scroll down or search for “#allow_anonymous true”. Change this to “allow_anonymous false”. Remember to delete the # so this is not commented out. This will force the broker to require usernames and passwords to connect.
10. Scroll down or search for “#user mosquitto”. Change this to “user nobody”. Remember to delete the # so this is not commented out.Puppy linux does not have a user named “mosquitto” but it does have one named “nobody”. Since Puppy Linux is a single user distribution, it does not allow you to add users.
11. Scroll down or search for “#password_file”. Change this to “password_file pw.txt”. Remember to delete the # so this is not commented out.
12. Scroll down or search for the text “#protocol mqtt”. Just after this line, add the following 4 new lines:
    1. listener 11883
    2. protocol mqtt
    3. listener 18080
    4. protocol websockets
13. Save and exit the file

What step 9 does is configure the broker with two listeners, one with standard mqtt (tcp) protocol and one with websockets.

While any port can be used, a one was added to the standard port numbers so our broker is not in conflict with the “well-known” mqtt ports. This could be important in the complicated event that your client is connected to two different brokers at the same time. In this case, the ports can only be open once. This eliminates potential conflicts.

Creating a password file

Using the file editor (edit icon from the desktop), save a blank file in the src folder:

/mnt/home/myprograms/mosquitto/src/pw.txt

Add a few username/password entries in this file in the format, for example:

username:password
user2:password2

Save the file. Also make a copy of this file for off-line storage.

Now run the password utility. This will change the plain text passwords in the file pw.txt to a hashed value. Run the password utility from the console:

cd /mnt/home/myprograms/mosquitto/src
./mosquitto_passwd -U pw.txt

If you open pw.txt, you will find the plain text passwords have been replaced with a hashed value.

Opening the Linux Firewall

Are we ready to run the broker yet? Almost. But there is one more thing needed in order to access the broker from another device. We need to configure Linux to allow external connections.

Open the file /etc/hosts.allow

It should contain one line:

ALL:LOCAL

Change this to:

ALL:ALL

Save and exit the file.

If you want a more restrictive environment, it is suggested that you research configuration settings for the Linux hosts.allow file. For the purpose of this example, we are opening the MQTT Broker to anyone with proper username/password credentials.

Starting the Broker

Now to start the broker, just go to the src folder and enter the following:

cd /mnt/home/myprograms/mosquitto/src
./mosquitto -c mosquitto.conf

The startup console should display:

The warning occurs because ipv6 is not supported. But this is of no concern for the typical ipv4 addresses. While the application was initially build in the ../src directory, it can be moved and executed from any location of your choosing. just remember to also include the password file utility, the config file and the password file. These are the files needed to run the application:

mosquitto
mosquitto_passwd
mosquitto.conf
pw.txt

Testing the Broker

You probably have your own devices to connect to and test the broker. With the configuration of this broker, you will need to include a username and password when connecting. I like to use the Google Chrome MQTTlens extension and the Android MyMQTT App when making quick MQTT connection checks. I’ve provided additional details for using these MQTT tools in this post.

Port Forwarding and DDNS 

While you can access this MQTT  broker from any device on your LAN subnet, world-wide broadband access will require configuring your router to forward MQTT port requests to the device running the broker. This is called port forwarding. Please refer to this post for more information about configuring port forwarding and DDNS.

In Closing

This broker setup provides you with all the features of the MQTT specification. One of the best features is the ability to operate a mixed protocol system. This provides an connection to devices using either mqtt (tcp) and websocket protocol together. And you can enable any level of security needed, from simple passwords to security certificates. And since it is running locally, you are in full control.

Next up. I am planning to update my MQTT App inventor template application to support the basic security credential provided by this broker.

I hope you find this information useful…

 

Share This:

Social tagging: Flash Driver MQTT Broker > MQTT Broker