Securing your IoT things is critical. MQTT connections are definitely at risk. But simply using username/password combinations for your MQTT connection is NOT secure.
Why not?
Well, even with passwords, everything sent over the Internet is unencrypted…open for any and all prying eyes to see…and possible maliciously manipulate.
Fortunately, we can solve that problem by creating TLS connections for secure MQTT data transfers. And yes, this encryption layer can even be brought to the ESP8266 platform.
But it does come at a cost…a huge bite out of the limited ESP RAM heap. With grave consequences. I certainly experienced catastrophic results during in my initial attempts. Finally, and after several iterations, the implementation presented here now manages the heap successfully. Using this approach, we achieve the desired results…a stable, reset-free ESP8266 platform with a secure interface to an MQTT broker.
Cutting to the Chase
At this point, if you would like to jump in without the benefit of additional explanation, please find the code for this Arduino 1.8.2 IDE compiled project here.
Compiler Issue
The first step towards adding TLS to the ESP8266 framework was to access a secure MQTT Broker. Installation of a TLS MQTT Broker was presented in my last post. After setting up the TLS MQTT broker on a VPS host, the next step was to update the ESP8266 IoT platform developed previously to support the secure connection. This required the addition of a new class to the project; WiFiClientSecure. This class supports secure TCP connections.
But I soon discovered the first problem…
You see, the project simply would not compile with the introduction of this class.
Through much research and frustration, I finally discovered the root cause of the problem and better yet, a solution.
As it turned out, one of the compiler flags used by the Arduino IDE was incompatible with the new WiFiClientSecure class library, and had to be revised. While your computer may have a different path to the compiler options file, mine was located on a Windows 10 PC at:
C:\Users\owner\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.2.0\platform.txt
Here is the line that needs to be changed in order to compile the project:
- #OLD: compiler.c.elf.libs=-lm -lgcc -lhal -lphy -lpp -lnet80211 -llwip -lwpa -lcrypto -lmain -lwps -lsmartconfig -lmesh <span style="color: #ff0000;">-lssl</span>
- #NEW: compiler.c.elf.libs=-lm -lgcc -lhal -lphy -lpp -lnet80211 -llwip -lwpa -lcrypto -lmain -lwps -lsmartconfig -lmesh <span style="color: #ff0000;">-laxtls</span>
A Heap of Trouble
After a successful compilation and upload to my development ESP8266 hardware, I soon discovered that the TLS connection consumed a huge chunk of the ESP heap (RAM). Somewhere between 12 and 15 kB. This was such a big resource hog that the ESP8266 would reset anytime a TLS MQTT connection was attempted.
And I also discovered that the configuration screen also used a large amount of ESP heap. That was because, even though the page’s HTML was stored in program memory as a constant, it had to be brought into the heap for run-time modifications before the page was rendered.
These two processes, ESP configuration and TLS MQTT connection negotiation were fighting for the same limited heap space. While each process would work on its own, they simply could not both exists at the same time.
So the project was modified to start up in one of two modes:
- Configuration Mode – Sets unique ESP8266 operational parameters
- MQTT Connection Mode – Connects to TLS Sensor and continuously reads sensors
Start-up Mode
Upon power-up, the ESP8266 had to know which mode to start up in. That would limit the heap usage to the requirements of the selected mode. This was implemented as a EEPROM location that is read in the programs setup() function.
- SetSysCfgFromEeprom(); // Set Session Cfg from EEPROM Values
Two EEPROM locations are now used for MQTT initialization.
- GetEepromVal(&Param, EEPROM_MQTTSSLEN, EEPROM_CHR);
- mqtt_ssl_enable = os_strcmp(Param.c_str(), "true")==0;
- GetEepromVal(&Param, EEPROM_MQTTCLEN, EEPROM_CHR);
- mqtt_cl_enable = os_strcmp(Param.c_str(), "true")==0;
If MQTT is configured to be enabled (mqtt_cl_enable), then the second parameter (mqtt_ssl_enable) is used to determine whether a TLS connection (sslClient) or a standard connection (espClient) is to be used at ESP8266 startup:
- if(mqtt_cl_enable) {
- client = mqtt_ssl_enable ? new PubSubClient(sslClient) : new PubSubClient(espClient);
- MqttServer_Init(); // Start MQTT Server
- }
These two parameters are set within the operational mode that the ESP8266 is currently operating.
Identifying the Start-up Mode
There are two methods to determine which mode the ESP8266 start up running. You can either look at the start-up output from the serial port or query the ESP8266.
Configuration Mode with Web Server:
The start-up output will include the following if the ESP has started up in configuration mode, with the web server running:
ESP8266 IP: 192.168.0.133
ESP8266 WebServer Port: 9705
ESP8266 Mode: Web Server Running – Type = SDK API
Of course, the port and IP values may vary if they are configured differently.
MQTT Mode:
The start-up output will include the following if the ESP has started up in MQTT Mode:
MQTT Rx Topic: mqtt_rx_18fe34a26629
MQTT Tx Topic: mqtt_tx_18fe34a26629
ESP8266 Mode: MQTT Client Running
Note the MQTT topics are provided. These values need to be known in order to communicate with the ESP, which acts like a server for this project.
Start-up Mode Query
The start-up mode can also be determined from the response to a query. This method may be preferred for applications that do not have access to the ESP serial port or are using the serial link for other purposes.
The ESP will not respond if it is not running in the queried mode. For configuration mode, simply enter the configuration URL (or any other valid web server URL). If the ESP responds, it is running in this mode.
Likewise, attempt to send a message to the ESP MQTT “server”. If is running in MQTT mode if a response is received
Configuration Mode
The configuration screen from the last iteration of this project was modified to add the new MQTT parameters. Just as before, this screen is accessed by entering the ESP8266 IP (or domain name) with it’s assigned web server port with the following suffix :
http://192.168.0.133:9705/config
If you want to switch to the TSL MQTT operating mode at power-up, just check the 2 new boxes, click SAVE, and then RESET ESP8266.
A URL has also been provided to switch modes within your application code:
http://192.168.0.133:9705/?request=MqttSslOn
This will change the EEPROM content to switch modes and reset the ESP8266 so it starts-up in the selected mode.
MQTT Mode
When the ESP8266 starts up in MQTT mode with TLS connection enabled, it will respond to server requests just like it did in the last iteration of this project. And switching back to ESP web server/configuration mode is simply a matter of sending the request in an MQTT message:
/?request=HttpOn
Conclusion
We now have a secure MQTT connection to the ESP8266. Here is the project code link again. And with careful management of the RAM (heap), it will perform reliably. More functionality in this case requires the distribution of tasks between two operating modes which are only loaded in memory when needed. This technique can be used in other cases as well to stretch the capabilities of this amazing little system on a chip (SoC) we call ESP8266.
Share This:
