Web Security

ESP8266 SSL/TLS MQTT Connection

Securing your IoT things is critical. MQTT connections are definitely at risk. But simply using username/password combinations for your MQTT connection is NOT secure.

Why not?

Well, even with passwords, everything sent over the Internet is unencrypted…open for any and all prying eyes to see…and possible maliciously manipulate.

Fortunately, we can solve that problem by creating TLS connections for secure MQTT data transfers. And yes, this encryption layer can even be brought to the ESP8266 platform.

But it does come at a cost…a huge bite out  of the limited ESP RAM heap. With grave consequences. I certainly experienced catastrophic results during in my initial attempts. Finally, and after several iterations, the implementation presented here now manages the heap successfully. Using this approach, we achieve the desired results…a  stable, reset-free ESP8266 platform with a secure interface to an MQTT broker.

Cutting to the Chase

At this point, if you would like to jump in without the benefit of additional explanation, please find the code for this Arduino 1.8.2 IDE compiled project here.

Compiler Issue

The first step towards adding TLS to the ESP8266 framework was to access a secure MQTT Broker. Installation of a TLS MQTT Broker was presented in my last post. After setting up the TLS MQTT broker on a VPS host, the next step was to update the ESP8266 IoT platform developed previously to support the secure connection. This required the addition of a new class to the project; WiFiClientSecure. This class supports secure TCP connections.

But I soon discovered the first problem…

You see, the project simply would not compile with the introduction of this class.

Through much research and frustration, I finally discovered the root cause of the problem and better yet, a solution.

As it turned out, one of the compiler flags used by the Arduino IDE was incompatible with the new WiFiClientSecure class library, and had to be   revised. While your computer may have a different path to the compiler options file, mine was located on a Windows 10 PC at:

C:\Users\owner\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.2.0\platform.txt

Here is the line that needs to be changed in order to compile the project:

 
 
  1. #OLD: compiler.c.elf.libs=-lm -lgcc -lhal -lphy -lpp -lnet80211 -llwip -lwpa -lcrypto -lmain -lwps -lsmartconfig -lmesh <span style="color: #ff0000;">-lssl</span>
  2. #NEW: compiler.c.elf.libs=-lm -lgcc -lhal -lphy -lpp -lnet80211 -llwip -lwpa -lcrypto -lmain -lwps -lsmartconfig -lmesh <span style="color: #ff0000;">-laxtls</span>

A Heap of Trouble

After a successful compilation and upload to my development ESP8266 hardware, I soon discovered that the TLS connection consumed a huge chunk of the ESP heap (RAM). Somewhere between 12 and 15 kB. This was such a big resource hog that the ESP8266 would reset anytime a TLS MQTT connection was attempted.

And I also discovered that the configuration screen also used a large amount of ESP heap. That was because, even though the page’s HTML was stored in program memory as a constant, it had to be brought into the heap for run-time modifications before the page was rendered.

These two processes, ESP configuration and TLS MQTT connection negotiation were fighting for the same limited heap space. While each process would work on its own, they simply could not both exists at the same time.

So the project was modified to start up in one of two modes:

  1. Configuration Mode – Sets unique ESP8266 operational parameters
  2. MQTT Connection Mode – Connects to TLS Sensor and continuously reads sensors

Start-up Mode

Upon power-up, the ESP8266 had to know which mode to start up in. That would limit the heap usage to the requirements of the selected mode. This was implemented as a EEPROM location that is read in the programs setup() function.

 
 
  1. SetSysCfgFromEeprom(); // Set Session Cfg from EEPROM Values

Two EEPROM locations are now used for MQTT initialization.

 
 
  1. GetEepromVal(&amp;Param, EEPROM_MQTTSSLEN, EEPROM_CHR);
  2. mqtt_ssl_enable = os_strcmp(Param.c_str(), "true")==0;
  3. GetEepromVal(&amp;Param, EEPROM_MQTTCLEN, EEPROM_CHR);
  4. mqtt_cl_enable = os_strcmp(Param.c_str(), "true")==0;

If MQTT is configured to be enabled (mqtt_cl_enable), then the second parameter (mqtt_ssl_enable) is used to determine whether a TLS connection (sslClient) or a standard connection (espClient) is to be used at ESP8266 startup:

 
 
  1. if(mqtt_cl_enable) {
  2. client = mqtt_ssl_enable ? new PubSubClient(sslClient) : new PubSubClient(espClient);
  3. MqttServer_Init(); // Start MQTT Server
  4. }

These two parameters are set within the operational mode that the ESP8266 is currently operating.

Identifying the Start-up Mode

There are two methods to determine which mode the ESP8266 start up running. You can either look at the start-up output from the serial port or query the ESP8266.

Configuration Mode with Web Server:
The start-up output will include the following if the ESP has started up in configuration mode, with the web server running:

ESP8266 IP: 192.168.0.133
ESP8266 WebServer Port: 9705
ESP8266 Mode: Web Server Running – Type = SDK API

Of course, the port and IP values may vary if they are configured differently.

MQTT Mode:
The start-up output will include the following if the ESP has started up in MQTT Mode:

MQTT Rx Topic: mqtt_rx_18fe34a26629
MQTT Tx Topic: mqtt_tx_18fe34a26629
ESP8266 Mode: MQTT Client Running

Note the MQTT topics are provided. These values need to be known in order to communicate with the ESP, which acts like a server for this project.

Start-up Mode Query
The start-up mode can also be determined from the response to a query. This method may be preferred for applications that do not have access to the ESP serial port or are using the serial link for other purposes.

The ESP will not respond if it is not running in the queried mode. For configuration mode, simply enter the configuration URL (or any other valid web server URL). If the ESP responds, it is running in this mode.

Likewise, attempt to send a message to the ESP MQTT  “server”. If is running in MQTT mode if a response is received

Configuration Mode

The configuration screen from the last iteration of this project was modified to add the new MQTT parameters. Just as before, this screen is accessed by entering the ESP8266 IP (or domain name) with it’s assigned web server port with the following suffix :

http://192.168.0.133:9705/config

If you want to switch to the TSL MQTT operating mode at power-up, just check the 2 new boxes, click SAVE, and then RESET ESP8266.

A URL has also been provided to switch modes within your application code:

http://192.168.0.133:9705/?request=MqttSslOn

This will change the EEPROM content to switch modes and reset the ESP8266 so it starts-up in the selected mode.

MQTT Mode

When the ESP8266 starts up in MQTT mode with TLS connection enabled, it will respond to server requests just like it did in the last iteration of this project. And switching back to ESP web server/configuration mode is simply a matter of sending the request in an MQTT message:

/?request=HttpOn

Conclusion

We now have a secure MQTT connection to the ESP8266. Here is the project code link again. And with careful management of the RAM (heap), it will perform reliably. More functionality in this case requires the distribution of tasks between two operating modes which are only loaded in memory when needed. This technique can be used in other cases as well to stretch the capabilities of this amazing little system on a chip (SoC) we call ESP8266.

 

Loading

Share This:
FacebooktwitterredditpinterestlinkedintumblrFacebooktwitterredditpinterestlinkedintumblr

How I got rid of smartdns proxy redirections

smartdns

One of the most annoying experiences while browsing the web is URL redirection. Have is ever happened to you? You click on a link or enter a website address expecting to land where you expected. But end up somewhere else. Your target URL has essentially been hijacked, landing your browser somewhere else.

My most recent issue was getting redirected to smartDNS proxy. No matter how many times I tried, many of my intended URL locations ended up there. An uninvited and unwanted intrusion. smartDNS proxy wanted you to sign up for their service to resolve the problem…for a fee of course. Extortion is the proper term for it.

And searching for a solution using google only results in long winded step-by-step instructions for getting rid of smartDNS proxy. Only problem was, in my case, it did not solve the problem. it appears to be a tactic to wear you down, so you will buy the quick solution, “SpyHunter”.

That also looks like a suspicious product to me.

Here is what I found to work for me. It is free. No charge! and only required a few steps. I hope this information may provide some relief for anyone encountering a similar problem.

Root cause assessment

The origin of the problem clicked to me as I observed that all of my home network devices exhibited the same undesirable redirection to smartDNS proxy. That included my smartphone. Yet when I used my 4G to browse instead of the network WIFI. the problem went away.

And that ruled out “SpyHunter” as a viable solution as it is supposed to clean a single “infected” device. The problem was not with the web browser, but rather originated in the router. That is the single common component that all network connected devices share.

Getting rid of DNS redirection

The router resolution of website domain names to IP addresses is accomplished through the DNS Servers that it is configured  to use. Every router has a unique way to access these settings. You will need to consult your router documentation to determine how to access your router’s DNS server settings.

When I looked up my router DNS servers, they were set to “23.21.43.50” (primary DNS server) and “54.229.171.243” (secondary DNS server). Using the “whois” utility at https://www.ultratools.com/tools/ipWhoisLookup, I was surprised to note that both DNS servers were owned by Amazon Web Services. And it was not clear as to how those servers had been set in my router.

Still, while it did not seem likely that the trusted Amazon name was being hijacked by smartdns proxy, the only way to find out was to change the DNS servers. And sure enough, changing the DNS servers completely got rid of my smartdns proxy redirection.

A free solution.

Without the need for “SpyHunter”.

My router is now configured to use the google DNS Servers. And all is good!

Primary DNS Server: 8.8.8.8

Secondary DNS Server: 8.8.4.4

I hope this information is useful to many web users that have had a similar problem.

 

Loading

Share This:
FacebooktwitterredditpinterestlinkedintumblrFacebooktwitterredditpinterestlinkedintumblr

Press Ctrl+C to copy the following code.
"