MQTT for App Inventor – Adding Configurable Settings

21st January 2016 by InternetOfHomeThings in All Posts, Arduino, ESP8266 : An IoT System on Chip, Internet of Things | 5 Comments

App Inventor to MQTT Communication

In my last post, a basic MQTT client portal for App Inventor was introduced. As result, there has been some interest expressed for TCP socket support. Note that the implementation presented relies on WebSockets. Due to security vulnerabilities, pure TCP sockets are simply not possible with this design. If TCP sockets are needed for MQTT communication at your IoT device, a broker capable of both WebSockets and TCP sockets should be used – Websockets on the App Inventor side, and TCP sockets on the IoT device end.

Adding Configurable Setting

My next step was to refine the App Inventor project to support configurable MQTT setting.

NOTE: If you just want to review and use the updated App Inventor project without the walk-through that follows, simply use the files provided in this Github repository.

The first challenge while adding this capability was the inability to grant file system access to the JavaScript code. Native Android Apps can overcome this with a change to the App’s manifest file. But App Inventor does not expose this application setting.

So the App Inventor file component had to be used to read and write to the file system. The configuration file settings are then passed on to the JavaScript to update the MQTT connection parameters.

App File Location

This App now uses two files stored on the Android device. These files have been moved to a common location which should be the same on most Android devices:

	File	Description
	/sdcard/mqtt/mqtt_appinventor.html	The JavaScript MQTT App Inventor Server
	/sdcard/mqtt/cfg.txt	MQTT Configuration

Configuration file

For this example, 3 parameters are configurable:

Broker: This can be any on-line broker, on your own network or either publicly or privately provided.
Request Topic: MQTT topic used to send a client request
Reply Topic: MQTT topic used to reply results to the client.

Since this example uses the publicly available broker ‘test.mosquitto.org’, username/password authentication is not provided. You could easily be add this to the example provided here, however, if you are using a broker requiring authentication.

The file is formatted as a JSON string. This makes it very easy for the JavaScript to parse the values. The first time the App is run, these configurable parameters are set to the values shown in this JSON string:

{
“mqtt_broker”:”ws://test.mosquitto.org:8080/mqtt”,
“mqtt_txtopic”:”MyMqttSvrRqst”,
“mqtt_rxtopic”:”MyMqttSvrRply”
}

App Inventor File Read/Write

When the screen used to edit the configurable parameters is open, it first reads the cfg.txt file, extracts the parameter setting, and uses those values to populate the screen fields.

But when using App Inventor, there are no (at least none that I am aware) libraries available for parsing JSON. So I created the procedure “GetParam”, which extracts the cfg.txt values with 3 calls, one for each parameter.

Two values are determined initially in order to extract the value. First is the position in the string that the value starts. The other value is the string length. These values are determined by using App Inventor’s string search (starts at) feature to find something unique (StartPiece and LenPiece) around the value wanted, and an offset from the parameter value’s exact position in the string.

Once extracted, the “Edit Configuration” screen is populated with the values from this file.

Saving the values is just the opposite. The values from the screen are formatted into a JSON string and then saved back to the same file, cfg.txt.

When this screen is exited, the new configuration values are sent to the JavaScript code. The JavaScript uses these new values and closes, then opens the MQTT connection with the new values. Here is how that has been implemented…

Updated Main Screen

As I have gained knowledge coding with the App Inventor, it became obvious that meaningful names were needed for project components. Much to my surprise, this does not extend to the main screen. It is called “Screen1” by default, and cannot be changed.

The screen GUI was changed somewhat for this update. Two new buttons were added. One to open the configuration screen and one to exit the application. The components were also move to the bottom of the screen (don’t want to cover the mosquito, do we?).

Every time the main screen is open, it’s 100 ms timer restarts. That is where the command is sent to the JavaScript to update the configuration settings.

Once.

It is sent 1/2 second (500 ms) after the screen is open. An important note: I found that the command did not execute if initiated concurrently with the opening of the screen. So the 500 ms delay (5 timer iterations) was introduced using a timer, which works every time.

At 500 ms, the cfg.txt file is read. “CFG:” is inserted at the beginning of the string before being sent to JavaScript via the WebViewString.

A handler is also included for the first time this application is run. In that case, the cfg.txt file does not exists. As result, an exception is thrown when attempting to “ReadFrom” the file. An error handler was added for this condition.

Error 2101 occurs when the file does not exist. In this case, a file is created with default values for the configurable parameters.

JavaScript for Configurable MQTT

The JavaScript also required revisions to support MQTT configurable parameters. The original version only handled IoT requests. These were identified with ‘GET:’ as the first 4 characters. Similarly, the added configuration command is now recognized as WebViewStrings beginning with “CFG:”

// 10 Hz WebViewString polling timer function -------------------------------------->
function AppInventorServer() {
var request = window.AppInventor.getWebViewString(); // Get WebViewString
if(request.substring(0, 4)=="GET:") { // Validate request
window.AppInventor.setWebViewString(""); // Reset String (process once)
$("#request").val(request.substring(4, request.length)); //set request html textbox
SendMqttRequest(); // Send Mqtt Request
}
if(request.substring(0, 4)=="CFG:") { // Validate request
window.AppInventor.setWebViewString(""); // Reset String (process once)
var cfgpar = JSON.parse(request.substring(4, request.length));
txtopic = cfgpar.mqtt_txtopic;
rxtopic = cfgpar.mqtt_rxtopic;
mqtturl = cfgpar.mqtt_broker;
client.disconnect();
}
setTimeout(AppInventorServer, 100); // run AppInventorServer() in 100 ms
}

As you can see, parsing and extracting values from a JSON string is very simple and straight-forward with JavaScript. Once the new values are set, the MQTT broker connection is updated with a call to ‘client.disconnect()’.

// Callback executed upon MQTT broker disconnection -------------------------------->
client.ondisconnect = function(rc){
client.connect(mqtturl);
};
// Callback executed upon MQTT broker connection ----------------------------------->
client.onconnect = function(rc){
client.subscribe(rxtopic, 0);
};

Improving the Arduino Digital Commands

The original project used hard-coded values for the Digital Get and Set commands. Ok for an example, but not so good if you need access to a channel outside the card-coded value. That was rectified with this update, which now supports digital channel selection.

Select “Set Arduino Digital Channel” and a new screen is open. This screen allows you to select the channel to set.

This screen is open with a “start value” received from the main screen that called it.

The value returned when OK is clicked depends on the startValue. If it is a “Set” command, an Arduino SetDigital request string is returned. The selected channel and logic state from the screen’s selections are also returned with the request.

Notice that a call to close the screen is made before the main screen (Screen1) is re-opened. That is necessary when returning a startValue. If the screen is not closed, it remains open for the duration of the App execution.

Another call to this window will open another instance of the window. This is effectively a memory leak which will crash you Android device eventually. Takeaway: Close the screen before opening a new screen when passing a start value to avoid memory leaks.

As you can see, a similar value is returned for “Get” requests. But when “Get” requests are received, the channel state is not selected, it is returned from the request. When this screen is open with a “Get” request, the channel state selection GUI is suppressed.

Sending the Arduino Digital Request

When the Digital channel selection screen is closed, the main screen is reopened. But how does it know to send the Arduino command to MQTT? After all, the screen is opened fresh, just like when the app is started.

Not quite. Remember the ‘startValue’ returned from the digital selection? That is checked to determine whether an arduino request is needed. It is done in the 100ms timer callback.

At 500 ms after the screen is open, the MQTT configuration is sent to the JavaScript. But then, at 1000 ms (10 100 ms intervals), the startval string is check for a non-null value. If a string is present, it is sent to the JavaScript via the WebViewString, by calling the SendRequest procedure.

Exiting the Application

Just one more thing and we are done with this update. I thought it would be nice to add an ‘Exit’ button to close the app. What I found was that it required several clicks before the app would actually close. What was happening was that every time a different screen returned to the main screen, the previous main screen remained open and a new main screen instance was created.

Not exactly what I had in mind. Another memory leak!

This was corrected by adding a ‘close screen’ before each time one of the two added screens was open. This guarantees only one screen is open at a time while the app is running.

With this correction, the Exit button works as intended, with only one click required.

In Closing

This example should provide a framework for anyone needing to use MQTT with an App Inventor project. The main caveat is that a broker that supports WebSockets is required. I leave it up to you as an exercise to add username and password credentials to the mqtt connection settings. The basic structure is here. And it should also be a simple task to add “GetAnalog” to the Arduino request options. Go ahead, you can do it! If you need it.

Again, here are the project files.

I hope you find this information useful…

MQTT For App Inventor

13th January 2016 by InternetOfHomeThings in All Posts, ESP8266 : An IoT System on Chip, Internet of Things | 15 Comments

App Inventor is a visual, easy-to-use online Android Application development platform. It is a great vehicle that empowers the do-it-yourself enthusiast to develop mobile IoT clients without battling the steep learning curve of traditional text-based development platforms.

Aside from the basic programming language constructs, all you need is a method of accessing your Internet enabled devices. And many developers prefer to use MQTT, a machine-to-machine connectivity protocol, to communicate externally with their IoT “things”.

But up to now, there has not been a lot of information available showing how to use MQTT with App Inventor. Here is what I have come up. It’s a working example of an Android App Inventor project using MQTT.

The Concept

When I first looking into this, it seemed that those looked into this were trying to adapt the existing App Inventor Web components to work with MQTT. But that just won’t work. MQTT requires a TCP connection to remain open for the entire time the App is running. And App Inventor only provides an http interface.

App Inventor also did not appear to provide a method of adding a c-based MQTT library. Another possibility was to interface with JavaScript. After all, I had already been successful in implementing an MQTT webpage using JavaScript.

But how can you interface with JavaScript using App Inventor?

After investigating App Inventor features deeper, the answer popped into my head like a light bulb flipping on. The solution was right there…

WebViewString .

That’s it! You can set the WebViewString from App Inventor. And JavaScript can read and act upon the contents of the WebViewString object. All I had to do was setup a basic communication scheme with this and MQTT communication would be possible.

Overall Communication Structure

Here is how it works…

App inventor sets the WebViewString to the MQTT topic payload that it wants to publish. Then, a JavaScript file monitoring the value of the WebViewString for MQTT requests picks up the payload and publishes it. The IoT device that subscribes to the topic then detects the request, acts on it, and returns a reply via an MQTT topic publication.

The rest of the sequence should be obvious. The JavaScript subscribing to the MQTT topic published by the IoT device then forwards the reply back to App Creator by changing the value of the WebViewString .

That’s it. And here is the implementation…

App Inventor GUI

For this example, the App Inventor GUI has intentionally been kept simple. Just one user action provided: Click on the button to select an MQTT request and wait for the reply string to appear.

Since this example uses the Triple Server Update ESP8266 sketch from my previous post, these are the options that can be selected when the button is clicked. They can be easily customized for your own system requirements.

These selections are currently just the raw strings sent to the IoT based MQTT server. The App Inventor code could be refined, if desired, to make it more user friendly. Such as using a channel selector input as well as a logic state selector (HI or LO). But for this example, raw strings are suffice.

App Inventor Initialization Code

Here is the App Inventor code used to initialize the MQTT request options. This code executes when the App’s one and only screen opens. Just change the text in the list for your own MQTT topic payloads.

Note that the JavaScript MQTT server file pointer is also initialized here. This file, locally stored in the Android SDcard, is open when the App starts. More about that later in this article.

App Inventor Send MQTT Request Code

When one of the request options are selected (picked), the WebViewString is set to this request value. It is prefaced with “GET:” so the JavaScript code knows what is to follow is an MQTT request.

App Inventor Receive Reply Code

Finally, the App Inventor continuously monitors the WebViewString for replies from the JavaScript. While I do not prefer to use polling loops, that was the only mechanism I was able to come up with for this. The polling timer is set to repeat every 100 ms, or 10 times per second.

WebViewString is first validated as a reply by confirming it is not blank and does not start with the request “GET:” prefix. If these conditions are met, the GUI reply string is set to the WebViewString value. Note that the WebViewString value is set to “” after read so this polling function does not process it more than once.

JavaScript For MQTT

Since I already had a working MQTT webpage from a prior project, it was simply reused for this example. Of course, the irrelevant parts were remove and the WebViewString object was added. While it can be saved anywhere, for this project, the file is stored on the Android device’s SD card. File installation instructions are provided in the GitHub repository for this project.

JavaScript Initialization Code

The initialization code first brings in the required MQTT and jquery libraries from online sources:

<script type="text/javascript" src="http://test.mosquitto.org/mosquitto.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js"></script>

Next, default values are set for the request, MQTT broker, and the request/reply topics. An instance of a Mosquitto client is also created,

// Set Default Request ------------------------------------------------------------->
$("#request").val("/?request=GetSensors");
// Set Default MQTT Url ------------------------------------------------------------>
var mqtturl = "ws://test.mosquitto.org:8080/mqtt";
var txtopic = "MyMqttSvrRqst";
var rxtopic = "MyMqttSvrRply";
// Create MQTT client instance ----------------------------------------------------->
client = new Mosquitto();
client.run = 0;

Finishing the initialization, the MQTT broker connection is established and the WebViewString polling loop is started.

//executes once after window is loaded --------------------------------------------->
function windowloaded() {
client.connect(mqtturl); // Connect to MQTT broker
AppInventorServer(); // Start polling WebViewString
}
window.onload = windowloaded; // Launch windowloaded()

JavaScript Receive Request Code

Here is the polling loop, which checks for new MQTT requests from App Inventor.

// 10 Hz WebViewString polling timer function -------------------------------------->
function AppInventorServer() {
var request = window.AppInventor.getWebViewString(); // Get WebViewString
if(request.substring(0, 4)=="GET:") { // Validate request
window.AppInventor.setWebViewString(""); // Reset String (process once)
$("#request").val(request.substring(4, request.length)); //set request html textbox
SendMqttRequest(); // Send Mqtt Request
}
setTimeout(AppInventorServer, 100); // run AppInventorServer() in 100 ms
}
// Publishes MQTT topic contained in html #request object -------------------------->
function SendMqttRequest() {
client.publish(txtopic,$("#request").val(),0,0);
}

If the WebViewString starts with “GET:”, it is considered to be valid. In that case, the WebViewString is first set to “” to avoid processing the request again at the next 100 ms interval. The request string is then stripped from the WebViewString and published as an MQTT topic. This polling loop restarts every 100 ms.

JavaScript Reply Callback Code

Whenever the subscribed MQTT topic message is received, a JavaScript callback function is executed.

// Callback executed upon receipt of MQTT message ---------------------------------->
client.onmessage = function(topic, payload, qos){
if(topic==rxtopic) {
setpayload(payload);
}
};
// Sets WebViewString and html payload text box to 'payld' ------------------------->
function setpayload(payld) {
window.AppInventor.setWebViewString(payld); // Set WebViewString
$("#payloadmqqt").html(payld); // Set html textbox
}

This callback simply passes the reply (MQTT topic payload) back to App Inventor through the WebViewString. An html textbox is also filled with this reply string.

That’s it!

The App Inventor project file and JavaScript html file is available on GitHub here. Instructions for installing the JavaScript file on an Android device is also provided at this same GitHub location.

Hope you find this information useful.

And stay tuned. My next step will be adding non-volatile configuration for the MQTT parameters. This will provide the capability to change MQTT brokers and topics without changing the contents of the installed html file.

Triple Server Update – Part 2: Web Configuration

11th January 2016 by InternetOfHomeThings in All Posts, Arduino, ESP8266 : An IoT System on Chip, Internet of Things | 8 Comments

Have you ever moved your ESP8266 system from one Wifi location to another? Or perhaps made an extra one for your friend? In many cases, the initial setup requires a change to the sketch, to adjust the Wifi parameters to match the new access point—that is—unless the local settings are stored in the ESP8266 EEPROM.

What’s great about the EEPROM is that once updated, the configuration is non-volatile, That is, the values remain the same after the system power is removed and re-applied.

This post provides a method of setting and saving parameters likely to change with WiFi location, without changing the installed firmware sketch.

Once you’ve got you IoT device finished, you really should not have to modify the software again. That is, unless enhancements are made or bugs corrected.

Configurable Parameters

I started out with intentions on building a reusable configuration template. This would include the parameters most likely to change with system location and cloud-based servers used.

Using the Triple Server sketch I had developed recently as a test case, the configurable network and MQTT parameters were first identified. Then, these parameters were mapped to the ESP8266 EEPROM.

For the project presented in this post, the following parameters were singled out to be configurable:

WIFI SSID
WIFI Password
IP – Static IP assigned to ESP8266 when used in Station mode
NM – Network Mask
GW – Gateway (Wifi Router IP)
AP – Static IP assigned to ESP8266 when used in Access Point mode
Web Server network port
MQTT Broker
MQTT Username
MQTT Password
MQTT Request Topic
MQTT Reply Topic
Serial Port baud
Arduino Server Enable

EEPROM Memory Map

The ESP8266 EEPROM configuration space is typically the first 512 bytes of memory. And here is how I allocated the configurable parameters to this EEPROM.

Parameter	EEPROM	Default	Size(bytes)
WIFI SSID	0x10	YOURWIFISSID	32
WIFI Password	0x30	YOURWIFIPASSWORD	16
IP0 - IP Byte 1	0x40	192	1
IP1 - IP Byte 2	0x41	168	1
IP2 - IP Byte 3	0x42	0	1
IP3 - IP Byte 4	0x43	132	1
NM0 - Network Mask	0x44	255	1
NM1 - Network Mask	0x45	255	1
NM2 - Network Mask	0x46	255	1
NM3 - Network Mask	0x47	0	1
GW0 - Gateway	0x48	192	1
GW1 - Gateway	0x49	168	1
GW2 - Gateway	0x4A	0	1
GW3 - Gateway	0x4B	1	1
AP0 = Access point	0x4C	192	1
AP1 = Access point	ox4D	168	1
AP2 = Access point	ox4E	4	1
AP3 = Access point	ox4F	1	1
Server Port	0x50	9701	2
MQTT Broker	0x60	test.mosquitto.org	32
MQTT User Name	0x80	mqtt_username	32
MQTT Password	0xA0	mqtt_password	32
MQTT Request Topic	0xC0	MyMqttSvrRqst	32
MQTT Reply Topic	0xE0	MyMqttSvrRply	32
Serial Baud	0x100	1200	3
Arduino Server Enable	0x108		1

Note that while 32 bytes are allocated for the character array parameters, this size is the maximum; it is not required to use this entire space. But each character array must be terminated in EEPROM with a NULL (0x00) character.

Webpage Structure

The next thing needed was a way to construct a webpage for viewing, altering, and saving these configuration parameters. All of the page rendering is done on the client side browser. With this in mind, the web page was developed using only html, css for styling, and Javascript.

In order to conserve program memory, the webpage source is saved into flash memory. Two character arrays were used, one for the html and JavaScript and a separate one for the css styling.

Next time, I am planning to separate out the JavaScript as well for cleaner partitioning. This can easily be done from the example code provided here and is highly encouraged as an exercise.

A token was placed at the top of the html character array for insertion of the css styling. Likewise, tokens have been placed in the Javascript jquery section for each configurable parameter. These tokens are replaced with the values stored in EEPROM prior to sending the completed webpage string to the client browser for rendering.

In this example:

$("#ssid").val("set_ssid");
$("#password").val("set_pass");
$("#ip_0").val("set_ip0");
$("#ip_1").val("set_ip1");
$("#ip_2").val("set_ip2");
$("#ip_3").val("set_ip3");
.
.

set_ssid is the first parameter token, which is replaced with the value read from EEPROM.

Here is what the rendered configuration page looks like:

The parameters are structured in html as a form with the two buttons shown. The save button is of type “submit” which sends all of the parameters back to the ESP8266 as a GET request for saving to the EEPROM.

But this merely saves the values back to EEPROM. Since the startup setup() function has already been run, the saved parameters do not become effect until an ESP8266 reset is performed. That is accomplished either with a power cycle or clicking on the “RESET ESP8266” button on this configuration page.

ESP8266 Webpage Request/Reply Code

Code was added to the ESP8266 URL request processing algorithm to handle web page requests. This is performed in the sketch’s Server_ProcessRequest function. For webpage requests, a slightly different URL format is used than requests to read a sensor.

The familiar example is the URL to get the sensor values:

192.168.0.132:9701/?request=GetSensors

When we want the ESP8266 to display the configuration page, enter:

192.168.0.132:9701/config

Note that there is no “?” for webpage requests. In order to parse the URL correctly for this case, the SdkWebServer_parse_url_params() function required modification.

Note that while the sketch provided in this example only supports the configuration page, a review of the code should make it obvious how to expand this to support your own custom webpages.

// --------------------------------------------------------------
// Serving Web Pages
// --------------------------------------------------------------
//
// ------------- Load Config Page -------------------------------
if(os_strcmp(pURL_Param->pParam[0], "config")==0) {
GetEepromCfg(&WebPage);
SdkWebServer_senddata_html(ptrespconn, true,
(char *) WebPage.c_str());
}
// -------------- Save config or Reset ESP8266 -------------------
if(os_strcmp(pURL_Param->pParam[0], "ssid")==0) {
if(os_strcmp(pURL_Param->pParVal[0], "reset")==0){
// ------------- Reset ESP8266 -------------------------------
WebPage = reinterpret_cast<const __FlashStringHelper *>
(PAGE_WaitAndReset);
SdkWebServer_senddata_html(ptrespconn, true,
(char *) WebPage.c_str());
os_timer_arm(&ResetEspTimer, 3000, false);
}
else {
// --- Save config to EEPROM and reload config page ------
SetEepromCfg((void *)pURL_Param);
WebPage = reinterpret_cast<const __FlashStringHelper *>
(PAGE_WaitAndReload); //reload config page
SdkWebServer_senddata_html(ptrespconn, true, (char *)
WebPage.c_str());
}
}

The first parameter parsed is used to determine the webpage request type. This example only processes 2 possibilities. If the first parameter is “config”, the configuration page is rendered.

That is accomplished by building the webpage string. First using the template, and then substituting values for the css and parameter tokens. This is all done in the GetEepromCfg() function:

void GetEepromCfg(String *page)
{
*page = reinterpret_cast<const __FlashStringHelper *>
(PAGE_NetCfg);
String css = reinterpret_cast<const __FlashStringHelper *>
(PAGE_Style_css);
page->replace("ADDSTYLE",css);
css = "";
SetCfgPageWithEepromVal(page, "set_ssid", EEPROM_WIFISSID,
EEPROM_CHR);
SetCfgPageWithEepromVal(page, "set_pass", EEPROM_WIFIPASS,
EEPROM_CHR);
SetCfgPageWithEepromVal(page, "set_ip0", EEPROM_WIFI_IP0,
EEPROM_INT);
SetCfgPageWithEepromVal(page, "set_ip1", EEPROM_WIFI_IP1,
EEPROM_INT);
.
.

But if the first parameter is ssid, the value of the parameter must first be read in order to determine what action is requested. That is because the method of returning values to the ESP8266 code for both of the WebPage button is by submitting the form.

When the reset button is clicked, the ssid value is changed to “reset” before submitting. In that case (above code), as message is displayed indicating the ESP8266 is resetting. Note that a 3-second delay is implemented using a timer before the ESP8266 resets itself. That is necessary to allow time for the reloading of the configuration webpage to complete prior to the reset.

And in the case that the Save button is clicked, the ssid value from the webpage form input is not modified (set to “reset”) and thus, the values are simply saved to EEPROM.

Webpage Expansion

Again, all you need to do is add a suffix to the ESP8266 access url and add your processing code to expand this webpage server:

192.168.0.132:9701/YOURCUSTOMPAGE

if(os_strcmp(pURL_Param->pParam[0], "YOURCUSTOMPAGE")==0) {
//Do whatever you want here to handle this request
}

In case you missed part 1, the same code is used for this part 2. It is available on GitHub here.

In Closing

That’s it. A working framework for saving configuration parameters to ESP8266 EEPROM. And as an added bonus, you have a structure to expand this to render webpages of your own choosing. To control or monitor your own IoT things visually.

Hope you find this information as useful to you as it is for me…

Credits

Developing this example required research and reuse of material found in public domain sources. Special credit must be given to the following source which inspired my effort presented here. It also may be of use to others…

MQTT and Easy Web Config – by “The Godfather” – Posted October 22, 2015

Triple Server Update – Part 1: Serving Arduino

9th January 2016 by InternetOfHomeThings in All Posts, Arduino, ESP8266 : An IoT System on Chip, Internet of Things | 2 Comments

Happy New Year! I’m picking things up right where they ended last year…

There were some basic enhancements needed to my Triple Protocol Server. But in the process of implementing the updates, the limitations of the ESP8266 resources were reached. To make room for the new features, something had to go.

In this case, the CoAP server was eliminated. The choice was obvious. This protocol consumed a large chunk of the available heap, leaving little room for program execution. As result, the system became unstable. CoAP also does not lend itself easily to web browser based clients.

So with that gone, a valuable new feature was added. A light-weight modification that did not significantly burden the system. Something that Arduino users should find extremely useful.

What is it?

An Arduino Serial Port Web Server

What? We already have access from the Arduino using the AT command set. “So what’s so special about this?”, you say.

Well…this implementation removes most of the web server tasks from the Arduino and puts it in the ESP8266. The only thing the Arduino needs to do is read small strings (requests) from the ESP8266 and return a small string back. This frees the Arduino to concentrate on sensor readings and control functions.

This balances the load on the two micro-controllers, reducing the load on the Arduino while fully leveraging the available power of the ESP8266 as a web server.

So how does this work?

It’s really quite simple. The ESP8266 web server is already in place. All that is needed is a few new URL-based (or MQTT topic payload) commands to provide web-based access to the Arduino resources.

The New Arduino Commands

The commands are parsed and decoded by the ESP8266. The command is reduced to a small string that is passed to the Arduino for action with it’s hardware resources.

	URL Suffix	ESP8266 to Arduino	Arduino Reply
Set Digital Channel 04 HI	/?arduino=SetDigital&chan=04&state=1	Arduino_SD041	Digital Channel 04 is HI
Get Digital Channel 04	/?arduino=GetDigital&chan=04	Arduino_GD04	Digital Channel 04 is HI
Get Analog Channel 04	/?arduino=GetAnalog&chan=04	Arduino_GA04	Analog Channel 04 is 2.234

The Arduino returns a reply to the ESP8266, which, in turn, is returned to the http or mqtt client.

This structure is similar to the slash (/) delineated bridge interface used with the Arduino Yun to communicate between it’s Arduino and Linux processors. Instead of using a pricey Arduino Yun, you can use any model Arduino along with the very inexpensive ESP8266. In this test case, I used a $7 Arduino nano clone.

The example presented here only includes the basic digital and analog access to the Arduino. The digital interface is obvious, setting or reading the state of each pin. The analog channel reads returns a voltage between 0 and 5 volts, based on the 10-bit Arduino ADC channel reading.

From this command set, it should be obvious how to expand it to include bus oriented sensors such as 1-wire temperature and i2c connected devices.

ESP8266 Sketch

The ESP8266 Arduino IDE sketch is based upon my triple server sketch presented in a prior post. The default parameters are defined at the top of the sketch in the section titled “Initial EEPROM Values”. These can be revised to match your network setting as described in Part 2 of this post, and in the sketch’s github repository.

Additional code was needed to support the Arduino requests. The commands received are repacked into a small string to be sent to the Arduino via the ESP8266 serial port. The server code to process the three recognized requests (Set Digital, GetDigital and Get Analog) shown here can easily be expanded for addition requests:

// -----------------------------------------------------------------------
// Serving Arduino via serial port
// -----------------------------------------------------------------------
if(os_strcmp(pURL_Param->pParam[0], "arduino")==0) {
//-------------- Request = SetDigital ---------------------
if(os_strcmp(pURL_Param->pParVal[0], "SetDigital")==0){
if(os_strcmp(pURL_Param->pParam[1], "chan")==0) {
ArduinoRequest = "Arduino_SD";
ArduinoRequest += pURL_Param->pParVal[1];
}
else {
ArduinoRequest = "Invalid Request";
}
if((os_strcmp(pURL_Param->pParam[2], "state")==0)&&(os_strcmp(
ArduinoRequest.c_str(), "Invalid Request")!=0)) {
ArduinoRequest += pURL_Param->pParVal[2];
}
else {
ArduinoRequest = "Invalid Request";
}
}
//-------------- Request = GetDigital ----------------------
else if(os_strcmp(pURL_Param->pParVal[0], "GetDigital")==0){
if(os_strcmp(pURL_Param->pParam[1], "chan")==0) {
ArduinoRequest = "Arduino_GD";
ArduinoRequest += pURL_Param->pParVal[1];
}
else {
ArduinoRequest = "Invalid Request";
}
}
//-------------- Request = GetAnalog ----------------------
else if(os_strcmp(pURL_Param->pParVal[0], "GetAnalog")==0){
if(os_strcmp(pURL_Param->pParam[1], "chan")==0) {
ArduinoRequest = "Arduino_GA";
ArduinoRequest += pURL_Param->pParVal[1];
}
else {
ArduinoRequest = "Invalid Request";
}
}
//-------------- Request is not recognized -----------------
else {
ArduinoRequest = "Invalid Request";
}
//-------- Execute valid request & get reply string --------
if(os_strcmp(ArduinoRequest.c_str(), "Invalid Request")==0) {
payld = ArduinoRequest;
}
else {
rparam.request = ARDUINO_REQUEST;
rparam.requestval = 0;
Server_ExecuteRequest(servertype, rparam, &payld, ArduinoRequest);
}
Server_SendReply(servertype, REPLY_TEXT, payld);
}

Arduino Sketch

A simple Arduino sketch is provided here as an example of how to receive and process requests from the ESP8266. Using the low-end Arduino nano, this example uses a software serial port (Arduino digital pins 10 and 11) to communicate with the ESP8266.

Due to the limited bandwidth for reliable operation with this software port, the baud was reduced to 1200 baud. If a dedicated hardware serial port is used, such as found with the Arduino mega, the baud can be increased considerably.

The example provided is fully functional, yet simple enough to understand and build upon with even modest programming skills.

Part 2: Web-based ESP8266 Configuration

So there you have it, an ESP8266 web server sketch that supports Arduino communication using either MQTT or url-based http protocols. But there is more to this update. I’ve also added web-browser configuration of the Network and MQTT settings.

Read on here for the details…

Archives for January 2016

MQTT for App Inventor – Adding Configurable Settings

App Inventor to MQTT Communication

Adding Configurable Setting

App File Location

Configuration file

App Inventor File Read/Write

Updated Main Screen

JavaScript for Configurable MQTT

Improving the Arduino Digital Commands

Sending the Arduino Digital Request

Exiting the Application

In Closing

Share This:

MQTT For App Inventor

The Concept

Overall Communication Structure

App Inventor GUI

App Inventor Initialization Code

App Inventor Send MQTT Request Code

App Inventor Receive Reply Code

JavaScript For MQTT

JavaScript Initialization Code

JavaScript Receive Request Code

JavaScript Reply Callback Code

That’s it!

Share This:

Triple Server Update – Part 2: Web Configuration

Configurable Parameters

EEPROM Memory Map

Webpage Structure

ESP8266 Webpage Request/Reply Code

Webpage Expansion

In Closing

Credits

Share This:

Triple Server Update – Part 1: Serving Arduino

Share This:

Looking for something?

List All Post

Post Categories

Archived Posts

Website Tag Cloud

Recent Comments

Let’s keep in touch!

Follow Us