MQTT for App Inventor – More Configurable Settings

26th July 2016 by InternetOfHomeThings in All Posts, ESP8266 : An IoT System on Chip, Internet of Things, MQTT | 9 Comments

More MQTT Configurable Parameters via App Inventor

My last post on this subject introduced a few configurable setting to an MQTT App Inventor project. It was a no frills, bare bones version. But a few essential parameters were left out. This article fills in the some of the gaps, including connection authentication, websocket port assignment, timeout, last will and testimonial, and the clientID. With this update, the following MQTT settings become configurable:

MQTT Broker domain name
Request Topic
Reply Topic
User Name (Leave Blank if not used)
Password (Leave Blank if not used)
MQTT Websocket Port
Client ID
Keep Alive Timeout (seconds)
Last Will Topic
Last Will Message
Last Will QoS

NOTE: If you want to skip the implementation details presented below and simply use this project now, it is available on GitHub here.

A New Client Library

My initial App Inventor MQTT project used the Mosquitto client library. While that was great for developing the “proof-of-concept”, demonstrating that you could indeed link an App Inventor project to an MQTT broker, some serious shortcomings soon became evident.

When attempting to refine the project to support broker connections with authentication, I discovered that the Mosquitto library did not support this basic feature. So a new library was needed. Fortunately, an existing open-source client library is available with full support for password-enabled logins.

The library is called Eclipse Paho. And just as with my initial application, the API provides a JavaScript interface. This interface supports all the features identified above, configurable through App Inventor.

Expanding The Configurable Parameters

Here is how the expanded App Inventor configuration screen looks. When user name/password credentials are not used, the fields should remain blank. Just as before, the configuration settings are stored in an Android device file. Upon start-up, the system initializes with the values stored in that file (cfg.txt).

A last will message has been added to the set of App Inventor configurable parameters. As per the MQTT specification, the last will message is sent when the MQTT connection is closing.

Test Case

The HiveMQ broker “broker.mqttdashboard.com” was used to demonstrate the capabilities of this project. And the on-line client used is available at: http://www.hivemq.com/demos/websocket-client/

A simple test: Open a browser to the MQTT client: http://www.hivemq.com/demos/websocket-client/

Enter the Host broker as shown below and click “Connect”.

Subscribe to the App Inventor MQTT Request and Last Will topics:

Request Topic: mqtt_request

Last Will Topic: lwt

Open the updated MQTT App on your Android device. Click on the “Configure MQTT” icon.

Edit or take note of the last will topic and message. Close the App Inventor app and verify the HiveMQ client displays the last will message.

Note: Use a broker that supports username/password logins to verify that new configurable parameter pair. Refer to this post if you would like to setup your own MQTT broker with login authentication enabled.

JavaScript Updates

The App Inventor WebViewString is used to communicate between the application and the Paho MQTT JavaScript library. Here are the highlights of the revised JavaScript interface.

Paho MQTT API Fork

I started this project update with paho-javascript version 1.0.2. After debugging the changes using a web browser, the code was moved to the target Android device. Unfortunately, after much troubleshooting, an unsupported AppInventor “WebViewer” component feature was identified. The required feature for the Paho library is called “localStorage”.

Fortunately, there is an alternative to localStorage. I have modified the Paho library to use Cookies instead of localStorage. With this change, the Paho library/App Inventor communication has been verified to function properly.

Three sections of the library had to be modified:

Verify localStorage is supported (Comment out this check)

// Check dependencies are satisfied in this browser.
if (!("WebSocket" in global && global["WebSocket"] !== null)) {
throw new Error(format(ERROR.UNSUPPORTED, ["WebSocket"]));
}
//cookies used since localstorage not supported with appinventor
/*
if (!("localStorage" in global && global["localStorage"] !== null)) {
throw new Error(format(ERROR.UNSUPPORTED, ["localStorage"]));
}
*/

2. Replace localStorage with cookies

//localStorage.setItem(prefix+this._localKey+wireMessage.messageIdentifier, JSON.stringify(storedMessage));
setCookie(prefix+this._localKey+wireMessage.messageIdentifier, JSON.stringify(storedMessage), 1);
};
ClientImpl.prototype.restore = function(key) {
//var value = localStorage.getItem(key);
var value = getCookie(key);

3. Add Get/Set cookies functions

function setCookie(cname, cvalue, exdays) {
var d = new Date();
d.setTime(d.getTime() + (exdays*24*60*60*1000));
var expires = "expires="+d.toUTCString();
document.cookie = cname + "=" + cvalue + "; " + expires;
}
function getCookie(cname) {
var name = cname + "=";
var ca = document.cookie.split(';');
for(var i = 0; i < ca.length; i++) {
var c = ca[i];
while (c.charAt(0) == ' ') {
c = c.substring(1);
}
if (c.indexOf(name) == 0) {
return c.substring(name.length, c.length);
}
}
return "";
}

The AppInventor to PAHO MQTT API JavaScript

The MQTT broker is no longer automatically loaded after the webview page is loaded by the App Inventor application:

//executes once after window is loaded --------------------------------------------->
function windowloaded() {
//client.connect(connectOptions); // Connect to MQTT broker
AppInventorServer(); // Start polling WebViewString
}
window.onload = windowloaded; // Launch windowloaded()

The JavaScript must now receive a command to connect to the configured MQTT broker. The command received is called CFG, which loads the current connection settings stored in a file on the android device. The ensures the most current configuration is used for the connection.

There are 4 commands the JavaScript recognizes from the App Inventor App via the WebViewString.

GET: Used to send an MQTT request message from the AppInventor App
CFG: Update configurable parameters, disconnect and reconnect to MQTT broker
CNX: Connect to MQTT broker
KILLCNX: Disconnect from MQTT broker

//GET: Send MQTT message -------------------------------------------------------------
//------------------------------------------------------------------------------------
if(request.substring(0, 4)=="GET:") { // Validate request
window.AppInventor.setWebViewString(""); // Reset String (process once)
$("#request").val(request.substring(4, request.length)); //set request html textbox
SendMqttRequest(); // Send Mqtt Request
}
//CFG: Update configurable files in Android Device File ------------------------------
//------------------------------------------------------------------------------------
if(request.substring(0, 4) == "CFG:") { // Validate request
window.AppInventor.setWebViewString(""); // Reset String (process once)
if(typeof(client) !== 'undefined') { // Disconnect if connected
client.disconnect();
}
var cfgpar = JSON.parse(request.substring(4, request.length));
txtopic = cfgpar.mqtt_txtopic;
rxtopic = cfgpar.mqtt_rxtopic;
mqtt_url = cfgpar.mqtt_broker;
mqtt_port = Number(cfgpar.mqtt_port);
mqtt_clientId = cfgpar.mqtt_clientId;
mqtt_keepalive = cfgpar.mqtt_keepalive;
mqtt_lastwilltopic = cfgpar.mqtt_lastWillTopic;
mqtt_lastwillmessage = cfgpar.mqtt_lastWillMessage;
mqtt_lastwillqos = cfgpar.mqtt_lastWillQoS;
connectOptions.userName = cfgpar.mqtt_un;
connectOptions.password = cfgpar.mqtt_pw;
connectOptions.keepAliveInterval= Number(mqtt_keepalive);
// Create a Last-Will-and-Testament
var lwt = new Paho.MQTT.Message(mqtt_lastwillmessage);
lwt.destinationName = mqtt_lastwilltopic;
lwt.qos = Number(mqtt_lastwillqos);
lwt.retained = false;
connectOptions.willMessage = lwt;
client = new Paho.MQTT.Client(mqtt_url, mqtt_port, mqtt_clientId);
client.onConnectionLost = onConnectionLost;
client.onMessageArrived = onMessageArrived;
client.connect(connectOptions); // Connect to MQTT broker
}
//CNX: Connect to MQTT broker --------------------------------------------------------
//------------------------------------------------------------------------------------
if(request.substring(0, 4)=="CNX:") { // Validate request
window.AppInventor.setWebViewString(""); // Reset String (process once)
if(typeof(client) == 'undefined') {
// Create MQTT client instance -------------------------------------------------->
client = new Paho.MQTT.Client(mqtt_url, mqtt_port, mqtt_clientId);
// set callback handlers
client.onConnectionLost = onConnectionLost;
client.onMessageArrived = onMessageArrived;
}
client.connect(connectOptions); // Connect to MQTT broker
}
//KILLCNX: Diconnect from MQTT broker ------------------------------------------------
//------------------------------------------------------------------------------------
if(request.substring(0, 7)=="KILLCNX") { // Validate request
window.AppInventor.setWebViewString(""); // Reset String (process once)
conn_kill = "Y";
client.disconnect();
}
hAppInvSvr = setTimeout(AppInventorServer, 100); // run AppInventorServer() in 100 ms

The callbacks executed upon MQTT broker connection, disconnection, and receipt of subscribed message can easily be understood through a review of the JavaScript file (mqtt_appinventor_paho.html).

App Inventor Updates

Changes to the App Inventor code simply expand the scope of the configurable parameters. Again, a review of the code should make these changes self-evident.

Here is the code

The AppInventor and JavaScript code is available on GitHub here. Installation instructions are included in the readme.md file.

In Conclusion

There you have it. It is with great pleasure to present this update which supports username/password MQTT connections using the AppInventor. I hope you find this information useful.

MQTT For App Inventor

13th January 2016 by InternetOfHomeThings in All Posts, ESP8266 : An IoT System on Chip, Internet of Things | 15 Comments

App Inventor is a visual, easy-to-use online Android Application development platform. It is a great vehicle that empowers the do-it-yourself enthusiast to develop mobile IoT clients without battling the steep learning curve of traditional text-based development platforms.

Aside from the basic programming language constructs, all you need is a method of accessing your Internet enabled devices. And many developers prefer to use MQTT, a machine-to-machine connectivity protocol, to communicate externally with their IoT “things”.

But up to now, there has not been a lot of information available showing how to use MQTT with App Inventor. Here is what I have come up. It’s a working example of an Android App Inventor project using MQTT.

The Concept

When I first looking into this, it seemed that those looked into this were trying to adapt the existing App Inventor Web components to work with MQTT. But that just won’t work. MQTT requires a TCP connection to remain open for the entire time the App is running. And App Inventor only provides an http interface.

App Inventor also did not appear to provide a method of adding a c-based MQTT library. Another possibility was to interface with JavaScript. After all, I had already been successful in implementing an MQTT webpage using JavaScript.

But how can you interface with JavaScript using App Inventor?

After investigating App Inventor features deeper, the answer popped into my head like a light bulb flipping on. The solution was right there…

WebViewString .

That’s it! You can set the WebViewString from App Inventor. And JavaScript can read and act upon the contents of the WebViewString object. All I had to do was setup a basic communication scheme with this and MQTT communication would be possible.

Overall Communication Structure

Here is how it works…

App inventor sets the WebViewString to the MQTT topic payload that it wants to publish. Then, a JavaScript file monitoring the value of the WebViewString for MQTT requests picks up the payload and publishes it. The IoT device that subscribes to the topic then detects the request, acts on it, and returns a reply via an MQTT topic publication.

The rest of the sequence should be obvious. The JavaScript subscribing to the MQTT topic published by the IoT device then forwards the reply back to App Creator by changing the value of the WebViewString .

That’s it. And here is the implementation…

App Inventor GUI

For this example, the App Inventor GUI has intentionally been kept simple. Just one user action provided: Click on the button to select an MQTT request and wait for the reply string to appear.

Since this example uses the Triple Server Update ESP8266 sketch from my previous post, these are the options that can be selected when the button is clicked. They can be easily customized for your own system requirements.

These selections are currently just the raw strings sent to the IoT based MQTT server. The App Inventor code could be refined, if desired, to make it more user friendly. Such as using a channel selector input as well as a logic state selector (HI or LO). But for this example, raw strings are suffice.

App Inventor Initialization Code

Here is the App Inventor code used to initialize the MQTT request options. This code executes when the App’s one and only screen opens. Just change the text in the list for your own MQTT topic payloads.

Note that the JavaScript MQTT server file pointer is also initialized here. This file, locally stored in the Android SDcard, is open when the App starts. More about that later in this article.

App Inventor Send MQTT Request Code

When one of the request options are selected (picked), the WebViewString is set to this request value. It is prefaced with “GET:” so the JavaScript code knows what is to follow is an MQTT request.

App Inventor Receive Reply Code

Finally, the App Inventor continuously monitors the WebViewString for replies from the JavaScript. While I do not prefer to use polling loops, that was the only mechanism I was able to come up with for this. The polling timer is set to repeat every 100 ms, or 10 times per second.

WebViewString is first validated as a reply by confirming it is not blank and does not start with the request “GET:” prefix. If these conditions are met, the GUI reply string is set to the WebViewString value. Note that the WebViewString value is set to “” after read so this polling function does not process it more than once.

JavaScript For MQTT

Since I already had a working MQTT webpage from a prior project, it was simply reused for this example. Of course, the irrelevant parts were remove and the WebViewString object was added. While it can be saved anywhere, for this project, the file is stored on the Android device’s SD card. File installation instructions are provided in the GitHub repository for this project.

JavaScript Initialization Code

The initialization code first brings in the required MQTT and jquery libraries from online sources:

<script type="text/javascript" src="http://test.mosquitto.org/mosquitto.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js"></script>

Next, default values are set for the request, MQTT broker, and the request/reply topics. An instance of a Mosquitto client is also created,

// Set Default Request ------------------------------------------------------------->
$("#request").val("/?request=GetSensors");
// Set Default MQTT Url ------------------------------------------------------------>
var mqtturl = "ws://test.mosquitto.org:8080/mqtt";
var txtopic = "MyMqttSvrRqst";
var rxtopic = "MyMqttSvrRply";
// Create MQTT client instance ----------------------------------------------------->
client = new Mosquitto();
client.run = 0;

Finishing the initialization, the MQTT broker connection is established and the WebViewString polling loop is started.

//executes once after window is loaded --------------------------------------------->
function windowloaded() {
client.connect(mqtturl); // Connect to MQTT broker
AppInventorServer(); // Start polling WebViewString
}
window.onload = windowloaded; // Launch windowloaded()

JavaScript Receive Request Code

Here is the polling loop, which checks for new MQTT requests from App Inventor.

// 10 Hz WebViewString polling timer function -------------------------------------->
function AppInventorServer() {
var request = window.AppInventor.getWebViewString(); // Get WebViewString
if(request.substring(0, 4)=="GET:") { // Validate request
window.AppInventor.setWebViewString(""); // Reset String (process once)
$("#request").val(request.substring(4, request.length)); //set request html textbox
SendMqttRequest(); // Send Mqtt Request
}
setTimeout(AppInventorServer, 100); // run AppInventorServer() in 100 ms
}
// Publishes MQTT topic contained in html #request object -------------------------->
function SendMqttRequest() {
client.publish(txtopic,$("#request").val(),0,0);
}

If the WebViewString starts with “GET:”, it is considered to be valid. In that case, the WebViewString is first set to “” to avoid processing the request again at the next 100 ms interval. The request string is then stripped from the WebViewString and published as an MQTT topic. This polling loop restarts every 100 ms.

JavaScript Reply Callback Code

Whenever the subscribed MQTT topic message is received, a JavaScript callback function is executed.

// Callback executed upon receipt of MQTT message ---------------------------------->
client.onmessage = function(topic, payload, qos){
if(topic==rxtopic) {
setpayload(payload);
}
};
// Sets WebViewString and html payload text box to 'payld' ------------------------->
function setpayload(payld) {
window.AppInventor.setWebViewString(payld); // Set WebViewString
$("#payloadmqqt").html(payld); // Set html textbox
}

This callback simply passes the reply (MQTT topic payload) back to App Inventor through the WebViewString. An html textbox is also filled with this reply string.

That’s it!

The App Inventor project file and JavaScript html file is available on GitHub here. Instructions for installing the JavaScript file on an Android device is also provided at this same GitHub location.

Hope you find this information useful.

And stay tuned. My next step will be adding non-volatile configuration for the MQTT parameters. This will provide the capability to change MQTT brokers and topics without changing the contents of the installed html file.

Archives for App Inventor MQTT