MQTT for App Inventor – More Configurable Settings

26th July 2016 by InternetOfHomeThings in All Posts, ESP8266 : An IoT System on Chip, Internet of Things, MQTT | 9 Comments

configs

More MQTT Configurable Parameters via App Inventor

My last post on this subject introduced a few configurable setting to an MQTT App Inventor project. It was a no frills, bare bones version. But a few essential parameters were  left out. This article fills in the some of the gaps, including connection authentication, websocket port assignment, timeout, last will and testimonial, and the clientID. With this update, the following MQTT settings become configurable:

  • MQTT Broker domain name
  • Request Topic
  • Reply Topic
  • User Name (Leave Blank if not used)
  • Password (Leave Blank if not used)
  • MQTT Websocket Port
  • Client ID
  • Keep Alive Timeout (seconds)
  • Last Will Topic
  • Last Will Message
  • Last Will QoS

NOTE: If you want to skip the implementation details presented below and simply use this project now, it is available on GitHub here.

A New Client Library

My initial App Inventor MQTT project used the Mosquitto client library. While that was great for developing the “proof-of-concept”, demonstrating that you could indeed link an App Inventor project to an MQTT broker, some serious shortcomings soon became evident.

When attempting to refine the project to support broker connections with authentication, I discovered that the Mosquitto library did not support this basic feature. So a new library was needed. Fortunately, an existing open-source client library is available with full support for password-enabled logins.

The library is called Eclipse Paho. And just as with my initial application, the API provides a JavaScript interface. This interface supports all the features identified above, configurable through App Inventor.

Expanding The Configurable Parameters

Here is how the expanded App Inventor configuration screen looks. When user name/password credentials are not used, the fields should remain blank. Just as before, the configuration settings are stored in an Android device file. Upon start-up, the system initializes with the values stored in that file (cfg.txt).

mqtt_cfg

A last will message has been added to the set of App Inventor configurable parameters. As per the MQTT specification, the last will message is sent when the MQTT connection is closing.

Test Case

The HiveMQ broker “broker.mqttdashboard.com” was used to demonstrate the capabilities of this project. And the on-line client used is available at: http://www.hivemq.com/demos/websocket-client/

A simple test: Open a browser to the MQTT client: http://www.hivemq.com/demos/websocket-client/

Enter the Host broker as shown below and click “Connect”.

hiveMQ

Subscribe to the App Inventor MQTT Request and Last Will topics:

Request Topic: mqtt_request

Last Will Topic: lwt

subscribe

Open the updated MQTT App on your Android device. Click on the “Configure MQTT” icon.

new_cfg

Edit or take note of the last will topic and message. Close the App Inventor app and verify the HiveMQ client displays the last will message.

Note: Use a broker that supports username/password logins to verify that new configurable parameter pair. Refer to this post if you would like to setup your own MQTT broker with login authentication enabled.

JavaScript Updates

The App Inventor WebViewString is used to communicate between the application and the Paho MQTT JavaScript library. Here are the highlights of the revised JavaScript interface.

Paho MQTT API Fork

I started this project update with paho-javascript version 1.0.2. After debugging the changes using a web browser, the code was moved to the target Android device. Unfortunately, after much troubleshooting, an unsupported AppInventor “WebViewer” component feature was identified.  The  required feature for the Paho library is called “localStorage”.

Fortunately, there is an alternative to localStorage. I have modified the Paho library to use Cookies instead of localStorage. With this change, the Paho library/App Inventor communication has been verified to function properly.

Three sections of the library had to be modified:

  1. Verify localStorage is supported (Comment out this check)
 
 
  1. // Check dependencies are satisfied in this browser.
  2. if (!("WebSocket" in global && global["WebSocket"] !== null)) {
  3. throw new Error(format(ERROR.UNSUPPORTED, ["WebSocket"]));
  4. }
  5. //cookies used since localstorage not supported with appinventor
  6. /*
  7. if (!("localStorage" in global && global["localStorage"] !== null)) {
  8. throw new Error(format(ERROR.UNSUPPORTED, ["localStorage"]));
  9. }
  10. */

2. Replace localStorage with cookies

 
 
  1. //localStorage.setItem(prefix+this._localKey+wireMessage.messageIdentifier, JSON.stringify(storedMessage));
  2. setCookie(prefix+this._localKey+wireMessage.messageIdentifier, JSON.stringify(storedMessage), 1);
  3. };
  4. ClientImpl.prototype.restore = function(key) {
  5. //var value = localStorage.getItem(key);
  6. var value = getCookie(key);

3. Add Get/Set cookies functions

 
 
  1. function setCookie(cname, cvalue, exdays) {
  2.     var d = new Date();
  3.     d.setTime(d.getTime() + (exdays*24*60*60*1000));
  4.     var expires = "expires="+d.toUTCString();
  5.     document.cookie = cname + "=" + cvalue + "; " + expires;
  6. }
  7. function getCookie(cname) {
  8.     var name = cname + "=";
  9.     var ca = document.cookie.split(';');
  10.     for(var i = 0; i < ca.length; i++) {
  11.         var c = ca[i];
  12.         while (c.charAt(0) == ' ') {
  13.             c = c.substring(1);
  14.         }
  15.         if (c.indexOf(name) == 0) {
  16.             return c.substring(name.length, c.length);
  17.         }
  18.     }
  19.     return "";
  20. }

The AppInventor to PAHO MQTT API JavaScript

The MQTT broker is no longer automatically loaded after the webview page is loaded by the App Inventor application:

 
 
  1. //executes once after window is loaded --------------------------------------------->
  2. function windowloaded() {
  3. //client.connect(connectOptions);     // Connect to MQTT broker
  4.     AppInventorServer();                  // Start polling WebViewString
  5. }
  6. window.onload = windowloaded;  // Launch windowloaded()

The JavaScript must now receive a command to connect to the configured MQTT broker. The command received is called CFG, which loads the current connection settings stored in a file on the android device. The ensures the most current configuration is used for the connection.

There are 4 commands the JavaScript recognizes from the App Inventor App via the WebViewString.

  1. GET: Used to send an MQTT request message from the AppInventor App
  2. CFG: Update configurable parameters, disconnect and reconnect to MQTT broker
  3. CNX: Connect to MQTT broker
  4. KILLCNX: Disconnect from MQTT broker
 
 
  1. //GET: Send MQTT message -------------------------------------------------------------
  2. //------------------------------------------------------------------------------------
  3. if(request.substring(0, 4)=="GET:") {                // Validate request
  4. window.AppInventor.setWebViewString("");         // Reset String (process once)
  5. $("#request").val(request.substring(4, request.length)); //set request html textbox
  6. SendMqttRequest();                               // Send Mqtt Request
  7. }
  8. //CFG: Update configurable files in Android Device File ------------------------------
  9. //------------------------------------------------------------------------------------
  10. if(request.substring(0, 4) == "CFG:") { // Validate request
  11. window.AppInventor.setWebViewString(""); // Reset String (process once)
  12. if(typeof(client) !== 'undefined') { // Disconnect if connected
  13. client.disconnect();
  14. }
  15. var cfgpar = JSON.parse(request.substring(4, request.length));
  16. txtopic = cfgpar.mqtt_txtopic;
  17. rxtopic = cfgpar.mqtt_rxtopic;
  18. mqtt_url = cfgpar.mqtt_broker;
  19. mqtt_port = Number(cfgpar.mqtt_port);
  20. mqtt_clientId = cfgpar.mqtt_clientId;
  21. mqtt_keepalive = cfgpar.mqtt_keepalive;
  22. mqtt_lastwilltopic = cfgpar.mqtt_lastWillTopic;
  23. mqtt_lastwillmessage = cfgpar.mqtt_lastWillMessage;
  24. mqtt_lastwillqos = cfgpar.mqtt_lastWillQoS;
  25. connectOptions.userName = cfgpar.mqtt_un;
  26. connectOptions.password = cfgpar.mqtt_pw;
  27. connectOptions.keepAliveInterval= Number(mqtt_keepalive);
  28. // Create a Last-Will-and-Testament
  29. var lwt = new Paho.MQTT.Message(mqtt_lastwillmessage);
  30. lwt.destinationName = mqtt_lastwilltopic;
  31. lwt.qos = Number(mqtt_lastwillqos);
  32. lwt.retained = false;
  33. connectOptions.willMessage = lwt;
  34. client = new Paho.MQTT.Client(mqtt_url, mqtt_port, mqtt_clientId);
  35. client.onConnectionLost = onConnectionLost;
  36. client.onMessageArrived = onMessageArrived;
  37. client.connect(connectOptions); // Connect to MQTT broker
  38. }
  39. //CNX: Connect to MQTT broker --------------------------------------------------------
  40. //------------------------------------------------------------------------------------
  41. if(request.substring(0, 4)=="CNX:") { // Validate request
  42. window.AppInventor.setWebViewString(""); // Reset String (process once)
  43. if(typeof(client) == 'undefined') {
  44. // Create MQTT client instance -------------------------------------------------->
  45. client = new Paho.MQTT.Client(mqtt_url, mqtt_port, mqtt_clientId);
  46. // set callback handlers
  47. client.onConnectionLost = onConnectionLost;
  48. client.onMessageArrived = onMessageArrived;
  49. }
  50. client.connect(connectOptions); // Connect to MQTT broker
  51. }
  52. //KILLCNX: Diconnect from MQTT broker ------------------------------------------------
  53. //------------------------------------------------------------------------------------
  54. if(request.substring(0, 7)=="KILLCNX") { // Validate request
  55. window.AppInventor.setWebViewString(""); // Reset String (process once)
  56. conn_kill = "Y";
  57. client.disconnect();
  58. }
  59. hAppInvSvr = setTimeout(AppInventorServer, 100); // run AppInventorServer() in 100 ms

The callbacks executed upon MQTT broker connection, disconnection, and receipt of subscribed message can easily be understood through a review of the JavaScript file (mqtt_appinventor_paho.html).

App Inventor Updates

Changes to the App Inventor code simply expand the scope of the configurable parameters. Again, a review of the code should make these changes self-evident.

appinventor_update

Here is the code

The AppInventor and JavaScript code is available on GitHub here. Installation instructions are included in the readme.md file.

In Conclusion

There you have it. It is with great pleasure to present this update which supports username/password MQTT connections using the AppInventor. I hope you find this information useful.

Loading

Share This:
FacebooktwitterredditpinterestlinkedintumblrFacebooktwitterredditpinterestlinkedintumblr
Social tagging: > > >

9 Responses to MQTT for App Inventor – More Configurable Settings

  1. Enrique says:
    30th July 2016 at 1:02 am

    Thanks for this update, really useful.
    But I do not understand this “localStorage” issue. When I discovered that the old mqtt library was not available, I also changed to paho, and I have not found any issue, and the library worked as the previous one.

    Is it related to the new configuratin parameters?.

    I enclosed the code I am using now, based in the your first post.
    eHc.

    MQTT JavaScript Client Example


    ESP8266 MQTT Server

    Request:

    Request via MQTT

    ///////////////////////////////////////////////////////////
    // Javascript for MQTT Server Communication
    ///////////////////////////////////////////////////////////

    // Set Default Request ————————————————————->
    $(“#request”).val(“/ESTADO”);

    //Set Default MQTT Url ————————————————————->
    var mqtturl = “ws://test.mosquitto.org:8080/mqtt”;
    var txtopic = “eHcORD”;
    var rxtopic = “eHcRES”;

    var options = {
    // port: 8080,
    clientId: ‘javascript’,
    // userName: “”,
    // password: “”
    };

    // var client = mqtt.connect(mqtturl,options);

    var client = mqtt.connect(‘ws://test.mosquitto.org:8080/mqtt’, {
    clientId: ‘javascript’
    });

    // Create MQTT client instance —————————————————–>

    //executes once after window is loaded ———————————————>
    function windowloaded() {
    client.on(‘connect’, function(){
    console.log(‘client has connected to: ‘,mqtturl);
    client.subscribe(rxtopic);
    });

    AppInventorServer(); // Start polling WebViewString
    }
    window.onload = windowloaded; // Launch windowloaded()

    // 10 Hz WebViewString polling timer function ————————————–>
    function AppInventorServer() {
    var request = window.AppInventor.getWebViewString(); // Get WebViewString
    if(request.substring(0, 4)==”GET:”) { // Validate request
    window.AppInventor.setWebViewString(“”); // Reset String (process once)
    $(“#request”).val(request.substring(4, request.length)); //set request html textbox
    SendMqttRequest(); // Send Mqtt Request
    }
    setTimeout(AppInventorServer, 100); // run AppInventorServer() in 100 ms
    }

    // Callback executed upon MQTT broker disconnection ——————————–>
    client.ondisconnect = function(rc){
    mqtt.connect(mqtturl);
    };

    // Callback executed upon receipt of MQTT message ———————————->

    client.on(‘message’, function(topic, message) {
    console.log(‘new message:’, topic, message.toString());
    if(topic==rxtopic) {
    // setpayload(payload);
    setpayload(message.toString())
    }
    });

    // Callback for “Request via MQTT” button click ————————————>
    $(“#sendrequestmqtt”).click(function() {
    SendMqttRequest();
    });

    // Publishes MQTT topic contained in html #request object ————————–>
    function SendMqttRequest() {
    client.publish(txtopic,$(“#request”).val(),0,0);
    console.log(‘mensaje: ‘,$(“#request”).val())

    }

    // Sets WebViewString and html payload text box to ‘payld’ ————————->
    function setpayload(payld) {
    window.AppInventor.setWebViewString(payld);
    $(“#payloadmqqt”).html(payld);
    }

    ///////////////////////////////////////////////////////////
    // End of Javascript for MQTT Server Communication
    ///////////////////////////////////////////////////////////

    Reply
  2. Jon says:
    27th September 2016 at 11:35 am

    Hi, I'm trying to implement this project with a Raspberry Pi 2 instead of an ESP-8266. What sort of changes would I need to do to make this work? Would I just be putting the server on the Pi rather than the Esp?

    Reply
    • facebook-profile-picture InternetOfHomeThings says:
      27th September 2016 at 12:31 pm

      This project communicates with any device that subscribes to the same MQTT message. That device can be an ESP8266, a Raspberry Pi, Arduino or any other device that supports MQTT, I suggest you make a simple Raspberry Pi project that supports MQTT and get the communication working to an MQTT client. One possible MQTT client is mentioned in my article. I do not have any Raspberry Pis but I would say that it would most certainly support MQTT. <a href="https://learn.adafruit.com/diy-esp8266-home-security-with-lua-and-mqtt/configuring-mqtt-on-the-raspberry-pi" target="_blank">Here </a> is one guide to setting up the Raspberry Pi to support the MQTT interface. I found it with a quick google search. You might try it, and then add the code from my project, or search for another one that suites your needs. It really should not be that difficult to do with your Raspberry Pi.

      One note that I am compelled to share. All the experiences users have shared using the Raspberry Pi as an MQTT broker suggest you will not be pleased with the resulting performance. I suggest you use your Raspberry Pi as an MQTT client and use a separate platform to satisfy your MQTT broker need. I am using a small netbook as an MQTT broker for all my project, which works very well.

      Good luck with your project.

      Reply
  3. Jon says:
    28th September 2016 at 12:16 pm

    The goal is to have App Inventor talk to the RPi through MQTT. I do have a RPi running a broker and communicating with other MQTT client apps found on the Play Store. I also have node-red installed and comms working as well. However I am having problems with getting AI to RPi MQTT. Im not opposed to making the RPi a client and using a home PC as the broker, as long as I can still communicate and control the I/O's on the Arduino through App Inventor. Not sure what I'm doing wrong, but after alot of back and forth, still nothing.

    Reply
  4. Andre says:
    9th October 2016 at 11:19 pm

    It would be appropriate to rename the modified mqttws31.js

    Reply
  5. Andre says:
    12th October 2016 at 7:02 am

    Hello! Please tell me how to use WEB invisible component in your App Inventor project.
    Thank you.

    Reply
  6. Jichun says:
    17th April 2017 at 9:58 am

    Hi,
    does your code support TLS? I'm learning AWS IOT, it requires private key and certificate for mutual identification. Thanks.
    Jichun

    Reply
    • facebook-profile-picture InternetOfHomeThings says:
      19th April 2017 at 5:10 pm

      My <a href="https://wp.me/p5NRQ8-sk" target="_blank">latest post</a> provides instructions for setting up a secure SSL/TLS MQTT Broker. Next will be a secure ESP8266 MQTT client connection to the broker example. Eventually, I intend to update the AppInventor code to support secure TLS connections, but the code presented currently does not support TLS.

      Reply
  7. Jichun says:
    20th April 2017 at 12:22 am

    Hi,

    thanks very much.

    Jichun

    Reply

Leave a Reply

Press Ctrl+C to copy the following code.
"