Internet of Things

ESP8266 as a CoAP Client

coap1

Several examples can be found to configure the ESP8266 as a CoAP server. Last year I too had developed one. Here is my working example.

But what about using the device as a CoAP client? Using the ESP8266 to send requests to a CoAP server? With a two-ESP8266 module setup, this would provide the capability to set up a CoAP client/server architecture.

While I was unable to find a working example, much to my surprise, it was not that difficult to develop a client from my working server. Here is how it was done…

Turning the tables

After reviewing my ESP8266 CoAP server code, the seeds for a client were obvious. The server listens for CoAP request, acts on it, and sends a reply. To make a client, all that is necessary is to reverse the order.

First, use the server’s reply code to send a request to a server. Then simply listen for a reply from the CoAP server using the “listen” code.

Communication Structure 

This project uses a standard http server to initiate requests to a CoAP server. With this structure, a standard web browser can be used to test the CoAP client functionality.


coap_client

 

Separate Requests and Replies 

Using http request to initiate a CoAP client request presents a slight complication. That is, I found that checking for a CoAP reply (UDP) from a request while the http (TCP) connection was open caused the ESP8266 to crash. The problem was that the TCP connection callback function blocks the CoAP UDP reply packet. It was stuck waiting until the ESP8266 watchdog timed out, and then… Crash!

One solution is to use an Asynchronous TCP library such as the one found here. But that is beyond the scope of this simple ESP8266 CoAP client project. However, I do intend to explore that option in a future post on this subject.

But for now, it was easier to resolve this problem by breaking down an http GET initiated CoAP communication  exchange into two parts. The first http GET sends a CoAP request to the target CoAP server. After the TCP connection is closed. the CoAP UDP reply is received and stored. A second http GET is then needed to retrieve the CoAP reply.

 Sending a CoAP Request 

A CoAP request is comprised of a UDP packet per RFC7252. This example uses a simple CoAP server installed on a second ESP8266 that supports just 3 actions.

  1. Turn Led On/Off
  2. Blink Led
  3. Get Sensor Values

The CoAP UDP request packets are pre-defined in unique byte arrays:

 
 
  1. uint8_t packetbuf_LED_ON[]     = {0x40, 0x03, 0x00, 0x00, 0xB5, 0x6C, 0x69, 0x67, 0x68, 0x74, 0xFF, 0x31};
  2. uint8_t packetbuf_LED_OFF[]    = {0x40, 0x03, 0x00, 0x00, 0xB5, 0x6C, 0x69, 0x67, 0x68, 0x74, 0xFF, 0x30};
  3. uint8_t packetbuf_LED_BLINK[]  = {0x40, 0x03, 0x00, 0x00, 0xBB, 0x6C, 0x69, 0x67, 0x68, 0x74, 0x5F, 0x62, 
  4. 0x6C, 0x69, 0x6E, 0x6B, 0xFF, 0x35};
  5. uint8_t packetbuf_GETSENSORS[] = {0x40, 0x03, 0x00, 0x00, 0xB7, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 
  6. 0xFF, 0x2F, 0x3F, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x3D, 0x47, 
  7. 0x65, 0x74, 0x53, 0x65, 0x6E, 0x73, 0x6F, 0x72, 0x73};

The first byte (0x40) identifies the CoAP version as 1.0. The ‘0’ in the lower 4 bits indicate no tokens will follow the message header. The second byte (0x03) identifies the CoAP code as 0.03, indicating the PUT method used in the UDP request packet. Bytes 3 and 4 contain the messageID. This value starts at 0 upon ESP8266 startup and is incremented every time a message is sent.

The lower 4 bits identify the packet option length and is followed by the option value. Then, the byte following the option is the payload deliminator, and is set to oxFF.

Size (bytes) Value (Hex) Value(char) Parameter
LED ON 5 6C 69 67 68 74 light 1
LED OFF 5 6C 69 67 68 74 light 0
LED BLINK 11 6C 69 67 68 74 5F 62 6C 69 6E 6B light_blink 0-9
GET SENSORS 72 65 71 75 65 73 74 request /?request=GetSensors

The packet payload, which follows the 0xFF deliminator, contains the request message. As shown in the table above, the parameter is set to 1 to turn the CoAP server ESP8266 circuit LED on and 0 to turn the LED off.

And just as with other web server request, the CoAP requests are sent (udp_send) after an http GET is received.

 
 
  1.                 // Send CoAP Request to ESP8266 (192.168.0.141)
  2.                 else if(os_strcmp(pURL_Param->pParVal[0], "CoAPLedOn")==0) {
  3.                     coaplen = sizeof(packetbuf_LED_ON);
  4.                     incrMessageID(packetbuf_LED_ON);            //Increment Mesaage ID 
  5.                     udp_send(packetbuf_LED_ON, coaplen);
  6.                     payld = "http GET 'CoAPGetReply' required to get CoAP reply"; 
  7.                 }
  8.                 // Send CoAP Request to ESP8266 (192.168.0.141)
  9.                 else if(os_strcmp(pURL_Param->pParVal[0], "CoAPLedOff")==0) {
  10.                     coaplen = sizeof(packetbuf_LED_OFF);
  11.                     incrMessageID(packetbuf_LED_OFF);          //Increment Mesaage ID 
  12.                     udp_send(packetbuf_LED_OFF, coaplen);
  13.                     payld = "http GET 'CoAPGetReply' required to get CoAP reply"; 
  14.                 }
  15.                 // Send CoAP Request to ESP8266 (192.168.0.141)
  16.                 else if( (os_strcmp(pURL_Param->pParVal[0], "CoAPLedBlink")==0)&&(os_strcmp(pURL_Param->pParam[1], "cnt")==0) ) {
  17.                     coaplen = sizeof(packetbuf_LED_BLINK);
  18.                     //Insert Blink Count (1-9)
  19.                     *pURL_Param->pParVal[1] = (*pURL_Param->pParVal[1] < 0x31) ? 0x31 : *pURL_Param->pParVal[1];
  20.                     *pURL_Param->pParVal[1] = (*pURL_Param->pParVal[1] > 0x39) ? 0x39 : *pURL_Param->pParVal[1];
  21.                     packetbuf_LED_BLINK[coaplen-1] = *pURL_Param->pParVal[1];
  22.                     incrMessageID(packetbuf_LED_BLINK);       //Increment Mesaage ID 
  23.                     udp_send(packetbuf_LED_BLINK, coaplen);   //Send Packet
  24.                     payld = "http GET 'CoAPGetReply' required to get CoAP reply"; 
  25.                }
  26.                 // Send CoAP Request to ESP8266 (192.168.0.141)
  27.                 else if(os_strcmp(pURL_Param->pParVal[0], "CoAPGetSensors")==0) {
  28.                     coaplen = sizeof(packetbuf_GETSENSORS);
  29.                     incrMessageID(packetbuf_GETSENSORS);     //Increment Mesaage ID 
  30.                     udp_send(packetbuf_GETSENSORS, coaplen);
  31. payld = "http GET 'CoAPGetReply' required to get CoAP reply";
  32.                 }
  33.                 // Send CoAP Request to ESP8266 (192.168.0.141)
  34.                 else if(os_strcmp(pURL_Param->pParVal[0], "CoAPGetReply")==0) {
  35.                     payld = String(coap_reply_c);
  36.                 }

Receiving the CoAP Reply 

Note in the above code that the CoAP reply is set (payld) after the UDP packet it sent to the CoAP server. But the reply simply states that you must send a ‘CoAPGetReply’ request in order to get the CoAP reply. As noted previously, that is due to the blocking nature of the TCP callback.

The actual reply is stored in the global character string ‘coap_reply_c’.To get the reply, simply send the second GET request:

<ESP_IP>:<ESP_PORT>/?request=CoAPGetReply

Example Transactions

So here is the browser based requests you can use the test this CoAP server. This example assumes the ESP IP is 192.168.0.141 and the TCP port is set to 9706.

URL (192.168.0.141:9706) Suffix http Reply
Set CoAP Server LED Off /?request=CoAPLedOff http GET 'CoAPGetReply' required to get CoAP reply
Set CoAP Server LED Off /?request=CoAPGetReply 0
Set CoAP Server LED On /?request=CoAPLedOn http GET 'CoAPGetReply' required to get CoAP reply
Set CoAP Server LED On /?request=CoAPGetReply 1
Set CoAP Server LED Blinking 3 times /?request=CoAPLedBlink&cnt=3 http GET 'CoAPGetReply' required to get CoAP reply
Get CoAP Server Sensor Values /?request=CoAPGetSensors http GET 'CoAPGetReply' required to get CoAP reply
Get CoAP Server Sensor Values /?request=CoAPGetReply { "Ain0":"142.00", "Ain1":"143.00", "Ain2":"143.00", "Ain3":"145.00", "Ain4":"144.00", "Ain5":"145.00", "Ain6":"144.00", "Ain7":"140.00", "SYS_Heap":"32896", "SYS_Time":"37" }

This project’s GitHub Repository 

In Closing 

There you have it. A simple ESP8266 CoAP client structure to use in your custom application. But this is just a starting framework. There are obvious enhancements that can be made to this working example. Here are some of the things I would do next…

  1. Add web configurable options – Example
  2. Revise to support Async TCP
  3. Implementation of additional RFC7252 options

Hope you find this information useful…

Share This:
Facebooktwittergoogle_plusredditpinterestlinkedintumblrFacebooktwittergoogle_plusredditpinterestlinkedintumblr

MQTT for App Inventor – More Configurable Settings

configs

More MQTT Configurable Parameters via App Inventor

My last post on this subject introduced a few configurable setting to an MQTT App Inventor project. It was a no frills, bare bones version. But a few essential parameters were  left out. This article fills in the some of the gaps, including connection authentication, websocket port assignment, timeout, last will and testimonial, and the clientID. With this update, the following MQTT settings become configurable:

  • MQTT Broker domain name
  • Request Topic
  • Reply Topic
  • User Name (Leave Blank if not used)
  • Password (Leave Blank if not used)
  • MQTT Websocket Port
  • Client ID
  • Keep Alive Timeout (seconds)
  • Last Will Topic
  • Last Will Message
  • Last Will QoS

NOTE: If you want to skip the implementation details presented below and simply use this project now, it is available on GitHub here.

A New Client Library

My initial App Inventor MQTT project used the Mosquitto client library. While that was great for developing the “proof-of-concept”, demonstrating that you could indeed link an App Inventor project to an MQTT broker, some serious shortcomings soon became evident.

When attempting to refine the project to support broker connections with authentication, I discovered that the Mosquitto library did not support this basic feature. So a new library was needed. Fortunately, an existing open-source client library is available with full support for password-enabled logins.

The library is called Eclipse Paho. And just as with my initial application, the API provides a JavaScript interface. This interface supports all the features identified above, configurable through App Inventor.

Expanding The Configurable Parameters

Here is how the expanded App Inventor configuration screen looks. When user name/password credentials are not used, the fields should remain blank. Just as before, the configuration settings are stored in an Android device file. Upon start-up, the system initializes with the values stored in that file (cfg.txt).

mqtt_cfg

A last will message has been added to the set of App Inventor configurable parameters. As per the MQTT specification, the last will message is sent when the MQTT connection is closing.

Test Case

The HiveMQ broker “broker.mqttdashboard.com” was used to demonstrate the capabilities of this project. And the on-line client used is available at: http://www.hivemq.com/demos/websocket-client/

A simple test: Open a browser to the MQTT client: http://www.hivemq.com/demos/websocket-client/

Enter the Host broker as shown below and click “Connect”.

hiveMQ

Subscribe to the App Inventor MQTT Request and Last Will topics:

Request Topic: mqtt_request

Last Will Topic: lwt

subscribe

Open the updated MQTT App on your Android device. Click on the “Configure MQTT” icon.

new_cfg

Edit or take note of the last will topic and message. Close the App Inventor app and verify the HiveMQ client displays the last will message.

Note: Use a broker that supports username/password logins to verify that new configurable parameter pair. Refer to this post if you would like to setup your own MQTT broker with login authentication enabled.

JavaScript Updates

The App Inventor WebViewString is used to communicate between the application and the Paho MQTT JavaScript library. Here are the highlights of the revised JavaScript interface.

Paho MQTT API Fork

I started this project update with paho-javascript version 1.0.2. After debugging the changes using a web browser, the code was moved to the target Android device. Unfortunately, after much troubleshooting, an unsupported AppInventor “WebViewer” component feature was identified.  The  required feature for the Paho library is called “localStorage”.

Fortunately, there is an alternative to localStorage. I have modified the Paho library to use Cookies instead of localStorage. With this change, the Paho library/App Inventor communication has been verified to function properly.

Three sections of the library had to be modified:

  1. Verify localStorage is supported (Comment out this check)
 
 
  1. // Check dependencies are satisfied in this browser.
  2. if (!("WebSocket" in global &amp;&amp; global["WebSocket"] !== null)) {
  3. throw new Error(format(ERROR.UNSUPPORTED, ["WebSocket"]));
  4. }
  5. //cookies used since localstorage not supported with appinventor
  6. /*
  7. if (!("localStorage" in global &amp;&amp; global["localStorage"] !== null)) {
  8. throw new Error(format(ERROR.UNSUPPORTED, ["localStorage"]));
  9. }
  10. */

2. Replace localStorage with cookies

 
 
  1. //localStorage.setItem(prefix+this._localKey+wireMessage.messageIdentifier, JSON.stringify(storedMessage));
  2. setCookie(prefix+this._localKey+wireMessage.messageIdentifier, JSON.stringify(storedMessage), 1);
  3. };
  4. ClientImpl.prototype.restore = function(key) {
  5. //var value = localStorage.getItem(key);
  6. var value = getCookie(key);

3. Add Get/Set cookies functions

 
 
  1. function setCookie(cname, cvalue, exdays) {
  2.     var d = new Date();
  3.     d.setTime(d.getTime() + (exdays*24*60*60*1000));
  4.     var expires = "expires="+d.toUTCString();
  5.     document.cookie = cname + "=" + cvalue + "; " + expires;
  6. }
  7. function getCookie(cname) {
  8.     var name = cname + "=";
  9.     var ca = document.cookie.split(';');
  10.     for(var i = 0; i &lt; ca.length; i++) {
  11.         var c = ca[i];
  12.         while (c.charAt(0) == ' ') {
  13.             c = c.substring(1);
  14.         }
  15.         if (c.indexOf(name) == 0) {
  16.             return c.substring(name.length, c.length);
  17.         }
  18.     }
  19.     return "";
  20. }

The AppInventor to PAHO MQTT API JavaScript

The MQTT broker is no longer automatically loaded after the webview page is loaded by the App Inventor application:

 
 
  1. //executes once after window is loaded ---------------------------------------------&gt;
  2. function windowloaded() {
  3. //client.connect(connectOptions);     // Connect to MQTT broker
  4.     AppInventorServer();                  // Start polling WebViewString
  5. }
  6. window.onload = windowloaded;  // Launch windowloaded()

The JavaScript must now receive a command to connect to the configured MQTT broker. The command received is called CFG, which loads the current connection settings stored in a file on the android device. The ensures the most current configuration is used for the connection.

There are 4 commands the JavaScript recognizes from the App Inventor App via the WebViewString.

  1. GET: Used to send an MQTT request message from the AppInventor App
  2. CFG: Update configurable parameters, disconnect and reconnect to MQTT broker
  3. CNX: Connect to MQTT broker
  4. KILLCNX: Disconnect from MQTT broker
 
 
  1. //GET: Send MQTT message -------------------------------------------------------------
  2. //------------------------------------------------------------------------------------
  3. if(request.substring(0, 4)=="GET:") {                // Validate request
  4. window.AppInventor.setWebViewString("");         // Reset String (process once)
  5. $("#request").val(request.substring(4, request.length)); //set request html textbox
  6. SendMqttRequest();                               // Send Mqtt Request
  7. }
  8. //CFG: Update configurable files in Android Device File ------------------------------
  9. //------------------------------------------------------------------------------------
  10. if(request.substring(0, 4) == "CFG:") { // Validate request
  11. window.AppInventor.setWebViewString(""); // Reset String (process once)
  12. if(typeof(client) !== 'undefined') { // Disconnect if connected
  13. client.disconnect();
  14. }
  15. var cfgpar = JSON.parse(request.substring(4, request.length));
  16. txtopic = cfgpar.mqtt_txtopic;
  17. rxtopic = cfgpar.mqtt_rxtopic;
  18. mqtt_url = cfgpar.mqtt_broker;
  19. mqtt_port = Number(cfgpar.mqtt_port);
  20. mqtt_clientId = cfgpar.mqtt_clientId;
  21. mqtt_keepalive = cfgpar.mqtt_keepalive;
  22. mqtt_lastwilltopic = cfgpar.mqtt_lastWillTopic;
  23. mqtt_lastwillmessage = cfgpar.mqtt_lastWillMessage;
  24. mqtt_lastwillqos = cfgpar.mqtt_lastWillQoS;
  25. connectOptions.userName = cfgpar.mqtt_un;
  26. connectOptions.password = cfgpar.mqtt_pw;
  27. connectOptions.keepAliveInterval= Number(mqtt_keepalive);
  28. // Create a Last-Will-and-Testament
  29. var lwt = new Paho.MQTT.Message(mqtt_lastwillmessage);
  30. lwt.destinationName = mqtt_lastwilltopic;
  31. lwt.qos = Number(mqtt_lastwillqos);
  32. lwt.retained = false;
  33. connectOptions.willMessage = lwt;
  34. client = new Paho.MQTT.Client(mqtt_url, mqtt_port, mqtt_clientId);
  35. client.onConnectionLost = onConnectionLost;
  36. client.onMessageArrived = onMessageArrived;
  37. client.connect(connectOptions); // Connect to MQTT broker
  38. }
  39. //CNX: Connect to MQTT broker --------------------------------------------------------
  40. //------------------------------------------------------------------------------------
  41. if(request.substring(0, 4)=="CNX:") { // Validate request
  42. window.AppInventor.setWebViewString(""); // Reset String (process once)
  43. if(typeof(client) == 'undefined') {
  44. // Create MQTT client instance --------------------------------------------------&gt;
  45. client = new Paho.MQTT.Client(mqtt_url, mqtt_port, mqtt_clientId);
  46. // set callback handlers
  47. client.onConnectionLost = onConnectionLost;
  48. client.onMessageArrived = onMessageArrived;
  49. }
  50. client.connect(connectOptions); // Connect to MQTT broker
  51. }
  52. //KILLCNX: Diconnect from MQTT broker ------------------------------------------------
  53. //------------------------------------------------------------------------------------
  54. if(request.substring(0, 7)=="KILLCNX") { // Validate request
  55. window.AppInventor.setWebViewString(""); // Reset String (process once)
  56. conn_kill = "Y";
  57. client.disconnect();
  58. }
  59. hAppInvSvr = setTimeout(AppInventorServer, 100); // run AppInventorServer() in 100 ms

The callbacks executed upon MQTT broker connection, disconnection, and receipt of subscribed message can easily be understood through a review of the JavaScript file (mqtt_appinventor_paho.html).

App Inventor Updates

Changes to the App Inventor code simply expand the scope of the configurable parameters. Again, a review of the code should make these changes self-evident.

appinventor_update

Here is the code

The AppInventor and JavaScript code is available on GitHub here. Installation instructions are included in the readme.md file.

In Conclusion

There you have it. It is with great pleasure to present this update which supports username/password MQTT connections using the AppInventor. I hope you find this information useful.

Share This:
Facebooktwittergoogle_plusredditpinterestlinkedintumblrFacebooktwittergoogle_plusredditpinterestlinkedintumblr

USB LAMP Web Server Part 6 – Code Compiler

All Under One Roof

The LAMP server in this series has been built using the Precise Puppy Linux Distribution. But in my case, I had experimented with several different distributions. And my development environment for compiling  code was on a different distribution of Linux than the LAMP. In addition, the main application in this other Linux distribution, an MQTT Broker, is needed often for IoT setups.

This became very messy. Swapping out the USB sticks for different projects. The solution is obvious. Consolidate the development environment and the MQTT Broker into the USB LAMP Server.

One USB stick for everything…

Web Server, mySQL, WordPress, MQTT Broker, and other application that needs to be compiled.

These added capabilities have been broken down into two parts:

USB LAMP Web Server Part 6 –  Code Compiler

USB LAMP Web Server Part 7 –  MQTT Broker

Adding the Code Compiler

First thing to do is to get the Linux development tools for the Precise Puppy Distribution. They are bundled in the file “devx_precise_5.7.sfs”, which you can download here.

Copy this file to the USB LAMP drive root directory (from a Windows OS computer).

Now let’s configure the USB LAMP to startup with the development tools. Here is how to do it…

Start the USB LAMP.

First, click on the menu in the lower left hand of the start-up screen, selecting:

Menu>System>BootManager configure bootup

Click on the icon to the right of “Choose which extra SFS files to load at bootup:”

Select “devx_precise_5.7.sfs” and click “OK”.

Now close all the open windows and reboot Linux from the menu.

Menu>Exit>Reboot

That’s it!

Everything you need to create and execute programs are now installed. To test this setup, let’s code and run the classic “Hello World!” program…

Let’s keep things tidy by creating a folder for programs we create. And to keep things visible when we use the USB stick with the Windows OS running, the folder needs to be located under the “Home” folder. That is the Linux folder that appears as the “root” folder under the drive letter assigned by Windows.

To navigate to the Home folder using the Puppy GUI, first click on the “file” icon from the startup screen.

compiler_file

Click on the left “up arrow” icon to get to the parent folder.

parent-folder

Then select the “mnt” folder.

folder-mnt-sel

And finally, the “Home” folder.

home

To create a folder, right-click the mouse in the home folder list of files and select:

New>Directory

folder-create

Change “NewDir” to “MyPrograms” and click on the “Create” button.

folder-create-MyPrograms

Now click on the “MyPrograms” folder. We shall now create one additional folder under this one for our first program. Just like before, to create a folder, right-click the mouse in the MyPrograms folder list of files and select:

New>Directory

Change “NewDir” to “HelloWorld” and click on the “Create” button.

Now click on the “HelloWorld” folder. This time, to create our source code file, right-click the mouse in the HelloWorld folder list of files and select:

New>Blank file

folder-MyPrograms-Blank

Change “NewFile” to “HelloWorld.c” and click on the “Create” button.

Click on the file listed as “HelloWorld.c” to open the file editor.

select-helloworld-file

For the minimal first program, enter the following:

save-helloworld

Click on the “save” icon to finish the creation of this file.

Now we need a “Makefile” to define how to compile and build the program. This will be a minimal Makefile for this first program. Just like the “c” file, create a file named “Makefile” and enter the following contents:

makefile

Got it?

Now we are ready to build and execute the program. This can be done two different ways.

  1. Using the GUI editor we currently have open.
  2. From the command line.

Since the GUI file editor is already open, let’s first build and run the program from that environment. This is really easy. Like 1-2-3…

Build and Execute Using the GUI

Step 1: Select the “HelloWord.c” source code file tab.
Step 2: Click on the “Build” icon.

build-helloworld
Step 3: Click on the “Execute” icon.

execute-helloworld

Our one-line “prinf” command correctly displays on the console when the program is executed.

That’s great for a quick compile and run. But what I found is that if your program contains errors, nothing is displayed indicating what went wrong. Fortunately, there is a way to get needed error messages, when they occur.  It requires you to use the command line to build and execute.

Fear not! It really is not that difficult…

Build and Execute Using the Command Line

Now let’s do the same thing from the command line. To open a command line window, click on the “console” icon.

compiler_console

Then switch to the HelloWorld Folder:

change-dir-hello

You can view a directory listing now to verify the source and “Makefile” files are present with the “ls” command.

dir-listing

Now we can build the program with the command “gcc -o HelloWorld HelloWorld.c”. And run the program with the “./HelloWorld” command.

cmd-line-demo

That’s it. You are now setup to build and execute programs built for Linux.

Now let’s see what happens if we put an error into the program. Enter a bogus line after the printf statement. Like “junk;”, for example. When we try to “make” this program with the error injected, an error is returned. Unlike the GUI, with no error information provided, this provides you with an indication of what went wrong.

So you can quickly correct the error.

error

Conclusions

Here you have a simple reference to use to setup a Linux development environment. It is another tool to add to your “bag of tricks”. You never know when it will be needed. Like when the only solution available must be run using Linux.

Now that we have the development environment setup, we are set to build and configure an MQTT Broker on our USB LAMP. That will be the topic covered in my next post.

Hope you find this information useful.

Share This:
Facebooktwittergoogle_plusredditpinterestlinkedintumblrFacebooktwittergoogle_plusredditpinterestlinkedintumblr

USB LAMP Web Server Part 5 – WordPress

wordpress_icon

Alas we come to the final installment of this 5-part series. Last but not least, let’s install WordPress on our private USB-based LAMP server. And in case you just joined us, here is a link to the first part, which also provides references to all the other pieces to this puzzle.

WordPress is the most widely used Content Management System (CMS) used today. As of April, 2016, 22% of all new website use WordPress. With all the options available today, that is impressive market penetration!

Major Shortcut!!!

And, just like with my last post, for those who do not want to go through all the setup steps, a Linux session save file is included with this post.

Simply replace your existing save file with this one and you will be 99% complete with a web server that supports both php and mySQL server. Here it is.

Now let’s get to the step-by-step instructions…

Adding WordPress to the USB Web Server

Step 1: Installing the WordPress Package.

Start the USB installed Precision Puppy Linux from part 4. Then launch the Puppy Package Manager. In the search window (Find), enter “wordpress” and search all repositories.

Select wordpress_3.3.1 (or higher revision if available).

Then click “Examine dependencies”   and “Download and install selected packages”.

Now close and restart the Puppy Package Manager. In the search window (Find), enter “php5-gd” and search all repositories.

Select php5-gd_5.3.10 (or higher revision if available)

If the Description says “ALREADY INSTALLED”, then quit Puppy Package Manager
and continue to the next step.

Otherwise, click “Examine dependencies”   and “Download and install selected packages”.

Step 2: Create a WordPress Database and user in mySQL.

Start the mySQL server and open 2 console windows.

In the first windows, start the mySQL server by entering (adjust IP to your conversion):

# mysqld –bind-address 192.168.0.11

In the second widow, start the Apache Server by entering:

# /etc/init.d/apache2 restart

On another computer on the same network as the LAMP server, enter the following in a web browser:

http://192.168.0.11:9777/phpmyadmin/


wordpress_mysql1

Use the Username “root” with no password and click “Go” to enter phpMyAdmin, the mySQL database administration tool installed in part 4 of this series of USB LAMP server articles.

From the page that is now displayed, click on the databases tab.

wordpress_mysql2

Now lets create a database called “wordpress” to check out our installation.  Enter “wordpress” for the database name and click “Create”:

wordpress_mysql3

The new database will appear in the list of databases. We are now going to link a user to this database. Click “Check Privileges”.

wordpress_mysql4

 

 

Then click “Add a new user”.

wordpress_mysql5

 

 

 

Enter “demowpuser” in the “User name”, “Password”, and “Re-type” fields. Then click “Go” at the bottom of this page.

wordpress_mysql6

This demo WordPress database setup is now complete.

Step 3: Get the WordPress files.

The WordPress files can be downloaded either from a separate computer or the USB LAMP server. Assuming you are using the USB LAMP server:

Open the Puppy Linux Web Browser by clicking on the “browse” icon on the desktop. Then got to the URL: https:/wordpress.org/download.

Click “Download WordPress”.

Save the zip file to “/mnt/home”.

Using the Pupply Linux file Manager (Menu>Filesystem>ROX-Filer file manager), navigate to “/mnt/home” and click on the WordPress zip file that was downloaded.

Right-click on the “wordpress” folder and select “Extract”.

wordpress_zipfile

Enter “/var/www” as the “Directory to extract to”.

wordpress_extract

Click “OK”.

Step 4: Setup the WordPress Configuration

Using the ROX-Filer file manager, navigate to “/var/www/wordpress”.

Open the file “wp-config-sample.php” (right-click on it and select “Open as Text”).

Find the section that contains:

// ** MySQL settings - You can get this info from your web host ** //
/** The name of the database for WordPress */
define('DB_NAME', 'database_name_here');

/** MySQL database username */
define(‘DB_USER’, ‘username_here’);

/** MySQL database password */
define(‘DB_PASSWORD’, ‘password_here’);

Change it to:

// ** MySQL settings - You can get this info from your web host ** //
/** The name of the database for WordPress */
define('DB_NAME', 'wordpress');

/** MySQL database username */
define(‘DB_USER’, ‘demowpuser’);

/** MySQL database password */
define(‘DB_PASSWORD’, ‘demowpuser’);
Save the file as wp-config.php and exit the file

wordpress_config

Step 5: Set folder permissions

Open a console window and enter:

# chmod -R o777 /var/www/wordpress

Step 6: Setting up your installation

Start your mySQL database and Apache servers:

  • Open two console windows from Puppy Linux
  • In the first window, start the mySQL database server by entering:

# mysqld –bind-address 192.168.0.11

  • In the second window, start the Apache Server by entering:

# /etc/init.d/apache2 restart

Now a web-browser on your network.

Assuming the Apache Server IP is: 192.168.0.11 (Change per your installation)

and the Apache listening port is 9777 (Change per your installation)

Enter the URL:

http://192.168.0.11:9777/wordpress/wp-admin

The screen should look similar to the following:

wordpress_mysql7

Congratulations, if you made it this far, your LAMP WordPress installation is successfully completed!

After filling out the form and clicking “Install WordPress”, you should receive a “Success! Prompt, indicating the installation is finished. Click “Login” to open the WordPress Login page.

From here, you can perform any and all of the actions the WordPress content management system offers. Like picking a theme, adding plug-ins, writing and posting blog entries, etc.

You can even copy an existing WordPress site to your USB LAMP server, copying the folder to the Apache server root folder:

/var/www/<YOURWORDPRESSFOLDER>

For more information, just look on-line at wordpress.org or the wealth of information available through a google search. Here is a great place to start.

In Closing

That’s it! If you have completed all 5 parts of this series, you now have a solid server platform to work from. What is great about this setup is that it is portable. Allowing you to run it anywhere, on any i386 computer.

I hope this information serves as a practical guide and reference for all that come across it…

 

Share This:
Facebooktwittergoogle_plusredditpinterestlinkedintumblrFacebooktwittergoogle_plusredditpinterestlinkedintumblr

USB LAMP Web Server Part 4 – phpMyAdmin for mySQL Administration

phpmyadmin

My last post completed the USB LAMP web server. And at this point, the server already has the powerful essential foundation. Here is what we have so far.

But there is still a need for at least one additional tool to fully utilize the mySQL server. The basic functions of creating and deleting databases, creating tables and stored procedure, and importing and exporting data and SQL require an admin tool.

And while there are some third-party tools available to externally support database administration (my favorite is mySQL Workbench), there is still a need for an embedded tool accessible on the server computer.

So here we go the installation of one of the most common mySQL administration tools on the USB LAMP server, phpMyAdmin.

And, just like with my last post, for those who do not want to go through all the setup steps, a Linux session save file is included with this post.

Simply replace your existing save file with this one and you will be 99% complete with a web server that supports both php and mySQL server. Here it is. You can also refer to my Quick and Easy section from the previous post on this series for minimal installation steps.

But if you would like to understand the steps to installing phpMyAdmin on the USB Precision Puppy Linux setup, read on.

Adding phpMyAdmin to the USB Web Server

Start the USB installed Precision Puppy Linux from part 3. Then launch the Puppy Package Manager. In the search window (Find), enter “phpMyAdmin” and search all repositories.

Select phpmyadmin_3.4.10.1-1.

Then click “Examine dependencies”   and “Download and install selected packages”.

And as with other packages, there are a few post-installation steps required. After the installation has completed, open a console windows and enter:

# ln -s /etc/phpmyadmin/apache.conf /etc/apache2/conf.d/phpmyadmin
# ln -s /etc/apache2/mods-available/mime* /etc/apache2/mods-enabled/

Now, to enable initial access to phpMyAdmin, passwordless logins need to be enabled. This is easy. Just open the file:

/etc/phpmyadmin/config.inc.php

Un-comment the line:

//$cfg[‘Servers’][$i][‘AllowNoPawwords’] = TRUE;

Testing phpMyAdmin

In order to access phpMyAdmin, the mySQL server must be running. And to start mySQL, you must know the IP of the computer hosting the server. This should already be known from installing parts 1-3. This is the same IP that should be in the host file.

A quick refresher…

Identify the IP :

From a console, enter:

# /sbin/ifconfig eth0

The computer will respond with several lines. The second line should contain the ip. My second line began with:

inet addr:192.168.0.11

The host file: /etc/host

Now lets start the mySQL and apache servers and test the phpMyAdmin installation:

Start the mySQL Server:

From a console, enter:

# mysqld –bind-address 192.168.0.11

Start Apache Server:

From a second console, enter:

# /etc/init.d/apache2 restart

Now go to another computer on your network and enter the following URL from a browser to start phpMyAdmin:

http://192.168.0.11:9777/phpmyadmin

The following initial screen should appear:

phpmyadmin

Use the Username “root” with no password and enter “Go” to begin the phpMyAdmin session.

The opening screen will look similar to this:

phpmyadmin-opened

Configuring phpMyAdmin

If you wish to add more users with password logins, click on the “Privileges” tab at the top of the phpMyAdmin screen. You will see a link to “Add a new User”.

phpmyadmin-adduser

After creating a new user, remember to set the user privileges (Use “edit Privileges” link) so that the user can gain access to selected databases and functions.

There is plenty of information available on-line on how to configure the phpMyAdmin utility. Just look for it.

phpMyAdmin Limitations

Note that this version of phpMyAdmin is 3.4.10. After using version 4.0.7 for a significant period of time, this older version lacks some of the features I had become used to being available. In fact, the only purpose to use this version of phpMyAdmin is to add and delete users, create databases, and import tables and procedures.

Essential functions, but you will need more…

 Still, I have found that even though the version of phpMyAdmin that is available with Precision Puppy is somewhat dated, it is sufficient for basic database administration. However, it is recommended once your database and users are initially setup with phpMyAdmin, subsequent table generation and stored procedure development use a third party tool. As noted earlier in this post, my favorite is the open-source MySQL Workbench.

In Closing

This series should provide  a great reference for anyone setting up a web server, either now or in the future. The server provides an excellent platform for several uses, including system development and test, learning about server configuration and capabilities, and even for deployment of a permanent installation.

There is just one more addition planned for the USB LAMP server…

The addition of the popular WordPress content management system (CMS). Stay tuned for that post coming soon.

 

Share This:
Facebooktwittergoogle_plusredditpinterestlinkedintumblrFacebooktwittergoogle_plusredditpinterestlinkedintumblr

USB LAMP Web Server Part 3 – Adding mySQL Server

phpmysql2

A web server is certainly not complete without a database server. Currently, the one-source server of choice is mySQL. At first, adding this database server to my Linux setup update appeared to be simple and straight-forward. Just add another package to the Puppy Linux installation and restart – instant mySQL server capability. But then I soon found that it was not going to be that simple.

Still, after persistent effort to overcome several patience-trying obstacles, the installation was finally complete. Presented here are the successful steps to adding mySQL Server, without having do deal with the problems. And for those who do not want to go through all the setup steps, a Linux session save file is included with this post.

Simply replace your existing save file with this one and you will be 99% complete with a web server that supports both php and mySQL server. Here it is. You can also refer to the Quick and Easy section of this post minimal installation steps.

But if you would like to understand the steps to installing  mySQL on the USB Puppy Linux setup, read on. Now let’s add the mySQL server to the USB LAMP web server installation from part 2 of this series.

Adding mySQL Server to the USB Web Server

IMPORTANT: The installation of mySQL Server will corrupt the save file. This will make it impossible to shut-down or start the USB Puppy Linux. Prior to proceeding, copy the save (*.sfs) file, located in the root USB directory (when viewed from Windows OS). It will be needed later to restore the Linux installation.

Start the USB installed Precision Puppy Linux from part 2. Then launch the Puppy Package Manager. In the search window (Find), enter “mysql”.

mysql_svr1

From the list of results, select mysql-server-5.5-47 metapackage.

Then click on “Examine dependencies”.

When a window pops up, click on the “Download and install selected packages” button.

Then click on “Download packages”.

After the installation finishes, the following prompt appears:

mysql_svr5

In order to add  the missing library, first close and then re-opening the Puppy Package Manager.

This time, enter “libatm” in the find input. You will probably only find one results from searching all the repositories, “libatm1_2.5.1-1.3”.

Select it and install the package as done for the mySQL server package.

Backing up the installed packages

Now is the time to create a backup of the installed programs. Do not exit Puppy Linux until this is completed.

First, install a blank USB memory stick of at least 4MB into the computer that is running Puppy Linux. This drive will appear as an added device in the bottom left of the Linux screen. Take note of the name assigned to this drive (it will be something like sdb1 or sdc1).

From a console window, enter:

cd /

mksquashfs etc root usr /mnt/<drive name from above>/mybackup.sfs 

Wait for the SFS file creation to complete.

Now remove the added thumb drive and then hold down the PC start button until the PC shuts off.

Restoring the backed up configuration

These steps need to be performed from a PC running Windows OS.

Using the same USB memory stick we have been using to prepare this setup, first copy the save file (see red text above) to the root directory.

Then copy the “mybackup.sfs” file created above to the USB memory stick root directory.

Now lets boot from the USB memory stick, which should start in the Puppy Linux OS again.

NOTE: Make sure the PC you are using for this step has at least 4 GB of ram installed. I tried it with a 1GB netbook which failed the process after running out of RAM.

Open two file browser windows (Menu->Filesystem->ROX-Filer).

In one window, navigate to /etc/home/mybackup.sfs. After clicking on the file, select “view”. A folder with the 3 backed up directories (etc root usr) will appear.

In the other file manager windows, navigate to the top-level folder. Among the other directories, this window will also include etc, root, and usr. Copy the files from the first window to the second window.

This can be done by pressing Ctl-A in a blank space in the first window (Selects the 3 folders) and dragging these selected folders to the second window.

I found that a prompt to confirm the over-write of each file was required and could not figure out how to suppress it. Since there are thousands of files to copy, this requires many mouse clicks or pressing the enter key. As a work-around to automate this step, I used an object that was heavy enough to hold the enter key down. A bottle of essential oil worked well for my enter key.

Use your imagination here!

20160321_151721

And here is where patience is needed. This copy process took over an hour to complete.

Remember, a save file (*.sfs) is provided with this post which already includes these steps, so you don’t have to repeat this lengthy but necessary step.

Once the copy is completed, restart the Linux computer. This should created a clean save file.

Alas, we are ready to resume the mySQL installation!

Console Entries

A few steps are now needed to complete the installation of mySQL. Open a console window and enter the following:

# mkdir /var/lib/mysql

# adduser mysql

# chown -R mysql:mysql /var/lib/mysql

# chgrp -R mysql /var/lib/mysql

#mysql_install_db

# mkdir /var/run/mysqld

# chown mysql:mysql /var/run/mysqld

That’s it. The installation of mySQL Server is complete.

mySQL Server commands

Here are a few console commands to run the server:

Start Server:

# mysqld

Stop Server (Do this from a second console window):

# mysqladmin -u root -p shutdown

Change rood password:

# mysqldadmin -u root password ‘new-password’

or

# /usr/bin/mysqldadmin -u root password ‘new-password’

Here is a reference of other commonly used mySQL server commands.

Quick and Easy Setup

And as promised, I am providing a system image of the full USB LAMP installation in this section, so you can skip most of the steps outlined above and in the first two posts of this series.

Just follow these simple steps, eliminating most of the time-consuming setup:

Step 1: Create a Linux USB Memory stick installation per the first part of this post. You can stop prior to installing Apache Server. That package is included with the *.2fs file referenced below.

Step 2: Shut-down the Linux OS running from the USB stick. This will create a save file.

Step 3: Boot a PC to Windows. Install the USB memory stick and delete the save (*.2fs) file from the root directory. Then copy the *.2fs file from this location to the root directory.

NOTE: The file is contained in a ZIP compressed file. The compressed file is over 280MB in size and 1GB uncompressed. Expect the download to require some time (17 minutes with my connection).

Step 4: Boot a PC with the USB Stick installed to Puppy Linux. This stick now has a complete LAMP stack. But you may need to adjust the IP and port of the server to your specific system. The defaults are IP=192.168.0.110 and port=9777.

Step 5: Set the IP for your system

Step 6: Set the listen port for your system

Step 7: Reboot one more time to create your own save file.

Be sure and review the 3 posts in this series as a guide to using this LAMP stack. A sample HTML and PHP file is provided in the root folder for web pages, /var/www

In Conclusion

In these 3 articles, we have created a USB memory stick Linux environment and added an Apache Web Server, php and mySQL client libraries, and now the popular mySQL Server. Next, we will add phpmyAdmin as a basic tool for administering the mySQL database.

As always, I hope you find this information useful…

 

Share This:
Facebooktwittergoogle_plusredditpinterestlinkedintumblrFacebooktwittergoogle_plusredditpinterestlinkedintumblr

Triple Server Update – Part 3: More Server Features

server_update_part3

More Server Features

Ever heard the saying “some’s good, but more is better” ? That idiom certainly rings true when it comes to the capabilities of the multi-purpose server I began developing recently.

The Triple Server introduced with this series initially supported http, mqtt and coap. Subsequently, coap was dropped and replaced with the addition of an Arduino interface. Web configuration was also introduced.

But even then, the system lacked a few highly desirable features. This update builds upon the basic  structure developed so far. Here are the new capabilities introduced with this update:

  • Configurable FOTA port and password
  • Configurable MQTT connection port and client id
  • URL decoding for configurable parameters with special characters
  • FOTA – sketch embedded wireless firmware update support
  • MQTT topics linked to ChipID(MAC)
  • Support for MQTT connection password enabled
  • Unsolicited Arduino MQTT Publications
  • Current GMT added to http header

There features are individually portable. When needed, they can be a great addition to many different IoT projects.

And in case you have not seen the prior posts in this series, here is the story from the beginning:

ESP8266 Triple Protocol Server

Triple Server Update – Part 1: Serving Arduino

Triple Server Update – Part 2: Web Configuration

Triple Server Update – Part 3: More Server Features

Let’s get to the details of the implementation…

Web Configuration

Starting with the Web Configuration panel:

WebConfig
FOTA password and network port settings are added to the configuration page options. When a FOTA password is specified, the user will be prompted to enter it prior to proceeding with a firmware update.

A setting for the MQTT client id has also been included with this update. The past versions of this project simply set the client id to the MQTT username. Finally, a check-box has been added to enable MQTT password connections.

However, the remaining MQTT connection options specified in MQTT Version 3.1.1 such as a last will message have not been implemented in this project. There additional options were not considered essential for this application.

URL decoding

The parameters updated on the Web Configuration page are passed from the html web code back the ESP8266 sketch code using an http “GET” request. Each parameter is added to the URL, a method that can easily be parsed by the ESP8266 c-code using the functions imported from the EspressIf SDK.

This all worked well until special characters were introduced in a password string. In these cases, the special characters are URL encoded, which converts the special characters into something an URL will accept. For example, a space is considered a special character and gets converted to “%20” in an URL.

The problem occurs when the parameter is saved. If “Hello World” is sent as an URL parameters, it becomes “Hello%20World”. This situation is resolved by using an URL decoder prior to saving the value. This restores the parameter to it’s original value, “Hello World” in this example.

Here’s the code used for URL decoding. It should be fairly easy to follow. The basic algorithm looks for a “%hh” code and converts it back to the original “special character” value in the string:

 
 
  1. /********************************************************
  2.  *  URL Message Decoder
  3.  ********************************************************/
  4.  
  5. int url_decode(char *encoded_str, char *decoded_str) {
  6.    
  7.     // While we're not at the end of the string (current character not NULL)
  8.     while (*encoded_str) {
  9.         // Check to see if the current character is a %
  10.         if (*encoded_str == '%') {
  11.     
  12.             // Grab the next two characters and move encoded_str forwards
  13.             encoded_str++;
  14.             char high = *encoded_str;
  15.             encoded_str++;
  16.             char low = *encoded_str;
  17.     
  18.             // Convert ASCII 0-9A-F to a value 0-15
  19.             if (high &gt; 0x39) high -= 7;
  20.             high &amp;= 0x0f;
  21.     
  22.             // Same again for the low byte:
  23.             if (low &gt; 0x39) low -= 7;
  24.             low &amp;= 0x0f;
  25.     
  26.             // Combine the two into a single byte and store in decoded_str:
  27.             *decoded_str = (high &lt;&lt; 4) | low;
  28.         } else {
  29.             // All other characters copy verbatim
  30.             *decoded_str = *encoded_str;
  31.         }
  32.     
  33.         // Move both pointers to the next character:
  34.         encoded_str++;
  35.         decoded_str++;
  36.     }
  37.     // Terminate the new string with a NULL character to trim it off
  38.     *decoded_str = 0;
  39. }

FOTA Support

Firmware over-the-air (FOTA) is a very useful option to include with your sketch. The Arduino makes this capability very clean and easy to embed in your code. I found it very useful for debugging code, as it freed up the serial port for test messages. The drawback, however, is that FOTA reduces your effective maximum sketch size in half. This was not a problem in this project, which currently only uses 26% of the available 4MB of my ESP8266-12 flash chip. Just be aware if you are using an older ESP8266-1 with this project, FOTA cannot be used as the sketch uses more than 50% of the 512K flash installed on many of the older ESP8266 versions.

Here is the implementation in this sketch:

A function was added and called in the setup() function.

init_FOTA();

 
 
  1. void init_FOTA() {
  2.     String buff;
  3.     int pt;
  4.     //Hostname defaults to esp8266-[ChipID] (no change to default, which is unique (ChipID = last 3 MAC HEX)
  5.     //ArduinoOTA.setHostname("myesp8266");
  6.     
  7.     //Set FOTA Network Port
  8.     GetEepromVal(&amp;buff, EEPROM_MQTT_PT, EEPROM_INT16);
  9.     pt = atoi(buff.c_str());
  10.     ArduinoOTA.setPort(pt);
  11.      
  12.     //Set OTA authentication (password)
  13.     GetEepromVal(&amp;buff, EEPROM_FOTA_PW, EEPROM_CHR);
  14.     ArduinoOTA.setPassword((char *)buff.c_str());
  15.     
  16.     ArduinoOTA.onStart([]() {
  17.         Serial.println("Start");
  18.     });
  19.     ArduinoOTA.onEnd([]() {
  20.         Serial.println("\nEnd");
  21.     });
  22.     ArduinoOTA.onProgress([](unsigned int progress, unsigned int total) {
  23.         Serial.printf("Progress: %u%%\r", (progress / (total / 100)));
  24.     });
  25.     ArduinoOTA.onError([](ota_error_t error) {
  26.         Serial.printf("Error[%u]: ", error);
  27.         if (error == OTA_AUTH_ERROR) Serial.println("Auth Failed");
  28.         else if (error == OTA_BEGIN_ERROR) Serial.println("Begin Failed");
  29.         else if (error == OTA_CONNECT_ERROR) Serial.println("Connect Failed");
  30.         else if (error == OTA_RECEIVE_ERROR) Serial.println("Receive Failed");
  31.         else if (error == OTA_END_ERROR) Serial.println("End Failed");
  32.     });
  33.     ArduinoOTA.begin();
  34.     Serial.print("FOTA Initialized using IP address: ");
  35.     Serial.println(WiFi.localIP());
  36. }

Note that the FOTA port and password values are set from values stored in EEPROM. That means they can be set from the application, so the sketch does not require modification to change the values.

More on that when we go through the updates to the Web Configuration page.

Unique MQTT topics

It a system is built using multiple devices, it may be desirable to ensure that each device sends and receives MQTT message over unique topic names. In order to link the MQTT topic to the ESP8266 device, the default topic now includes an expanded ChipID.

That is to say, while the official ChipID uses the last 3 hex characters of the device’s MAC address, the expanded id in this application uses all six MAC address hex values.

The implementation here simply appends the MAC to a topic prefix. The result is a topic name unique to each ESP8266 device.

 
 
  1.  void AddMAC(char * prefix, char * topic) {
  2.      uint8_t MAC_array[6];
  3.      WiFi.macAddress(MAC_array);
  4.      sprintf(topic,"%s", prefix);
  5.      for (int i = 0; i &lt; sizeof(MAC_array); ++i){
  6.           sprintf(topic,"%s%02x",topic, MAC_array[i]);
  7.      }
  8.  }

MQTT Password Connection

The initial design of the MQTT server in this project relied solely on the test.mosquitto.org broker. This simple test site does not offer the option of connecting with passwords. But, of course, a password protected connection is preferred. And recently, I had set up my own MQTT broker, with the capability to require username/password credentials with connections. So adding this to the ESP8266 based server was obviously needed.

Here is the mqtt connection code used to establish password enabled and password-free connections.  Note that the MQTT client class object uses a different client.connect() prototype when using a password to connect.:

 
 
  1. void MqttServer_reconnect(void) {
  2.     int fst=10;
  3.     bool connected = false;
  4.     // Loop until we're reconnected (give up after 10 tries)
  5.     while (!client.connected()&amp;&amp;(fst!=0)) {
  6.         Serial.print("Attempting MQTT connection...");
  7.         // Connect to MQTT Server
  8.         if(mqtt_pw_enable) {
  9.             connected = client.connect(mqtt_ci.c_str(),mqtt_un.c_str(),mqtt_pw.c_str()); 
  10.         }
  11.         else {
  12.             connected = client.connect(mqtt_ci.c_str()); 
  13.         }
  14.         if (connected) {
  15.             // Successful connection message &amp; subscribe
  16.             Serial.println("connected");
  17.             client.subscribe((char *)mqtt_rt.c_str());
  18.             fst--;
  19.         } else {
  20.             // Failed to connect message
  21.             Serial.print("failed, rc=");
  22.             Serial.print(client.state());
  23.             Serial.println(" try again in 5 seconds");
  24.             // Wait 5 seconds before retrying
  25.             delay(5000);
  26.         }
  27.     }
  28. }

Unsolicited Arduino MQTT Publications

The Arduino server has been designed to query the Arduino for sensor or pin status. These queries are initiated from external http or MQTT requests. But this structure did not allow the Arduino to independently publish messages to an MQTT topic. That capability has now been added. MQTT messages originating in the Arduino are now possible.

Adding this capability required a simple restructure of the serial part data handler. Now, any time the end of line is received from the serial port, the ESP8266 sends the response to the http or mqtt channels when initiated, and to the mqtt topic when received without an initial request originating from the ESP8266.

This change impacted the function that initiates requests to the Arduino via the serial port. With the revised implementation, the function no longer waits for a reply:

 
 
  1. String ArduinoSendReceive(String req) {
  2.     String fromArduino = "";
  3.     Serial.println(req);
  4.     long start = millis();
  5.     fromArduino = "";
  6.     return fromArduino;
  7. }

Instead, a new function, “MonitorSerialLine()”, was added to process all data received on the serial port, both initiated and unsolicited.

 
 
  1. void MonitorSerialLine() {
  2.     if(arduino_server) {
  3.         while (Serial.available()) {
  4.             // get the new byte:
  5.             char inChar = (char)Serial.read();
  6.             // add it to the inputString:
  7.             SerialInString += inChar;
  8.             // if the incoming character is a newline, set a flag
  9.             // so the following code can process it
  10.             if (inChar == '\n') {
  11.                 NewSerialLineRx = true;
  12.             }
  13.         }
  14.         //Send Received String as MQTT Message
  15.         if(NewSerialLineRx) {
  16.             //If not server request, just forward Arduino Message to MQTT
  17.             if(active_svr_rqst == SVR_NONE) {
  18.                 active_svr_rqst = SVR_MQTT;  
  19.             }
  20.             Server_SendReply(active_svr_rqst, REPLY_TEXT, SerialInString);
  21.             active_svr_rqst = SVR_NONE;
  22.             SerialInString="";
  23.             NewSerialLineRx=false;
  24.         }
  25.         //Check for timeout for Arduino Server requests
  26.          if(active_svr_rqst != SVR_NONE) {
  27.              if(start_wait==0) start_wait = millis();
  28.              if( (millis() - start_wait)  &gt;5000 ) { //5 sec timeout
  29.                   SerialInString = "no arduino reply received";
  30.                   Server_SendReply(active_svr_rqst, REPLY_TEXT, SerialInString);
  31.                   //Reset Request Parameters
  32.                   active_svr_rqst = SVR_NONE;
  33.                   start_wait=0;
  34.                   SerialInString="";
  35.                   NewSerialLineRx=false;
  36.              }
  37.          }
  38.     }
  39. }

Current GMT

gmt

The http response header requires an identification of current GMT (for the Date header) and a current date offeset (for expiration headers). Prior versions of this project simply added hard-coded, static values for these headers. While it did no impact the http response transmission, it was obviously inaccurate header content.

In order to create accurate date headers, current GMT must be known. So time is now retrieved from a NIST server. This required an initialization routine to be run in the sketch’s setup() function. The offset is set to 0 so we get GMT when time is requested.

 
 
  1. void init_GmtTime() {
  2.     configTime(0, 0, "pool.ntp.org", "time.nist.gov");
  3.     Serial.print("\nWaiting for time");
  4.     while (!time(nullptr)) {
  5.         Serial.print(".");
  6.         delay(1000);
  7.     }
  8.     Serial.print("\r\nTime has been acquired from internet time service\r\nCurrent GMT: ");
  9.     time_t now = time(nullptr);
  10.     Serial.println(ctime(&amp;now));
  11. }

Here is how time is now retrieved when creating an http header:

 
 
  1. String gmt,expire;
  2. time_t now = time(nullptr);
  3. gmt = ctime(&amp;now);
  4. now += 3600 * 24;  //Expires in 1 day
  5. expire = ctime(&amp;now);
  6. gmt = gmt.substring(0,3) + "," + gmt.substring(7,10) + gmt.substring(3,7) + gmt.substring(19,24) + 
  7. gmt.substring(10,19) + " GMT";
  8. expire = expire.substring(0,3) + "," + expire.substring(7,10) + expire.substring(3,7) + 
  9. expire.substring(19,24) + expire.substring(10,19) + " GMT";

The string returned from a call to ctime() is in a different format than what is needed in the http. The String class “substring” method is uses to re-sort the ctime() string into the needed http header format.

ctime() format:

Mon Mar 14 08:23:14 2016

http header format:

Mon,13 Mar 2016 08:23:14 GMT

In Closing

This update establishes a solid framework for building IoT projects with both http and MQTT server support. While Arduino is used in the example, any external device with a standard serial link can also be easily added to this structure. And when the system is deployed, updates can be deployed wirelessly, eliminating the need to connect a physical serial interface to the system.

Here is the updated GitHub repository for this project.

I hope you find this information useful…

Share This:
Facebooktwittergoogle_plusredditpinterestlinkedintumblrFacebooktwittergoogle_plusredditpinterestlinkedintumblr

USB LAMP Web Server

servericon

Your Personal http Server

If you would  like to have your own personal web server, running industry standard software, all from a boot-able USB stick, read on…

One of the most useful tools for home automation and web development is a host server. And the most widely used server is of course, Apache.

This article marks the start of a 5-post series that provides step-by-step instructions for setting up your own USB memory stick based http server. We’ll start with the basic LAMP stack in the first three articles, and then add-on some extras to extend the functionality.

What’s LAMP?

LAMP covers the essential ingredients of a web server:

  1. Linux – The operating system
  2. Apache – http server
  3. mySQL – database
  4. PHP – server-side scripting language

Here are the topics planned for this series:

Update: Bonus Topics

Part 1 – Setting up the Linux Apache Server

When finished with part one, you will have the LA (Linux and Apache) LAMP components up and running.

Let’s get to it…

Prerequisites

  • Blank USB Memory stick (4GB minimum)
  • PC with USB port that can be dedicated for the server

While I cannot claim to be a Linux expert, I do have some experience working with the single user Puppy Linux distribution. Being a familiar setting, that is what was selected for this exercise.

But hold on, before you start there is one more selection to make….

You see, there are several versions of Puppy Linux to pick from. My first attempt at this used the most current version, called Slacko Puppy 6.3. However….sparing you the details, I ran into some insurmountable obstacles to completing a successful installation of Apache server with Slacko Puppy.

So I moved on to a Puppy Linux version that works with the Apache Server.

The version that worked is called Precise Puppy 5.7.1

Installing the Linux Operating System

As with many Linux distributions, the installation is really quite simple. Just download the ISO image and a USB installer. That’s it. Here is where you can get both:

Now for the installation. Just install your USB stick and start the installer program. The start up screen will provide 3 easy-to-follow instruction steps.

  1. Select the distribution (Select “Precise Puppy”)
  2. Select the iso file (from the folder that has the downloaded ISO image)
  3. Select the USB stick drive letter.

UUSBinstaller

Then click on the “Create” button to start the installation.

Once the installer is finished, you can boot any PC to the newly installed Puppy Linux simply by putting this memory stick in the PC’s USB port and rebooting (Boot to USB first must be selected in the bios first, of course).

After the computer has booted to Linux and you have completed the self-guided first boot process, you need to perform one additional step before installing Apache.

NOTE: It is recommended that a wired internet connection be used for this and all processes during installation.

First, you need to update the package database. This is a very simply process. Just launch the Puppy Package Manager (Menu->Setup->Puppy Package Manager) and click on the “Configure package manager”. A window will appear with an “Update now” button.

updatenow

Click that button and press enter each time the yellow window prompts you. There will quite a few “Enter”s required to complete this step.

Once complete, exit the Puppy Package Manager. If this is still your first Linux session using the memory stick, it is also recommended that you restart the computer (from the bottom MENU option). The restart will create a save file that will be updated every time you exit Linux. This file contains all the changes you have made to the originally installed Linux distribution.

Adding Apache Server

Now let’s get Apache.

Launch the Puppy Package Manager again. In the search window (Find), enter “apache2”.

findapache

Select “apache2_2.2.22” from the listed results.

selectapache

A pop-up windows will appear as shown below:

examinedepenancies

Click the “Examine dependencies” button.

Click “Download-and-install selected packages” in the window below.

downloadinstall

Another window will appear. Click “Download packages” in this window.

downloadnow

Manual Apache Server Installation Steps

Once the download is complete, we are ready to make the necessary adjustments in order for the server to properly start. We will be adding a new user and add links to the files and directories the Apache server is expecting.

Open a terminal by clicking on the desktop icon “console”.

In the console window, enter the following lines. End each line with the <Enter> key. The entries are shown in red text:

# adduser www-data
adduser: /home/wwwdata: No such file or directory
Changing password for wwwdata
New password:
Retype password:
Password for wwwdata changed by root

Note that <enter> is pressed with no entry for the password.

Now lets continue with the next lines:

# touch /etc/apache2/httpd.conf

# ln -s /etc/apache2/mods-available/auth* /etc/apache2/mods-enabled/

# ln -s /etc/apache2/sites-available/default /etc/apache2/sites-enabled/

# ln -s /etc/apache2/mods-available/alias* /etc/apache2/mods-enabled/

# ln -s /etc/apache2/mods-available/ldap* /etc/apache2/mods-enabled/

# chown www-data:www-data /var/www

# touch index.html

# chown www-data:www-data /var/www/index.html

# ln -s /etc/apache2/mods-available/dir.* /etc/apache2/mods-enabled/

Server Test Page

Lets put the typical “hello world” message in our index.html file so we know it is served properly.

The file is in the /var/www directory, which is the apache server root folder. Open that file and fill it with the following text:

 
 
  1. &lt;html&gt;
  2.     &lt;body&gt;
  3.         &lt;h1&gt;Hello World!&lt;/h1&gt;
  4.     &lt;/body&gt;
  5. &lt;/html&gt;

Now there are only two more things needed to complete the server installation:

1. Set the IP
2. Set the Listening Port

Setting the IP

From the terminal, enter (a wired network cable should be installed):

# /sbin/ifconfig eth0

The computer will respond with several lines. The second line should contain the ip. My second line began with:

inet addr:192.168.0.110

This IP value needs to be added to the hosts file. It is at /etc/hosts.

The first line should be:

127.0.0.0 localhost <your computer id>

Change it to (using your IP value):

192.168.0.110 localhost <your computer id>

You can now save and close the hosts file.

Setting the Listen Port

Now lets finish this off by setting the server listening port. If you want to leave the server listening on the default port (80), you can skip this step.

Open the file: /etc/apache2/ports/ports.conf

Near the top of the file, you will see the following 2 lines:

NameVirtualHost  *:80

Listen 80

Change “80” in these lines to whatever port you want the server to listen on.

Save and close this file.

Now open the file: /etc/apache2/sites-enabled/default

The top line should be:

<VirtualHost *:80>

Change “80” in this line to the same port number in the first file.

Save and close this file.

Testing the Apache Server

First, start the server by entering the following in a console window:

/etc/init.d/apache2 restart

The console should respond with:

startapache

Now, using a web browser from a different computer on your network, enter (replacing 80 with the port number configured in the previous step):

192.168.0.110:80

The browser response should be (my listen port was 9777):

HelloWorld

Congratulations! Your Apache Server is now functional.

This is an enormous first step. You can now serve html files from this server, both within your local network and globally from anywhere on the planet!

In Closing

This is not the end, it is only the beginning of your own host server. I will share my setup as the components are added. Part 2, adding PHP to the server will be coming soon.

I hope you find this information useful…

Share This:
Facebooktwittergoogle_plusredditpinterestlinkedintumblrFacebooktwittergoogle_plusredditpinterestlinkedintumblr

Your Own MQTT Broker

mqtt

Like many folks, I too started out using the public MQTT broker at test.mosquitto.org. It’s a great way to get started – simple, easy to get working, and FREE! But it does not take long to realize it is unsecured. Anyone can listen in on your topics and there are no logon credentials required or offered as an option.

So I got to searching for a better broker…

One with security. And all the options available with the MQTT standard. Things like:

  • Security Authentication (passwords,certificates)
  • Simultaneous websockets and mqtt (tcp) listeners
  • Persistent Messages

But I did not want to pay for the service. The obvious solution was to host your own broker, either on a host server, or on your local network with broadband access via a router.

Since my host server does not permit continuously running scripts or programs, was limited to a local network solution. But with a broadband connection, it would be on-line and accessible anywhere.

After some research, the most promising options included:

  • PC Broker with Windows OS
  • PC Broker with Linux OS
  • Flash Driver Linux Distro Broker
  • Raspberry Pi2 Broker
  • Android Device Broker
  • Embedded micro-controller

I have read numerous comments about poor (slow) performance using a Raspberry Pi, and since I do not own one, that option was ruled out. For the same reason, I thought about hosting a broker on the trusty ESP8266 but decided against it, at least for now.

And while it would be great to use an old Android phone as an MTTQ broker, the path to get there was a bit murkier than using a Linux hosted server.  It can be done, but few have gone this path. That is, there is little in the way of guidance so this would require significant development.

Windows? Maybe with Windows Server OS running. But that’s not what I got. No.

So looking around at my inventory, I decided to use an unused Window 7 netbook. But, rather than overwriting the hard-drive, a USB flash drive installation was done.

Linux running an MQTT broker when booting to the flash drive.

Windows 7 with the flash drive removed.

While there were a few challenges along the way, it turned out to be a great solution. It has been running continuously now for over a week – flawlessly.

Here is how to set it up…

Linux Installation

Looking for a small, clean Linux distribution, I selected Puppy Linux.  The choice was easy to make, since it had already been setup and running. This post provides step-by-step instructions to configure your flash drive. Follow all the instructions as you will need the development environment to build the MQTT application.

Building the MQTT Broker Application

First thing needed is a copy of the Mosquitto 1.4.7 broker. You can get it here. Then, with the flash drive in a Windows PC, copy the unzipped contents of the folder org.eclipse.mosquitto-1.4.7 to the flash drive in a new folder in the path:

<flash drive>/MyPrograms/mosquitto

You can now install the flash drive in your target computer and reboot. It should start in Puppy Linux.

puppy-start

Before we can build the application, a couple of steps are needed.

  • Install Mosquitto Package
  • Install libwebsockets
  • Edit build configuration file

Let’s go…

Installing the Mosquitto Package

  1. From the Desktop, click on the blue “install” icon.
  2. Click on the “Install Applications” tab.
  3. Click on the Puppy Package Manager icon.
  4. Enter “mqtt” into the search box and hit the “Enter” key.
  5. Click on the mosquitto_0.15 Package.
  6. Click “Install” (Upper right of windows).

Installing the libwebsockets Library

  1. From the Desktop, click on the blue “install” icon.
  2. Click on the “Install Applications” tab.
  3. Click on the Puppy Package Manager icon.
  4. Enter “libwebsockets” into the search box and hit the “Enter” key.
  5. Click on the libwebsockets3_1.2.2.1 Package.
  6. Click on the libwebsockets-dev_1.2.2.1 Package.
  7. Click “Install” (Upper right of windows).

Edit build configuration file

  1. From the Desktop, click on the green “edit” icon.
  2. Click Open, then under “Places”, click “File System”.
  3. Under “Name”, click “mnt”. Then click open.
  4. Under “Name”, click “home”. Then click open.
  5. Under “Name”, click “MyPrograms”. Then click open.
  6. Under “Name”, click “mosquitto”. Then click open.
  7. Under “Name”, click “config.mk”. Then click open.
  8. Scroll down to “WITH_WEBSOCKETS:=no and change to “yes”
  9. Save the file and exit.

Building the broker application

We are now ready to build the application. This is really simple. First, open the console window by clicking on the “console” icon from the desktop. Now switch to the directory that contains the mosquitto source code by entering:

cd /mnt/home/myprograms/mosquitto/src

now build the application by entering:

make

Configuring the MQTT Broker Application

Edit mosquitto run-time configuration file

  1. From the Desktop, click on the green “edit” icon.
  2. Click Open, then under “Places”, click “File System”.
  3. Under “Name”, click “mnt”. Then click open.
  4. Under “Name”, click “home”. Then click open.
  5. Under “Name”, click “MyPrograms”. Then click open.
  6. Under “Name”, click “mosquitto”. Then click open.
  7. Under “Name”, click “mosquitto.conf”. Then click open.
  8. From the edit menu, select”Save as” and save this file to the src folder. The full file path should now be /mnt/home/myprograms/mosquitto/src/mosquitto.conf.
  9. Scroll down or search for “#allow_anonymous true”. Change this to “allow_anonymous false”. Remember to delete the # so this is not commented out. This will force the broker to require usernames and passwords to connect.
  10. Scroll down or search for “#user mosquitto”. Change this to “user nobody”. Remember to delete the # so this is not commented out.Puppy linux does not have a user named “mosquitto” but it does have one named “nobody”. Since Puppy Linux is a single user distribution, it does not allow you to add users.
  11. Scroll down or search for “#password_file”. Change this to “password_file pw.txt”. Remember to delete the # so this is not commented out.
  12. Scroll down or search for the text “#protocol mqtt”. Just after this line, add the following 4 new lines:
    1. listener 11883
    2. protocol mqtt
    3. listener 18080
    4. protocol websockets
  13. Save and exit the file

What step 9 does is configure the broker with two listeners, one with standard mqtt (tcp) protocol and one with websockets.

While any port can be used, a one was added to the standard port numbers so our broker is not in conflict with the “well-known” mqtt ports. This could be important in the complicated event that your client is connected to two different brokers at the same time. In this case, the ports can only be open once. This eliminates potential conflicts.

Creating a password file

Using the file editor (edit icon from the desktop), save a blank file in the src folder:

/mnt/home/myprograms/mosquitto/src/pw.txt

Add a few username/password entries in this file in the format, for example:

username:password
user2:password2

Save the file. Also make a copy of this file for off-line storage.

Now run the password utility. This will change the plain text passwords in the file pw.txt to a hashed value. Run the password utility from the console:

cd /mnt/home/myprograms/mosquitto/src
./mosquitto_passwd -U pw.txt

If you open pw.txt, you will find the plain text passwords have been replaced with a hashed value.

Opening the Linux Firewall

network-firewall-icon

Are we ready to run the broker yet? Almost. But there is one more thing needed in order to access the broker from another device. We need to configure Linux to allow external connections.

Open the file /etc/hosts.allow

It should contain one line:

ALL:LOCAL

Change this to:

ALL:ALL

Save and exit the file.

If you want a more restrictive environment, it is suggested that you research configuration settings for the Linux hosts.allow file. For the purpose of this example, we are opening the MQTT Broker to anyone with proper username/password credentials.

Starting the Broker

Now to start the broker, just go to the src folder and enter the following:

cd /mnt/home/myprograms/mosquitto/src
./mosquitto -c mosquitto.conf

The startup console should display:

start_mosq

The warning occurs because ipv6 is not supported. But this is of no concern for the typical ipv4 addresses. While the application was initially build in the ../src directory, it can be moved and executed from any location of your choosing. just remember to also include the password file utility, the config file and the password file. These are the files needed to run the application:

mosquitto
mosquitto_passwd
mosquitto.conf
pw.txt

Testing the Broker

You probably have your own devices to connect to and test the broker. With the configuration of this broker, you will need to include a username and password when connecting. I like to use the Google Chrome MQTTlens extension and the Android MyMQTT App when making quick MQTT connection checks. I’ve provided additional details for using these MQTT tools in this post.

Port Forwarding and DDNS 

icon-port-forwarding

While you can access this MQTT  broker from any device on your LAN subnet, world-wide broadband access will require configuring your router to forward MQTT port requests to the device running the broker. This is called port forwarding. Please refer to this post for more information about configuring port forwarding and DDNS.

In Closing

This broker setup provides you with all the features of the MQTT specification. One of the best features is the ability to operate a mixed protocol system. This provides an connection to devices using either mqtt (tcp) and websocket protocol together. And you can enable any level of security needed, from simple passwords to security certificates. And since it is running locally, you are in full control.

Next up. I am planning to update my MQTT App inventor template application to support the basic security credential provided by this broker.

I hope you find this information useful…

 

Share This:
Facebooktwittergoogle_plusredditpinterestlinkedintumblrFacebooktwittergoogle_plusredditpinterestlinkedintumblr

MQTT for App Inventor – Adding Configurable Settings

App Inventor to MQTT Communication

In my last post, a basic MQTT client portal for App Inventor was introduced. As result, there has been some interest expressed for TCP socket support. Note that the implementation presented relies on WebSockets. Due to security vulnerabilities, pure TCP sockets are simply not possible with this design. If TCP sockets are needed for MQTT communication at your IoT device, a broker capable of both WebSockets and TCP sockets should be used – Websockets on the App Inventor side, and TCP sockets on the IoT device end.

Adding Configurable Setting

My next step was to refine the App Inventor project to support configurable MQTT setting.

NOTE: If you just want to review and use the updated App Inventor project without the walk-through that follows, simply use the files provided in this Github repository.

The first challenge while adding this capability was the inability to grant file system access to the JavaScript code. Native Android Apps can overcome this with a change to the App’s manifest file. But App Inventor does not expose this application setting.

So the App Inventor file component had to be used to read and write to the file system. The configuration file settings are then passed on to the JavaScript to update the MQTT connection parameters.

mqtt_cfg

App File Location

This App now uses two files stored on the Android device. These files have been moved to a common location which should be the same on most Android devices:

File Description
/sdcard/mqtt/mqtt_appinventor.html The JavaScript MQTT App Inventor Server
/sdcard/mqtt/cfg.txt MQTT Configuration

Configuration file

For this example, 3 parameters are configurable:

  1. Broker: This can be any on-line broker, on your own network or either publicly or privately provided.
  2. Request Topic: MQTT topic used to send a client request
  3. Reply Topic: MQTT topic used to reply results to the client.

Since this example uses the publicly available broker ‘test.mosquitto.org’, username/password authentication is not provided. You could easily be add this to the example provided here, however, if you are using a broker requiring authentication.

The file is formatted as a JSON string. This makes it very easy for the JavaScript to parse the values. The first time the App is run, these configurable parameters are set to the values shown in this JSON string:

{
“mqtt_broker”:”ws://test.mosquitto.org:8080/mqtt”,
“mqtt_txtopic”:”MyMqttSvrRqst”,
“mqtt_rxtopic”:”MyMqttSvrRply”
}

App Inventor File Read/Write

When the screen used to edit the configurable parameters is open, it first reads the cfg.txt file, extracts the parameter setting, and uses those values to populate the screen fields.

But when using App Inventor, there are no (at least none that I am aware) libraries available for parsing JSON. So I created the procedure “GetParam”, which extracts the cfg.txt values with 3 calls, one for each parameter.

readfile

Two values are determined initially in order to extract the value. First is the position in the string that the value starts. The other value is the string length. These values are determined by using App Inventor’s string search (starts at) feature to find something unique (StartPiece and LenPiece) around the value wanted, and an offset from the parameter value’s exact position in the string.

getparam

Once extracted, the “Edit Configuration” screen is populated with the values from this file.

cfg_gui

Saving the values is just the opposite. The values from the screen are formatted into a JSON string and then saved back to the same file, cfg.txt.

save

When this screen is exited, the new configuration values are sent to the JavaScript code. The JavaScript uses these new values and closes, then opens the MQTT connection with the new values. Here is how that has been implemented…

Updated Main Screen

As I have gained knowledge coding with the App Inventor, it became obvious that meaningful names were needed for project components. Much to my surprise, this does not extend to the main screen. It is called “Screen1” by default, and cannot be changed.

The screen GUI was changed somewhat for this update. Two new buttons were added. One to open the configuration screen and one to exit the application. The components were also move to the bottom of the screen (don’t want to cover the mosquito, do we?).

main

Every time the main screen is open, it’s 100 ms timer restarts. That is where the command is sent to the JavaScript to update the configuration settings.

Once.

It is sent 1/2 second (500 ms) after the screen is open. An important note: I found that the command did not execute if initiated concurrently with the opening of the screen. So the 500 ms delay (5 timer iterations) was introduced using a timer, which works every time.

timer

At 500 ms, the cfg.txt file is read. “CFG:” is inserted at the beginning of the string before being sent to JavaScript via the WebViewString.

sendcfg

A handler is also included for the first time this application is run. In that case, the cfg.txt file does not exists. As result, an exception is thrown when attempting to “ReadFrom” the file. An error handler was added for this condition.

error2

Error 2101 occurs when the file does not exist. In this case, a file is created with default values for the configurable parameters.

JavaScript for Configurable MQTT

The JavaScript also required revisions to support MQTT configurable parameters. The original version only handled IoT requests. These were identified with ‘GET:’ as the first 4 characters. Similarly, the added configuration command is now recognized as WebViewStrings beginning with “CFG:”

 
 
  1. // 10 Hz WebViewString polling timer function --------------------------------------&gt;
  2. function AppInventorServer() {
  3. var request = window.AppInventor.getWebViewString(); // Get WebViewString
  4.     if(request.substring(0, 4)=="GET:") {                // Validate request
  5.      window.AppInventor.setWebViewString("");         // Reset String (process once)
  6.         $("#request").val(request.substring(4, request.length)); //set request html textbox
  7.         SendMqttRequest();                               // Send Mqtt Request
  8.     }
  9.     if(request.substring(0, 4)=="CFG:") {                // Validate request
  10.      window.AppInventor.setWebViewString("");         // Reset String (process once)
  11.         var cfgpar = JSON.parse(request.substring(4, request.length));
  12.         txtopic = cfgpar.mqtt_txtopic;
  13.         rxtopic = cfgpar.mqtt_rxtopic;
  14.         mqtturl = cfgpar.mqtt_broker;
  15.         client.disconnect();
  16.     }
  17.     setTimeout(AppInventorServer, 100);   // run AppInventorServer() in 100 ms
  18. }

As you can see, parsing and extracting values from a JSON string is very simple and straight-forward with JavaScript. Once the new values are set, the MQTT broker connection is updated with a call to ‘client.disconnect()’.

 
 
  1. // Callback executed upon MQTT broker disconnection --------------------------------&gt;
  2. client.ondisconnect = function(rc){
  3. client.connect(mqtturl);
  4. };
  5. // Callback executed upon MQTT broker connection -----------------------------------&gt;
  6. client.onconnect = function(rc){
  7. client.subscribe(rxtopic, 0);
  8. };

Improving the Arduino Digital Commands

The original project used hard-coded values for the Digital Get and Set commands. Ok for an example, but not so good if you need access to a channel outside the card-coded value. That was rectified with this update, which now supports digital channel selection.

dig_new2

Select “Set Arduino Digital Channel” and a new screen is open. This screen allows you to select the channel to set.

digital_sel

This screen is open with a “start value” received from the main screen that called it.

select_digit

The value returned when OK is clicked depends on the startValue. If it is a “Set” command, an Arduino SetDigital request string is returned. The selected channel and logic state from the screen’s selections are also returned with the request.

Notice that a call to close the screen is made before the main screen (Screen1) is re-opened. That is necessary when returning a startValue. If the screen is not closed, it remains open for the duration of the App execution.

Another call to this window will open another instance of the window. This is effectively a memory leak which will crash you Android device eventually. Takeaway: Close the screen before opening a new screen when passing a start value to avoid memory leaks.

As you can see, a similar value is returned for “Get” requests. But when “Get” requests are received, the channel state is not selected, it is returned from the request. When this screen is open with a “Get” request, the channel state selection GUI is suppressed.

get_digit

Sending the Arduino Digital Request

When the Digital channel selection screen is closed, the main screen is reopened. But how does it know to send the Arduino command to MQTT? After all, the screen is opened fresh, just like when the app is started.

Not quite. Remember the ‘startValue’ returned from the digital selection? That is checked to determine whether an arduino request is needed. It is done in the 100ms timer callback.

timer

At 500 ms after the screen is open, the MQTT configuration is sent to the JavaScript. But then, at 1000 ms (10 100 ms intervals), the startval string is check for a non-null value. If a string is present, it is sent to the JavaScript via the WebViewString, by calling the SendRequest procedure.

send_request

Exiting the Application

Just one more thing and we are done with this update. I thought it would be nice to add an ‘Exit’ button to close the app. What I found was that it required several clicks before the app would actually close. What was happening was that every time a different screen returned to the main screen, the previous main screen remained open and a new main screen instance was created.

Not exactly what I had in mind. Another memory leak!

This was corrected by adding a ‘close screen’ before each time one of the two added screens was open. This guarantees only one screen is open at a time while the app is running.

cfg_mqttt

With this correction, the Exit button works as intended, with only one click required.

In Closing

This example should provide a framework for anyone needing to use MQTT with an App Inventor project. The main caveat is that a broker that supports WebSockets is required. I leave it up to you as an exercise to add username and password credentials to the mqtt connection settings. The basic structure is here. And it should also be a simple task to add “GetAnalog” to the Arduino request options. Go ahead, you can do it! If you need it.

Again, here are the project files.

I hope you find this information useful…

Share This:
Facebooktwittergoogle_plusredditpinterestlinkedintumblrFacebooktwittergoogle_plusredditpinterestlinkedintumblr

Press Ctrl+C to copy the following code.
"