{"id":1486,"date":"2016-03-15T15:23:50","date_gmt":"2016-03-15T22:23:50","guid":{"rendered":"http:\/\/internetofhomethings.com\/homethings\/?p=1486"},"modified":"2016-03-15T15:23:50","modified_gmt":"2016-03-15T22:23:50","slug":"triple-server-update-part-3-more-server-features","status":"publish","type":"post","link":"https:\/\/internetofhomethings.com\/homethings\/?p=1486","title":{"rendered":"Triple Server Update &#8211; Part 3: More Server Features"},"content":{"rendered":"<p><a href=\"http:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2016\/03\/server_update_part3.jpg\" rel=\"attachment wp-att-1494\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-1494\" src=\"http:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2016\/03\/server_update_part3.jpg\" alt=\"server_update_part3\" width=\"672\" height=\"231\" srcset=\"https:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2016\/03\/server_update_part3.jpg 672w, https:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2016\/03\/server_update_part3-300x103.jpg 300w\" sizes=\"auto, (max-width: 672px) 100vw, 672px\" \/><\/a><\/p>\n<h2 style=\"text-align: center;\"><strong>More Server Features<\/strong><\/h2>\n<p>Ever heard the saying &#8220;some&#8217;s good, but more is better&#8221; ? That idiom certainly\u00a0rings true when it comes to the capabilities of the multi-purpose server I began developing recently.<\/p>\n<p>The Triple Server introduced with this series initially supported http, mqtt and coap. Subsequently, coap was dropped and replaced with the addition of an Arduino interface. Web configuration was also introduced.<\/p>\n<p>But even then, the system lacked a few highly desirable features. This update builds upon the basic \u00a0structure developed so far. Here are the new\u00a0capabilities introduced with this update:<\/p>\n<ul>\n<li>Configurable FOTA port and password<\/li>\n<li>Configurable MQTT connection port and\u00a0client id<\/li>\n<li>URL decoding for configurable parameters with special characters<\/li>\n<li>FOTA &#8211; sketch embedded wireless firmware update support<\/li>\n<li>MQTT topics linked to ChipID(MAC)<\/li>\n<li>Support for MQTT connection password enabled<\/li>\n<li>Unsolicited Arduino MQTT Publications<\/li>\n<li>Current GMT\u00a0added to http header<\/li>\n<\/ul>\n<p>There features are individually portable. When needed, they can be a great addition to many different IoT projects.<\/p>\n<p>And in case you have not seen the prior posts in this series, here is the story from the beginning:<\/p>\n<p><a href=\"http:\/\/wp.me\/p5NRQ8-jS\" target=\"_blank\">ESP8266 Triple Protocol Server<\/a><\/p>\n<p><a href=\"http:\/\/wp.me\/p5NRQ8-kB\" target=\"_blank\">Triple Server Update &#8211; Part 1: Serving Arduino<\/a><\/p>\n<p><a href=\"http:\/\/wp.me\/p5NRQ8-ku\" target=\"_blank\">Triple Server Update &#8211; Part 2: Web Configuration<\/a><\/p>\n<p><a href=\"http:\/\/wp.me\/p5NRQ8-nY\" target=\"_blank\">Triple Server Update &#8211; Part 3:\u00a0More Server Features<\/a><\/p>\n<p>Let&#8217;s get to the details of the implementation&#8230;<\/p>\n<h2 style=\"text-align: center;\"><strong>Web Configuration<\/strong><\/h2>\n<p>Starting with the Web Configuration panel:<\/p>\n<p><a href=\"http:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2016\/03\/WebConfig.jpg\" rel=\"attachment wp-att-1496\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-1496\" src=\"http:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2016\/03\/WebConfig.jpg\" alt=\"WebConfig\" width=\"522\" height=\"871\" srcset=\"https:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2016\/03\/WebConfig.jpg 522w, https:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2016\/03\/WebConfig-180x300.jpg 180w\" sizes=\"auto, (max-width: 522px) 100vw, 522px\" \/><\/a><br \/>\nFOTA password and network port settings are added to the configuration page options. When a FOTA password is specified, the user will be prompted to enter it prior to proceeding with a firmware update.<\/p>\n<p>A setting for the MQTT client id has also been included with this update. The past versions of this project simply set the client id to the MQTT username. Finally, a check-box has been\u00a0added to enable MQTT password connections.<\/p>\n<p>However, the remaining MQTT connection options specified in MQTT Version 3.1.1 such as a last will message have not been implemented in this project. There additional options were not considered essential for this application.<\/p>\n<h2 style=\"text-align: center;\"><strong>URL decoding<\/strong><\/h2>\n<p style=\"text-align: left;\">The parameters updated on the Web Configuration page are passed from the html web code back the ESP8266 sketch code using an http &#8220;GET&#8221; request. Each parameter is added to the URL, a method that can easily be parsed by the ESP8266 c-code using the functions imported from the EspressIf SDK.<\/p>\n<p style=\"text-align: left;\">This all worked well until special characters were introduced in a password string. In these cases, the special characters are <a href=\"http:\/\/www.w3schools.com\/tags\/ref_urlencode.asp\" target=\"_blank\">URL encoded<\/a>, which converts the special characters into something an URL will accept. For example, a space is considered a special character and gets converted to &#8220;%20&#8221; in an URL.<\/p>\n<p style=\"text-align: left;\">The problem occurs\u00a0when the parameter is saved. If &#8220;Hello World&#8221; is sent as an URL parameters, it becomes &#8220;Hello%20World&#8221;. This situation\u00a0is resolved by using an URL decoder prior to saving the value. This restores the parameter to it&#8217;s original value, &#8220;Hello World&#8221; in this example.<\/p>\n<p style=\"text-align: left;\">Here&#8217;s the code used for URL decoding. It should be fairly easy to follow. The basic algorithm looks for a &#8220;%hh&#8221; code and converts it back to the original &#8220;special character&#8221; value in the string:<\/p>\n<pre class=\"easycode; title:;lang:;\"> \/********************************************************\r\n\u00a0*\u00a0\u00a0URL\u00a0Message\u00a0Decoder\r\n\u00a0********************************************************\/\r\n\u00a0\r\nint\u00a0url_decode(char\u00a0*encoded_str,\u00a0char\u00a0*decoded_str)\u00a0{\r\n\u00a0\u00a0\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\/\/\u00a0While\u00a0we're\u00a0not\u00a0at\u00a0the\u00a0end\u00a0of\u00a0the\u00a0string\u00a0(current\u00a0character\u00a0not\u00a0NULL)\r\n\u00a0\u00a0\u00a0\u00a0while\u00a0(*encoded_str)\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/\u00a0Check\u00a0to\u00a0see\u00a0if\u00a0the\u00a0current\u00a0character\u00a0is\u00a0a\u00a0%\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0if\u00a0(*encoded_str\u00a0==\u00a0'%')\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/\u00a0Grab\u00a0the\u00a0next\u00a0two\u00a0characters\u00a0and\u00a0move\u00a0encoded_str\u00a0forwards\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0encoded_str++;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0char\u00a0high\u00a0=\u00a0*encoded_str;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0encoded_str++;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0char\u00a0low\u00a0=\u00a0*encoded_str;\r\n\u00a0\u00a0\u00a0\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/\u00a0Convert\u00a0ASCII\u00a00-9A-F\u00a0to\u00a0a\u00a0value\u00a00-15\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0if\u00a0(high\u00a0&gt;\u00a00x39)\u00a0high\u00a0-=\u00a07;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0high\u00a0&amp;=\u00a00x0f;\r\n\u00a0\u00a0\u00a0\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/\u00a0Same\u00a0again\u00a0for\u00a0the\u00a0low\u00a0byte:\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0if\u00a0(low\u00a0&gt;\u00a00x39)\u00a0low\u00a0-=\u00a07;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0low\u00a0&amp;=\u00a00x0f;\r\n\u00a0\u00a0\u00a0\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/\u00a0Combine\u00a0the\u00a0two\u00a0into\u00a0a\u00a0single\u00a0byte\u00a0and\u00a0store\u00a0in\u00a0decoded_str:\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0*decoded_str\u00a0=\u00a0(high\u00a0&lt;&lt;\u00a04)\u00a0|\u00a0low;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\u00a0else\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/\u00a0All\u00a0other\u00a0characters\u00a0copy\u00a0verbatim\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0*decoded_str\u00a0=\u00a0*encoded_str;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\r\n\u00a0\u00a0\u00a0\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/\u00a0Move\u00a0both\u00a0pointers\u00a0to\u00a0the\u00a0next\u00a0character:\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0encoded_str++;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0decoded_str++;\r\n\u00a0\u00a0\u00a0\u00a0}\r\n\u00a0\u00a0\u00a0\u00a0\/\/\u00a0Terminate\u00a0the\u00a0new\u00a0string\u00a0with\u00a0a\u00a0NULL\u00a0character\u00a0to\u00a0trim\u00a0it\u00a0off\r\n\u00a0\u00a0\u00a0\u00a0*decoded_str\u00a0=\u00a00;\r\n}<\/pre>\n<h2 style=\"text-align: center;\"><strong>FOTA Support<\/strong><\/h2>\n<p>Firmware over-the-air (FOTA) is a very useful option to include with your sketch. The Arduino makes this capability very clean and easy to embed in your code. I found it very useful for debugging code, as it freed up the serial port for test messages. The drawback, however, is that FOTA reduces your effective maximum sketch size in half. This was not a problem in this project, which currently only uses 26% of the available 4MB of my ESP8266-12 flash chip. Just be aware if you are using an older ESP8266-1 with this project, FOTA cannot be used as the sketch uses more than 50% of the 512K flash installed on many of the older ESP8266 versions.<\/p>\n<p>Here is the implementation in this sketch:<\/p>\n<p>A function was added and called in the setup() function.<\/p>\n<p>init_FOTA();<\/p>\n<pre class=\"easycode; title:;lang:;\"> void\u00a0init_FOTA()\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0String\u00a0buff;\r\n\u00a0\u00a0\u00a0\u00a0int\u00a0pt;\r\n\r\n\u00a0\u00a0\u00a0\u00a0\/\/Hostname\u00a0defaults\u00a0to\u00a0esp8266-[ChipID]\u00a0(no\u00a0change\u00a0to\u00a0default,\u00a0which\u00a0is\u00a0unique\u00a0(ChipID\u00a0=\u00a0last\u00a03\u00a0MAC\u00a0HEX)\r\n\u00a0\u00a0\u00a0\u00a0\/\/ArduinoOTA.setHostname(\"myesp8266\");\r\n\u00a0\u00a0\u00a0\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\/\/Set\u00a0FOTA\u00a0Network\u00a0Port\r\n\u00a0\u00a0\u00a0\u00a0GetEepromVal(&amp;buff,\u00a0EEPROM_MQTT_PT,\u00a0EEPROM_INT16);\r\n\u00a0\u00a0\u00a0\u00a0pt\u00a0=\u00a0atoi(buff.c_str());\r\n\u00a0\u00a0\u00a0\u00a0ArduinoOTA.setPort(pt);\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\/\/Set\u00a0OTA\u00a0authentication\u00a0(password)\r\n\u00a0\u00a0\u00a0\u00a0GetEepromVal(&amp;buff,\u00a0EEPROM_FOTA_PW,\u00a0EEPROM_CHR);\r\n\u00a0\u00a0\u00a0\u00a0ArduinoOTA.setPassword((char\u00a0*)buff.c_str());\r\n\u00a0\u00a0\u00a0\u00a0\r\n\u00a0\u00a0\u00a0\u00a0ArduinoOTA.onStart([]()\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Serial.println(\"Start\");\r\n\u00a0\u00a0\u00a0\u00a0});\r\n\u00a0\u00a0\u00a0\u00a0ArduinoOTA.onEnd([]()\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Serial.println(\"\\nEnd\");\r\n\u00a0\u00a0\u00a0\u00a0});\r\n\u00a0\u00a0\u00a0\u00a0ArduinoOTA.onProgress([](unsigned\u00a0int\u00a0progress,\u00a0unsigned\u00a0int\u00a0total)\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Serial.printf(\"Progress:\u00a0%u%%\\r\",\u00a0(progress\u00a0\/\u00a0(total\u00a0\/\u00a0100)));\r\n\u00a0\u00a0\u00a0\u00a0});\r\n\u00a0\u00a0\u00a0\u00a0ArduinoOTA.onError([](ota_error_t\u00a0error)\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Serial.printf(\"Error[%u]:\u00a0\",\u00a0error);\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0if\u00a0(error\u00a0==\u00a0OTA_AUTH_ERROR)\u00a0Serial.println(\"Auth\u00a0Failed\");\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0else\u00a0if\u00a0(error\u00a0==\u00a0OTA_BEGIN_ERROR)\u00a0Serial.println(\"Begin\u00a0Failed\");\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0else\u00a0if\u00a0(error\u00a0==\u00a0OTA_CONNECT_ERROR)\u00a0Serial.println(\"Connect\u00a0Failed\");\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0else\u00a0if\u00a0(error\u00a0==\u00a0OTA_RECEIVE_ERROR)\u00a0Serial.println(\"Receive\u00a0Failed\");\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0else\u00a0if\u00a0(error\u00a0==\u00a0OTA_END_ERROR)\u00a0Serial.println(\"End\u00a0Failed\");\r\n\u00a0\u00a0\u00a0\u00a0});\r\n\u00a0\u00a0\u00a0\u00a0ArduinoOTA.begin();\r\n\u00a0\u00a0\u00a0\u00a0Serial.print(\"FOTA\u00a0Initialized\u00a0using\u00a0IP\u00a0address:\u00a0\");\r\n\u00a0\u00a0\u00a0\u00a0Serial.println(WiFi.localIP());\r\n}<\/pre>\n<p>Note that the FOTA port and password values are set from values stored in EEPROM. That means they can be set from the application, so the sketch does not require modification to change the values.<\/p>\n<p>More on that when we go through the updates to the Web Configuration page.<\/p>\n<h2 style=\"text-align: center;\"><strong>Unique\u00a0MQTT topics<\/strong><\/h2>\n<p>It a system is built using multiple devices, it may be desirable to ensure that each device sends and receives MQTT message over unique topic names. In order to link\u00a0the MQTT topic to the ESP8266 device, the default topic now includes an\u00a0expanded ChipID.<\/p>\n<p>That is to say, while the official ChipID uses the last 3 hex characters of the device&#8217;s MAC address, the expanded id in this application uses all six MAC address hex values.<\/p>\n<p>The implementation here simply appends the MAC to a topic prefix. The result is a topic name unique to each ESP8266 device.<\/p>\n<pre class=\"easycode; title:;lang:;\"> \u00a0void\u00a0AddMAC(char\u00a0*\u00a0prefix,\u00a0char\u00a0*\u00a0topic)\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0uint8_t\u00a0MAC_array[6];\r\n\u00a0\u00a0\u00a0\u00a0\u00a0WiFi.macAddress(MAC_array);\r\n\u00a0\u00a0\u00a0\u00a0\u00a0sprintf(topic,\"%s\",\u00a0prefix);\r\n\u00a0\u00a0\u00a0\u00a0\u00a0for\u00a0(int\u00a0i\u00a0=\u00a00;\u00a0i\u00a0&lt;\u00a0sizeof(MAC_array);\u00a0++i){\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0sprintf(topic,\"%s%02x\",topic,\u00a0MAC_array[i]);\r\n\u00a0\u00a0\u00a0\u00a0\u00a0}\r\n\u00a0}<\/pre>\n<h2 style=\"text-align: center;\"><strong>MQTT Password Connection<\/strong><\/h2>\n<p>The initial design of the MQTT server in this project relied solely on the test.mosquitto.org broker. This simple test site does\u00a0not offer the option of connecting with passwords. But, of course, a password protected connection is preferred. And recently, I had set up my <a href=\"http:\/\/wp.me\/p5NRQ8-my\" target=\"_blank\">own MQTT broker<\/a>, with the capability to require username\/password credentials with connections. So adding this to the ESP8266 based server was obviously needed.<\/p>\n<p>Here is the mqtt connection code used to establish password enabled and password-free connections. \u00a0Note that the MQTT client class object uses a different client.connect() prototype when using a password to connect.:<\/p>\n<pre class=\"easycode; title:;lang:;\"> void\u00a0MqttServer_reconnect(void)\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0int\u00a0fst=10;\r\n\u00a0\u00a0\u00a0\u00a0bool\u00a0connected\u00a0=\u00a0false;\r\n\u00a0\u00a0\u00a0\u00a0\/\/\u00a0Loop\u00a0until\u00a0we're\u00a0reconnected\u00a0(give\u00a0up\u00a0after\u00a010\u00a0tries)\r\n\u00a0\u00a0\u00a0\u00a0while\u00a0(!client.connected()&amp;&amp;(fst!=0))\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Serial.print(\"Attempting\u00a0MQTT\u00a0connection...\");\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/\u00a0Connect\u00a0to\u00a0MQTT\u00a0Server\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0if(mqtt_pw_enable)\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0connected\u00a0=\u00a0client.connect(mqtt_ci.c_str(),mqtt_un.c_str(),mqtt_pw.c_str());\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0else\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0connected\u00a0=\u00a0client.connect(mqtt_ci.c_str());\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0if\u00a0(connected)\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/\u00a0Successful\u00a0connection\u00a0message\u00a0&amp;\u00a0subscribe\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Serial.println(\"connected\");\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0client.subscribe((char\u00a0*)mqtt_rt.c_str());\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0fst--;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\u00a0else\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/\u00a0Failed\u00a0to\u00a0connect\u00a0message\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Serial.print(\"failed,\u00a0rc=\");\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Serial.print(client.state());\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Serial.println(\"\u00a0try\u00a0again\u00a0in\u00a05\u00a0seconds\");\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/\u00a0Wait\u00a05\u00a0seconds\u00a0before\u00a0retrying\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0delay(5000);\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\r\n\u00a0\u00a0\u00a0\u00a0}\r\n}<\/pre>\n<h2 style=\"text-align: center;\"><strong>Unsolicited Arduino MQTT Publications<\/strong><\/h2>\n<p>The Arduino server has been designed to query the Arduino for sensor or pin status. These queries are initiated from external http or MQTT requests. But this structure did not allow the Arduino to independently publish messages to an MQTT topic. That capability has now been added. MQTT messages originating in the Arduino are now possible.<\/p>\n<p>Adding this capability required a simple restructure of the serial part data handler. Now, any time the end of line is received from the serial port, the ESP8266 sends the response to the http or mqtt channels when initiated, and to the mqtt topic when received without an initial request originating from the ESP8266.<\/p>\n<p>This change impacted the function that initiates requests\u00a0to the Arduino via the serial port. With the revised implementation, the function no longer waits for a reply:<\/p>\n<pre class=\"easycode; title:;lang:;\">String\u00a0ArduinoSendReceive(String\u00a0req)\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0String\u00a0fromArduino\u00a0=\u00a0\"\";\r\n\u00a0\u00a0\u00a0\u00a0Serial.println(req);\r\n\u00a0\u00a0\u00a0\u00a0long\u00a0start\u00a0=\u00a0millis();\r\n\u00a0\u00a0\u00a0\u00a0fromArduino\u00a0=\u00a0\"\";\r\n\u00a0\u00a0\u00a0\u00a0return\u00a0fromArduino;\r\n}\r\n<\/pre>\n<p>Instead, a new function, &#8220;MonitorSerialLine()&#8221;, was added to process all data received on the serial port, both initiated and unsolicited.<\/p>\n<pre class=\"easycode; title:;lang:;\"> void\u00a0MonitorSerialLine()\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0if(arduino_server)\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0while\u00a0(Serial.available())\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/\u00a0get\u00a0the\u00a0new\u00a0byte:\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0char\u00a0inChar\u00a0=\u00a0(char)Serial.read();\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/\u00a0add\u00a0it\u00a0to\u00a0the\u00a0inputString:\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0SerialInString\u00a0+=\u00a0inChar;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/\u00a0if\u00a0the\u00a0incoming\u00a0character\u00a0is\u00a0a\u00a0newline,\u00a0set\u00a0a\u00a0flag\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/\u00a0so\u00a0the\u00a0following\u00a0code\u00a0can\u00a0process\u00a0it\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0if\u00a0(inChar\u00a0==\u00a0'\\n')\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0NewSerialLineRx\u00a0=\u00a0true;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/Send\u00a0Received\u00a0String\u00a0as\u00a0MQTT\u00a0Message\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0if(NewSerialLineRx)\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/If\u00a0not\u00a0server\u00a0request,\u00a0just\u00a0forward\u00a0Arduino\u00a0Message\u00a0to\u00a0MQTT\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0if(active_svr_rqst\u00a0==\u00a0SVR_NONE)\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0active_svr_rqst\u00a0=\u00a0SVR_MQTT;\u00a0\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Server_SendReply(active_svr_rqst,\u00a0REPLY_TEXT,\u00a0SerialInString);\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0active_svr_rqst\u00a0=\u00a0SVR_NONE;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0SerialInString=\"\";\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0NewSerialLineRx=false;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/Check\u00a0for\u00a0timeout\u00a0for\u00a0Arduino\u00a0Server\u00a0requests\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0if(active_svr_rqst\u00a0!=\u00a0SVR_NONE)\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0if(start_wait==0)\u00a0start_wait\u00a0=\u00a0millis();\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0if(\u00a0(millis()\u00a0-\u00a0start_wait)\u00a0\u00a0&gt;5000\u00a0)\u00a0{\u00a0\/\/5\u00a0sec\u00a0timeout\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0SerialInString\u00a0=\u00a0\"no\u00a0arduino\u00a0reply\u00a0received\";\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Server_SendReply(active_svr_rqst,\u00a0REPLY_TEXT,\u00a0SerialInString);\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\/\/Reset\u00a0Request\u00a0Parameters\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0active_svr_rqst\u00a0=\u00a0SVR_NONE;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0start_wait=0;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0SerialInString=\"\";\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0NewSerialLineRx=false;\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\r\n\u00a0\u00a0\u00a0\u00a0}\r\n}<\/pre>\n<h2 style=\"text-align: center;\"><strong>Current GMT<\/strong><\/h2>\n<p><a href=\"http:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2016\/03\/gmt.png\" rel=\"attachment wp-att-1501\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-1501 size-full\" src=\"http:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2016\/03\/gmt.png\" alt=\"gmt\" width=\"124\" height=\"49\" \/><\/a><\/p>\n<p>The http response header requires an identification of current GMT (for the Date header) and a current date offeset (for expiration headers). Prior versions of this project simply added hard-coded, static values for these headers. While it did no impact the http response transmission, it\u00a0was\u00a0obviously inaccurate header content.<\/p>\n<p>In order to create accurate date headers, current GMT must be known. So time is now retrieved from a NIST server. This required\u00a0an initialization routine\u00a0to be run in the sketch&#8217;s\u00a0setup() function. The offset is set to 0 so we get GMT when time is requested.<\/p>\n<pre class=\"easycode; title:;lang:;\"> void\u00a0init_GmtTime()\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0configTime(0,\u00a00,\u00a0\"pool.ntp.org\",\u00a0\"time.nist.gov\");\r\n\u00a0\u00a0\u00a0\u00a0Serial.print(\"\\nWaiting\u00a0for\u00a0time\");\r\n\u00a0\u00a0\u00a0\u00a0while\u00a0(!time(nullptr))\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Serial.print(\".\");\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0delay(1000);\r\n\u00a0\u00a0\u00a0\u00a0}\r\n\u00a0\u00a0\u00a0\u00a0Serial.print(\"\\r\\nTime\u00a0has\u00a0been\u00a0acquired\u00a0from\u00a0internet\u00a0time\u00a0service\\r\\nCurrent\u00a0GMT:\u00a0\");\r\n\r\n\u00a0\u00a0\u00a0\u00a0time_t\u00a0now\u00a0=\u00a0time(nullptr);\r\n\u00a0\u00a0\u00a0\u00a0Serial.println(ctime(&amp;now));\r\n}\r\n<\/pre>\n<p>Here is how time is now retrieved when creating an http header:<\/p>\n<pre class=\"easycode; title:;lang:;\">String\u00a0gmt,expire;\r\ntime_t\u00a0now\u00a0=\u00a0time(nullptr);\r\ngmt\u00a0=\u00a0ctime(&amp;now);\r\nnow\u00a0+=\u00a03600\u00a0*\u00a024;\u00a0\u00a0\/\/Expires\u00a0in\u00a01\u00a0day\r\nexpire\u00a0=\u00a0ctime(&amp;now);\r\n\r\ngmt\u00a0=\u00a0gmt.substring(0,3)\u00a0+\u00a0\",\"\u00a0+\u00a0gmt.substring(7,10)\u00a0+\u00a0gmt.substring(3,7)\u00a0+\u00a0gmt.substring(19,24)\u00a0+\u00a0\r\n      gmt.substring(10,19)\u00a0+\u00a0\"\u00a0GMT\";\r\n\r\nexpire\u00a0=\u00a0expire.substring(0,3)\u00a0+\u00a0\",\"\u00a0+\u00a0expire.substring(7,10)\u00a0+\u00a0expire.substring(3,7)\u00a0+\u00a0\r\n         expire.substring(19,24)\u00a0+\u00a0expire.substring(10,19)\u00a0+\u00a0\"\u00a0GMT\";\r\n<\/pre>\n<p>The string returned from a call to ctime() is in a different format than what is needed in the http. The String class &#8220;substring&#8221; method is uses to re-sort the ctime() string into the needed http header\u00a0format.<\/p>\n<p>ctime() format:<\/p>\n<pre class=\"result notranslate\">Mon Mar 14 08:23:14 2016<\/pre>\n<p>http header format:<\/p>\n<pre class=\"result notranslate\">Mon,13 Mar 2016 08:23:14 GMT<\/pre>\n<h2 style=\"text-align: center;\"><strong>In Closing<\/strong><\/h2>\n<p>This update establishes a solid framework for building IoT projects with both http and MQTT server support. While Arduino is used in the example, any external device with a standard serial link can also be easily added to this structure. And when the system is deployed, updates can be deployed wirelessly, eliminating\u00a0the need to connect a physical serial interface to the system.<\/p>\n<p>Here is the <a href=\"https:\/\/github.com\/internetofhomethings\/Configurable-Web-Server\" target=\"_blank\">updated GitHub repository<\/a> for this project.<\/p>\n<p>I hope you find this information useful&#8230;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>More Server Features Ever heard the saying &#8220;some&#8217;s good, but more is better&#8221; ? That idiom certainly\u00a0rings true when it comes to the capabilities of the multi-purpose server I began &hellip;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1,64,10,19,111],"tags":[121,122,120],"class_list":["post-1486","post","type-post","status-publish","format-standard","hentry","category-alltheposts","category-arduino","category-esp8266","category-internet-of-things","category-mqtt","tag-fota-support","tag-serving-arduino","tag-url-decoder"],"_links":{"self":[{"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/posts\/1486","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1486"}],"version-history":[{"count":19,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/posts\/1486\/revisions"}],"predecessor-version":[{"id":1508,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/posts\/1486\/revisions\/1508"}],"wp:attachment":[{"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1486"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1486"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1486"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}