{"id":345,"date":"2015-04-24T15:26:44","date_gmt":"2015-04-24T22:26:44","guid":{"rendered":"http:\/\/internetofhomethings.com\/homethings\/?p=345"},"modified":"2015-09-04T07:55:10","modified_gmt":"2015-09-04T14:55:10","slug":"porting-spark-core-weather-sensor-iot-to-esp8266-12","status":"publish","type":"post","link":"https:\/\/internetofhomethings.com\/homethings\/?p=345","title":{"rendered":"Porting Spark Core Weather Sensor IoT to ESP8266-12"},"content":{"rendered":"<h2 style=\"text-align: center;\">\u00a0<strong>The Project<\/strong><\/h2>\n<p>After my initial\u00a0tinkering with\u00a0the ESP8266, I could\u00a0visualize of lots\u00a0of practical applications. Suddenly, the price barrier was shattered. Every little thing can be connected. First up was the task of freeing up my relatively expensive Spark Core application, replacing it with a\u00a0dirt cheap ESP8266.<\/p>\n<p>It seemed like an incredible wasted resource to use my $39 Spark Core to perform the duties that a $2.86 ESP8266-12 could just as easily perform.<\/p>\n<p>Or could it?<\/p>\n<p>You never know how a prototype will ultimately wind up looking like at the end of the day. My plan was to transfer the Weather Sensor functions that the Spark Core was performing to an ESP8266. While not overly complicated, 8 measurements using 3 sensor types were needed. This included:<\/p>\n<ul>\n<li>Three Temperature Sensors on a one-wire bus (Using DS18B20)<\/li>\n<li>A Humidity\/Temperature Sensor on a separate one-wire bus (Using DHT11)<\/li>\n<li>Barometric Pressure, Altitude, Temperature Sensor via I2C (Using BMP085)<\/li>\n<\/ul>\n<hr \/>\n<h2 style=\"text-align: center;\">\u00a0<strong>Hardware\u00a0Interface<\/strong><\/h2>\n<p>Here is my current interface using a Spark Core. It is mounted on a standard solder-less breadboard with a micro-USB connector for programming and power. Only two pull-up resistors were needed to provide a &#8216;strong&#8217; one-wire interface. This set-up has been working 24-7 non-stop for the past 9 months.<\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"http:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2015\/03\/spark-schematic3.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-352\" src=\"http:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2015\/03\/spark-schematic3-1024x469.png\" alt=\"Spark Core Weather Sensors\" width=\"620\" height=\"284\" srcset=\"https:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2015\/03\/spark-schematic3-1024x469.png 1024w, https:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2015\/03\/spark-schematic3-300x137.png 300w\" sizes=\"auto, (max-width: 620px) 100vw, 620px\" \/><\/a><\/p>\n<p style=\"text-align: center;\"><b>Current set-up using a Spark Core MPU<\/b><\/p>\n<p>This interface only\u00a0required 4 digital signals, those two pull-up\u00a0resistors and a 3.3V power source. So you see that this is\u00a0well within the capabilities of the ESP8266-12, which has 9 total (6\u00a0usable) general purpose digital IO pins exposed at the module interface.<\/p>\n<p>My current configuration pulls data from the Spark Core. This is accomplished by sending http GET commands to retrieve the sensor values\u00a0and storing them into a mySQL database once every hour by a scheduled CRON task on my web hosting platform. With the correct application loaded to the new module, a tweak to that script should be all that is necessary for the conversion to the ESP8266 data acquisition change.<\/p>\n<p><a href=\"http:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2015\/04\/ESP8266-schematic.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-420\" src=\"http:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2015\/04\/ESP8266-schematic-1024x436.png\" alt=\"ESP8266 Weather Sensor Schematic\" width=\"620\" height=\"264\" srcset=\"https:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2015\/04\/ESP8266-schematic-1024x436.png 1024w, https:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2015\/04\/ESP8266-schematic-300x128.png 300w\" sizes=\"auto, (max-width: 620px) 100vw, 620px\" \/><\/a><\/p>\n<p style=\"text-align: center;\"><b>New\u00a0set-up using an ESP8266 MPU<\/b><\/p>\n<p>Just like the old set-up, four digital pins are used to interface with the 3 sensors.<\/p>\n<p><center>[wptg_comparison_table id=&#8221;1&#8243;]<\/center><\/p>\n<p style=\"text-align: center;\"><b>Spark Core DIO vs ESP8266 GPIO usage<\/b><\/p>\n<p>Initially, I had planned to use GPIO16 for the DS18B20 one-wire interface. That would have physically routed all the pins used for the sensors on one side of the ESP8266. This would not work, however, since\u00a0GPIO16 can be used as an input or an output, but does not have the INPUT PULLUP,\u00a0and \u00a0OUTPUT_OPEN_DRAIN capability of the other\u00a0GPIO pins. That feature is needed for the one-wire interface. But since the ESP8266-12 has additional digital pins, I simply changed the connection to use GPIO4\u00a0instead.<\/p>\n<p>Note that the circuit has a 100 ohm series resistor with a 3.3V zener diode connected to the ESP8266 serial receive pin. This protects the module from potential damage from a 5V serial transmit source.<\/p>\n<p>A LM1117 3.3V regulator is used to provide Vcc voltage. This device can provide up to 800ma of current, well above the ESP8266 needs. Using a USB to serial converter, the USB is connected to a 5V external supply (wall, battery, car adapter&#8230;) for the fielded ESP8266-12 circuit.<\/p>\n<p>A large (470 uF) capacitor was placed across the 3.3V to stabilize the supply and minimize unwanted ESP8266 resets. An additional capacitor was connected to the reset signal, also to eliminate resets from spikes on the pin. Finally, for device decoupling, 0.1uF capacitors were placed across the ESP8266 and BMP085 Vcc to ground pins. These must be placed as close to the device pins as possible for maximum effectiveness.<\/p>\n<p>Switches were added to support flashing and warm resets.<\/p>\n<p>The circuit was assembled on a printed circuit board (PCB). I used 30 AWG wire to attach the 16 ESP8266-12 interface contacts to the PCB.\u00a0<a href=\"http:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2015\/04\/circuit21.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-390\" src=\"http:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2015\/04\/circuit21-1024x525.jpg\" alt=\"ESP8266-12 Weather Sensors\" width=\"620\" height=\"318\" srcset=\"https:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2015\/04\/circuit21-1024x525.jpg 1024w, https:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2015\/04\/circuit21-300x154.jpg 300w\" sizes=\"auto, (max-width: 620px) 100vw, 620px\" \/><\/a>The Barometric Pressure\/Temperature sensor (BMP085) was positioned in the center of the PCB. Interface to the external DS18B20 and DHT11 sensors are made at the green terminal block.\u00a0<a href=\"http:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2015\/04\/circuitpicture21.jpg\"><br \/>\n<\/a><a href=\"http:\/\/internetofhomethings.com\/homethings\/wp-content\/uploads\/2015\/04\/circuit2.jpg\">N<\/a>ote that the 3.3v signal (green wire) from the USB to serial adapter was clipped and not used. That source lacks the current capability needed for proper operation of the ESP8266.<\/p>\n<hr \/>\n<h2 style=\"text-align: center;\">\u00a0<strong>Software\u00a0Implementation<\/strong><\/h2>\n<p>This was my first serious project using the ESP8266 after doing the basic &#8220;getting started&#8221; exercises. I started with nodeMCU and lua, then migrated to the SDK, and finalized the code using the Arduino IDE. This was not by choice but by necessity as there were insurmountable problems with lua and the SDK.\u00a0And as of this post, there remains issues even with the Arduino IDE. But this article will remain focused on the porting solution. See the following posts related to issues I encountered during the development of the weather sensor porting firmware.<\/p>\n<ul>\n<li><a title=\"4 reasons I stopped using NodeMCU for ESP8266 Development\" href=\"http:\/\/wp.me\/p5NRQ8-6Q\" target=\"_blank\"><strong>4 reasons\u00a0I\u00a0stopped using NodeMcu\/Lua with the ESP8266<\/strong><\/a><\/li>\n<li><strong><a title=\"Why I had to abandon the ESP8266 SDK\" href=\"http:\/\/wp.me\/p5NRQ8-6S\" target=\"_blank\">Why I had to abandon the ESP8266 SDK &#8211; until a few things are corrected&#8230;<\/a><\/strong><\/li>\n<li><a title=\"Issues with the Arduino IDE for ESP8266\" href=\"http:\/\/wp.me\/p5NRQ8-6V\" target=\"_blank\"><strong>Issues with the Arduino IDE for ESP8266<\/strong><\/a><\/li>\n<li><strong><a title=\"3 ways to eliminate ESP8266 resets\" href=\"http:\/\/wp.me\/p5NRQ8-6o\" target=\"_blank\">4\u00a0ways to eliminate ESP8266 resets<\/a><\/strong><\/li>\n<li><strong><a href=\"http:\/\/wp.me\/p5NRQ8-8G\" target=\"_blank\">What to do when you can no longer flash new\u00a0ESP8266\u00a0firmware<\/a><\/strong><\/li>\n<\/ul>\n<hr \/>\n<p><strong>Arduino IDE Web server<\/strong><\/p>\n<p>My\u00a0code is\u00a0based on the Arduino IDE example &#8220;WiFiWebServer&#8221;. After confirming the example code worked to turn an LED on &amp; off \u00a0from an external Internet connection, modifications were made to support the ported sensor requirements. The sketch and forked library files are accessible in <a title=\"GitHub here\" href=\"https:\/\/github.com\/surfnturf57\/ESP8266-WeatherSensor-Webserver\" target=\"_blank\">GitHub here<\/a>.<\/p>\n<p>After many iterations, discovering what worked and what wouldn&#8217;t function, I came up with the following structure.<\/p>\n<p>Included libraries:<\/p>\n<p>#include &lt;OneWire.h&gt;<br \/>\n#include &lt;ESP8266WiFi.h&gt;<br \/>\n#include &lt;Wire.h&gt;<br \/>\n#include &lt;DHT.h&gt;<br \/>\n#include &lt;Adafruit_BMP085.h&gt;<br \/>\n#include &lt;UtilityFunctions.h&gt;<\/p>\n<p>Everything worked &#8220;off-the-shelf&#8221; except for the BMP085 driver. The problem was that the &#8220;pow&#8221; function, used to calculate altitude, was not linked properly from the built-in IDE libraries. And if I tried to include &#8220;math.h&#8221;, \u00a0which includes the &#8220;pow&#8221; function, the compiler failed with an out of memory error. I also attempted to implement a recursively called substitute function for the missing &#8220;pow&#8221;&#8230;unsuccessfully. I ended up\u00a0removing the calls to the altitude function, and the need for the pow function.\u00a0Not a big loss considering the fact that my\u00a0sensors are positioned in a fixed location, the altitude will never change. My implementation of the pow function remains in the GitHub repository in the UtilityFunctions.c file in case someone may wish to explore this further.<br \/>\n<a name=\"arduinoidestructure\"><\/a><\/p>\n<hr \/>\n<p><strong>Structure of the Arduino IDE loop()<\/strong><\/p>\n<p>Here are the key attributes of my code required for the most reliable operation:<\/p>\n<p><strong>1.\u00a0WiFi connected check<\/strong><\/p>\n<p>First thing I added to the top of the sketches loop was a check to determine if the WiFi was still connected. This became necessary when I noticed that sometimes the connection was dropped, resulting in a non-responsive ESP8266 to&#8221;http GET&#8221; requests.<\/p>\n<p><strong>2. Busy flag<\/strong><\/p>\n<p>As you may well know, sending an &#8220;http GET&#8221; request by entering an URL into a web browser also creates\u00a0<span style=\"line-height: 1.5;\">several request for &#8220;favicon&#8221;. This sometimes created a problem when the ESP8266\u00a0<\/span><span style=\"line-height: 1.5;\">sent it&#8217;s reply back to the browser and returned to the top of the loop. It appears that the reply, sent <\/span><span style=\"line-height: 1.5;\">using<\/span><span style=\"line-height: 1.5;\">\u00a0&#8220;client.print(msg);&#8221; is a non-blocking call. That means <\/span><span style=\"line-height: 1.5;\">t<\/span><span style=\"line-height: 1.5;\">he ESP8266 continues execution while the reply message send is in progress, This results in cases where the &#8220;favicon&#8221; request is received before <\/span><span style=\"line-height: 1.5;\">the\u00a0<\/span><span style=\"line-height: 1.5;\">reply is sent. I figured this may be the cause for some of the ESP8266 lock-ups and resets I was experiencing. So I added a busy flag to block the processing of any new &#8220;http GET&#8221; requests until the current one is complete.<\/span><\/p>\n<p><strong>3.\u00a0Sensor Reads<\/strong><\/p>\n<p>When all the sensor reads were attempted each iteration of the loop(), the ESP8266 kept resetting. I believe this was because the watchdog timer, set to about 5 seconds by default and does not appear to be controllable at this time, would timeout before the sensor reads were complete. Upon a timeout, the ESP8266 resets.<\/p>\n<p>The solution was to limit the sensor reads to one read every 2.5 seconds. or a total of about 20 seconds to refresh all 8 sensors. This worked, and the resets no longer occurred endlessly.<\/p>\n<p><strong>4. Returns<\/strong><\/p>\n<p>The watchdog timeouts and subsequent resets \u00a0occured frequently when all the steps in the sketch loop were executed every iteration. This was significantly reduced by returning from the loop after each significant\u00a0event was\u00a0processed.<\/p>\n<p>Loop sequence returns:<\/p>\n<ul>\n<li>After Wifi connected, if needed<\/li>\n<li>If busy<\/li>\n<li>After a sensor is read<\/li>\n<li>If no client detected<\/li>\n<li>After client is killed<\/li>\n<li>If &#8220;favicon&#8221; request detected<\/li>\n<\/ul>\n<p><strong>5.\u00a0Watchdog Timer Resets<\/strong><\/p>\n<p>The wdt_feed() should reset the watchdog timer. I have sprinkled some calls to wdt_feed() in my loop() after tasks that take some time to complete to avoid timeout resets.<\/p>\n<p><strong>6. Reply &#8211; json string encoding<\/strong><\/p>\n<p>The sensor data is returned as a json string for easy processing with a php or jquery script. I have attempted to add a few different json libraries to my Arduino IDE sketch, without success. They either would not compile or blew the memory space. So I ended up adding a simple json encoder to my sketch. It only supports key:value entries at the top level, but works flawlessly and uses an absolute minimum amount of memory. Check it out in my <a title=\"sketch\" href=\"https:\/\/github.com\/surfnturf57\/ESP8266-WeatherSensor-Webserver\" target=\"_blank\">sketch<\/a>.<\/p>\n<p><strong>7. Heartbeat Data logging<\/strong><\/p>\n<p>With all the problems I had with memory management and leaks using the nodeMCU\/lua environment, for potential troubleshooting, I added a serial print to log 3 parameters:<\/p>\n<ol>\n<li>free heap<\/li>\n<li>processor time since\u00a0last reset<\/li>\n<li>last sensor read<\/li>\n<\/ol>\n<p>This was output every time a sensor was read (2.5 second intervals) and logged to a file using my terminal program. It has been very helpful in debugging problems. just like with lua and the SDK, I noticed the free heap drops for each consecutive &#8220;http GET&#8221;, lingering for a minute or so. This means in the current state of the ESP8266 hardware\/software combination, you cannot continuously bang the unit with &#8220;http GET&#8221; requests. For applications that need to periodically extract information from the module over the network, a minimum of 1.5 minutes between requests is needed for a reliable, stable operation.<\/p>\n<hr \/>\n<h2 style=\"text-align: center;\">\u00a0<strong>Conclusion<\/strong><\/h2>\n<p>I can claim a\u00a0success in the porting of the Spark Core Weather sensor code to the ESP8266 platform. \u00a0All the same functionality worked within the ESP8266 constraints, with plenty of code space to spare.<\/p>\n<p>And while I am still in the process of performing some &#8220;stress tests&#8221; to determine whether\u00a0it is sufficiently \u00a0robust to be relied upon for around the clock operation, it is looking good so far! After implementing both hardware and code measures to eliminate resets, I have not seen one yet. But this has only been about two non-stop days so far&#8230;<\/p>\n<p>With the current level of interest, I expect the dependability of this device to improve with time. Higher quality flash chips, that reliably support more flash cycles than the current 25Q32 or 25Q40 chips shipped with new ESP8266 modules will be an essential component of the solution. And\u00a0a more robust API to control the watchdog timer is also needed. Application control of the timeout period as well as a user defined timeout callback will go a long way in resolving the issue\u00a0of unwanted resets.<\/p>\n<p>Hope you find this information useful.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u00a0The Project After my initial\u00a0tinkering with\u00a0the ESP8266, I could\u00a0visualize of lots\u00a0of practical applications. Suddenly, the price barrier was shattered. Every little thing can be connected. First up was the task &hellip;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1,10,19],"tags":[],"class_list":["post-345","post","type-post","status-publish","format-standard","hentry","category-alltheposts","category-esp8266","category-internet-of-things"],"_links":{"self":[{"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/posts\/345","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=345"}],"version-history":[{"count":55,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/posts\/345\/revisions"}],"predecessor-version":[{"id":898,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/posts\/345\/revisions\/898"}],"wp:attachment":[{"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=345"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=345"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=345"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}