{"id":777,"date":"2015-08-17T16:08:16","date_gmt":"2015-08-17T23:08:16","guid":{"rendered":"http:\/\/internetofhomethings.com\/homethings\/?p=777"},"modified":"2015-09-16T07:35:46","modified_gmt":"2015-09-16T14:35:46","slug":"mobile-web-sensors-using-esp8266-phase-3-google-map-and-excel-visualizations","status":"publish","type":"post","link":"https:\/\/internetofhomethings.com\/homethings\/?p=777","title":{"rendered":"Mobile Web Sensors Using ESP8266 – Phase 3: Google Map Visualizations"},"content":{"rendered":"

This is a continuation of the “Mobile Web Sensors Using ESP8266” series.<\/p>\n

Mobile Web Sensors Using ESP8266 – Phase 1<\/a>
\nMobile Web Sensors Using ESP8266 – Phase\u00a02<\/a><\/p>\n

\"\"<\/a><\/h3>\n

Google Map and Excel Visualizations<\/strong><\/h3>\n

This is where things get interesting. I just got back from a yoga class. This was a simple 4 mile bicycle trek across town; a perfect opportunity to collect some data using the Mobile Sensor System.<\/p>\n

For this test run. a USB battery was used to power the ESP8266-12 based device. I simply plugged a USB cable in to powered it up, stuffed it into a fanny pack, connected my Android phone to the device’s\u00a0WIFI signal and started the App.<\/p>\n

I could tell it was working by observing the UTC time change rapidly on the phone app’s GUI. Another look at the display upon arrival at my destination confirmed that the data collection was still \u00a0running. Success!<\/p>\n

But the raw data file was not in a very human readable form:<\/p>\n

timestamp,lat,lon,spd,alt,btmp,dtmp,bp,dh,syst,sysh,syslp\r\n1439830613611,34.2943768,-118.8439855,null,null,80.6,57.79,29.23,50.70,120,30840,37\r\n1439830614632,34.2943768,-118.8439855,null,null,80.6,57.90,29.23,50.59,121,30840,38\r\n1439830615646,34.2943715,-118.8439865,null,null,80.24,57.90,29.23,50.59,122,31280,39\r\n1439830618611,34.2945713,-118.8441038,0,180,80.24,57.90,29.23,50.50,125,30832,42\r\n1439830619607,34.2945664,-118.8441046,0,180,80.24,57.90,29.23,50.50,126,31280,43\r\n1439830621616,34.2945921,-118.8441143,0,181,80.42,57.90,29.24,50.50,128,30840,45\r\n1439830625549,34.2945638,-118.8440880,0,175,80.42,57.90,29.23,50.40,132,30840,49\r\n1439830625649,34.2945638,-118.8440880,0,175,80.42,57.90,29.23,50.40,132,30840,49\r\n1439830627653,34.2945510,-118.8440810,0,175,80.60,57.90,29.23,50.40,134,31280,51\r\n1439830630563,34.2945510,-118.8440807,0,175,80.60,58.0,29.23,50.79,136,31280,53\r\n1439830631631,34.2945508,-118.8440805,0,175,80.60,58.0,29.23,50.70,138,29112,55\r\n<\/pre>\n
Collecting this sensor data and saving it to a file\u00a0was an important step. And now, the next thing to do was\u00a0to present the data in a visually useful format. I had intended to import the CSV file containing the data into Excel. However, while creating custom Excel spreadsheet charts and graphs was\u00a0a\u00a0quick and simple method of visualizing the data, my first approach was to seek out possible\u00a0on-line tools already available\u00a0for this purpose.<\/p>\n
So the search began. It did not take long to find a way to import the spreadsheet into google maps. But would it be simple? would I have to crop out the latitude and longitude values from my data file for the mapping program to use it?<\/p>\n
I was\u00a0surprised at how easy it was…<\/p>\n
The web site was\u00a0www.gpsvisualizer.com<\/a>.<\/p>\n
After opening a web browser to that site, you simply click on “Choose File” and select the csv file created from the Android App.<\/p>\n
Then, just click on “Go” and a satellite map with the tracks from the data appears.<\/p>\n
\"Googlemap2\"<\/a><\/p>\n
No changes to the csv file were needed. What a great tool! It figured out the appropriate data fields for lat & long and overlay-ed them on the map with a red trace.<\/p>\n
Okay, what other options are available with this “GPS Visualizer”?<\/p>\n
\"gpsvisualizer1\"<\/a><\/p>\n
The “Choose an output format” revealed several image options as well as an elevation profile. The elevation profile looked like it might be interesting…<\/p>\n
But I noticed the default units were meters. Being a US resident, feet are preferred. These units, along with many other options, were select-able by clicking on the link “Profiles (elevation,etc.)”.<\/p>\n
\"Googlemap2-3\"<\/a><\/p>\n
Here, you could set the X and Y Axis plots to any of the fields shown above. I was stoked to see the list included “Heart rate”. My Heart rate sensor will be added to this project at some point soon. Plotting it against elevation change should present some interesting results.<\/p>\n
I created a test plot of elevation over distance traveled along this 4-mile route. While any field could be selected, this sample plot colorizes the speed. Again, the plot shows the speed to generally stay below 20 mph, typical for city street bicycling.<\/p>\n
\"Googlemap2-4\"<\/a><\/p>\n
That steep slope (and slow speed–red plot color) at about the 3.5 mile distance correlates with the only significant hill climb along the route. Nice to confirm that the data collected makes sense!<\/p>\n
But then it hit me. Nice to see that this “GPS Visualizer” can portray the GPS data in charts…but what about the customized sensors? Like temperature, barometric pressure, and the relative humidity? Could this tool also support these unique fields in the plots?<\/p>\n
I was pleased to discover that the answer is…<\/p>\n
YES!<\/p>\n
Note that the last field in the drop-down above is “custom”. Selecting this option allows you to enter any field name. Those are the field names in the first row of the CSV file of the data captured with our mobile sensor system. I selected “humidity” for the y-axis as a test case with a colorization of the speed.<\/p>\n
\"Googlemap2-5\"<\/a>
\nI was surprised to see the drop in relative humidity from the start to the finish of the ride. It was probably due to the sensor drying out someone with motion. It had been in the controlled, air conditioned environment of the yoga studio at the start and was then exposed to our dry Southern California air.<\/p>\n
Looks like this “GPS Visualizer” tool will come in handy to produce quick charts from the sensor data.<\/p>\n
Visualizations Using Excel<\/strong><\/p>\n
While most everyone if familiar with excel, this discussion would not be complete without a few chart example produced from the raw CSV file.<\/p>\n
But when the CSV file was opened using Excel, a couple of issues had to be addressed first before plotting the data.<\/p>\n
    \n
  1. The timestamp field was shown in scientific format. this was corrected by changing the format from “General” to “Number” with 0 decimal places.<\/li>\n
  2. The first 3 entries for the spd and alt fields were “null”. These were changed to the value of the 4th entry, the first one with valid values. Note: these values came from the GPS API in the Android code. Neither one accurately represents the actual speed and altitude. Yet the google plots were accurate.\u00a0This problem was due to assuming the parameter units incorrectly:\n
      \n
    1. The altitude<\/strong> was reported in meters. Since we were looking for feet, it must be converted using the scale factor
      \nof\u00a03.28084 feet\/meter. This has now been corrected in the Android App code.<\/li>\n
    2. The speed<\/strong> is reported in meters per second.\u00a0Since we were looking for MPH, it must be converted using the scale factor\u00a0of 2.23694 miles\/hour per meter\/second. This has now been corrected in the Android App code.<\/li>\n
    3. For the evaluation of the data collected in excel, two columns were added to perform the conversion: MPH and altitude.Problem with the speed, however, is that many of the samples returned 0 meters\/sec as the speed. To solve this, I may introduce a new speed algorithm to the mobile app code to convert two GPS coordinates to distance. This distance, along with the change in timestamp values would be used to\u00a0calculate speed.<\/li>\n<\/ol>\n<\/li>\n
    4. The “dtmp” values (temperature as read by the humidity sensor) are incorrect. The temperature starts at 57.79 degrees while the temperature read by the barometric sensor starts at 80.6. It looked to me like the conversion is missing the “plus 32” in the ESP8266 code. But a review of the code did not reveal any problems. The conversion did include the required “plus 32). I would like to hear from anyone else using the DH22 to see if they are all off in temperature or it is just my sensor that is incorrect. For now, the DH22 temperature reading will be discarded as invalid.<\/li>\n<\/ol>\n
      The first thing I tried was to plot the coordinates over the ESP system time(seconds since reset).<\/p>\n
      \"excel1\"<\/a><\/p>\n
      Problem with this plot is the relatively small change in latitude and longitude. It essentially looks like a straight line. This was easily rectified by plotting the\u00a0latitude and longitude separately.<\/p>\n
      \"exce2.\"<\/a>
      \nOkay, and now let’s plot the elevation again, like we did with the GPS visualizer.<\/p>\n
      \"exce3\"<\/a>
      \nYes, this does look the same as the GPS Visualizer. Except colorizing the plot with a 3rd parameter is not an option. Even with just these few sample plots, I am definitely liking the visualizer of standard Excel charts.<\/p>\n
      Conclusion<\/strong><\/h3>\n
      As you can see, there is more than one option available to visually represent data collected with IoT sensors. And it is worthwhile to investigate tools that may be readily available before attempting to build your own charts, from scratch. Here we just explored a fraction of what is available for this purpose.<\/p>\n
      Phase 4: Mobile App Gadgets is next. In that post,\u00a0I am going to add some visualizations to the Android application. This will add a real-time visual element to the crude data windows created in phase 1.<\/p>\n
      I hope you find this information useful…<\/p>\n","protected":false},"excerpt":{"rendered":"
      This is a continuation of the “Mobile Web Sensors Using ESP8266” series. Mobile Web Sensors Using ESP8266 – Phase 1 Mobile Web Sensors Using ESP8266 – Phase\u00a02 Google Map and …<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1,10,19],"tags":[58],"class_list":["post-777","post","type-post","status-publish","format-standard","hentry","category-alltheposts","category-esp8266","category-internet-of-things","tag-iot-data-visualizations"],"_links":{"self":[{"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/posts\/777","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=777"}],"version-history":[{"count":31,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/posts\/777\/revisions"}],"predecessor-version":[{"id":925,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=\/wp\/v2\/posts\/777\/revisions\/925"}],"wp:attachment":[{"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=777"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=777"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/internetofhomethings.com\/homethings\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=777"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}