Logging the Internet of Things

Updated 18/9/2014: I’ve been making gadgets for “the Internet of Things” since before the phrase was invented.. one of the issues I’ve always had was displaying data. Oh sure you  can set up a website and talk to it but now you have to maintain that site etc and one-man-band graphics are rarely up with the best.

I’ve been looking lately at services that will consume data from my gadgets and display it on mobile phones, PC, laptop etc.

Oh sure you could also send the data serially to your PC and use the power of that machine to process it – but ideally one would want to have each gadget send regular updates all on it’s own. And thanks to the services appearing out there and Arduino – now you can.

This is where I’m up to – having dismissed several blind alleys…(and thanks to all of you for the feedback).

So I have a controller in the house that talks to various sub-controllers – collates the information and makes it available to me on my phone – temperature, humidity etc and lets me control lights etc… what’s missing is logging – the chips I use don’t have a lot of storage and getting that out onto the Internet at speed would not be trivial even if they did – and I can’t be leaving the phone on 24/7 to log stuff – so instead I’ve been looking for services whereby I can send data out at regular intervals, a small package at a time – and have some server somewhere process that data.

For my first stop we’ll look at Grovestreams – https://www.grovestreams.com

Here you use a simple Ethernet client on the Arduino or similar and send a PUT request with some data – seriously – a simple command-line job on a PC which is hardly more complicated on an embedded machine. I send data every 10 minutes on internal, external temperatures and overall humidity from our home in Spain. Been doing that for weeks now. The result?


As you can see, it’s all been operating flawlessly since the back end of August – when we left it was dry and very hot – today the humidity is a lot higher (green), internal temperatures slowly falling and external temperatures going as low as 14c.  All of that on a free service (as long as you don’t overdo it – check their site for cost details).

What’s nice is the web-based zoom – see the last couple of days above – and the sliders the bottom of the image.. Also they handle the averaging and there are tons of options.


Neat – so next I tried Carriots – https://cpanel.carriots.com  and this has a free option – I’ve yet to get that working but the day I asked for help they came straight back to confirm they’d tried my code and it works – so I’m sure it’s something minor – more on that later but the site is worth a look.

Right now I’m playing with http://dweet.io and http://freeboard.io

So Dweet is a service that again your micro can call directly without major overhead – it will store your data for up to a day and is an intermediate service. Freeboard consumes that data and…..


So this is not in the same league as Grovestreams BUT it’s a dashboard… imagine you had a bunch of gadgets all connected to the Internet and you wanted to see data from them all on one screen – if that’s the case then Freeboard and Carriots might be for you.

And now onto EXOSITES (www.exosites.com) – they have a very nice interface – when I originally wrote this article I was having lots of problems connecting to them – their examples for Arduino used STRINGS which I’ve never trusted (it is SAID that in Arduino string library for 1.5 (beta) they are improved but for now I’m sticking with the official release) and their examples would not send data to their server reliably.  I wrote to technical support – who agreed with me about strings – and they’d also made available a version using character arrays to send string data. I tried that, it still didn’t work – however I have to say after a series of email conversations, last night they put up a version of their code that works! It has the STRANGEST issue in that sending data to them fails every 100th attempt – at this point I’ve no idea if it is their end or mine but I’m more than happy with the technical support up to now and I’ve just sent the entire evening’s logs off to them to look at – with absolutely reliable failure every 100th send… here’s the current library. https://github.com/exosite-garage/arduino_exosite_library

So – what do you do and what do you get with Exosites-  they provide a REALLY simple to use library – basically you have a KEY  – and you send data sets along the lines of X=Y – that’s it really. A mere few lines of simple code – it’s all done in the library. It’s fast and efficient and you get back a copy of what you sent for confirmation!


The graphs are not in the same league as Grovestreams nor is the control – BUT there again they don’t put a limit on the amount of data in the free account and the dashboard is nice – and works on mobile phones and tablets – you’ll need to decide for yourself what is best for you. Right now I have high hopes that last tiny niggle of data failing will be resolved and I’ll be more than happy with this.

More as I go on. Someone suggested this morning I take a look at PLOTLY.. at first sight I could not make head nor tail of this – it’s in BETA and no-where near as simple or straightforward as above – but you might want to take a look. https://plot.ly

Oh, along the way, another kind co-respondent pointed out that one of my images (I think Freeboards) uses something called high-charts… now, if you need charts for personal use – this looks THE BUSINESS – http://www.highcharts.com/products/highcharts I had a quick look at you download the library with a TON of examples.. and they really look GOOD. Indeed – all I needed was for some kind soul to put Highcharts together with an Arduino – and SD card.. and VOILA.. http://everettsprojects.com/2012/12/31/arduino-super-graphing-data-logger/  I MUST STRESS I have not tested this (give it time) – I have the sketch in front of me – it looks reasonable – but that doesn’t mean it will work.