MQTT and Arduino

Today I had my pal Jonathan over and we took a diversion from my current network on NETIO and Arduino-based home control – to look at something called MQTT which is a simple protocol for sending messages back and forth across the web from and to the likes of Arduino.

I say diversion because we came across some brilliant software which I’ll describe – but the documentation out there is designed for nerds – it is certainly not designed for real people with other things to do and the services are not entirely free so this certainly won’t replace what I’m working on – but for some monitoring and control purposes where COST is an issue – read on…

So – let’s get straight to it – you have an Arduino and you want to send commands to it over the Internet – or you may want IT to send out, say sensor data. You want to be able to control this remotely or monitor remotely – and you may want to LOG the data.

On the surface of it – easy – get an ETHERNET card for your Arduino, dump some software in there – you’re up and running. But serving web pages on an Arduino is VERY limited as is space for storage – by the time you add in SD card software you’re onto the next level up of board to get more memory etc etc… not necessary – read on.

To send info via MQTT and have it logged somewhere, you need an MQTT service – and this is where it gets complicated – some are free – all the ones we found come with severe service limitations unless you cough up. Can anyone show us otherwise? The service acts as a broker between the data you want to send – and wherever you want to send it – logging it in the process. This is all sub-second turnaround ideal for interactive stuff.

So for the purposes of explanation and testing, we found a free MQTT service that runs on your computer. This is ideal for testing but clearly a high resiliance external service would be better as for this you need a computer permanently connected to the Internet and left running.

It is called MOSQUITTO. Yes, the TT is correct. When you install Mosquitto on a PC, you run the Mosquitto.exe and it pops up a black box. That’s it running. It can also run as a service.

The idea being that your Arduino talks to this service – and subscribes to various topics (you make the names up and they are unique to your account – which is generally free but you’ll pay to connect more than a few devices and more than a bit of data. You can SEND some info to a topic or retrieve info from a topic. Elsewhere, a PC client or Android client may also be connected to Mosquitto and send information to the same topic (in which case the subscribing Arduino will receive that info) or simply poll the topic – in which case it will see the info the Arduino is putting out.

Think of it as a BROKER – an intermediate service.  You can add security – you can add encrypting and you can LOGS… taking most of the work away from the Arduino. Sadly the software was originally written for Linux and could we HELL figure out how to turn the log on. Advice welcome.

Anyway, here’s the link. http://mosquitto.org/

So you have that window open and running on your PC – that’s the service running – runs out of the box  – you open another window…. and run mosquitto_sub.exe -t test   – that window is now subscribed to a topic TEST – and will wait to show you anything coming in… your Arduino can publish to TEST and you’ll see the results on your PC.

image

Here you see after subscribing to TEST  (-t is TOPIC) the phrase “Hello World again!” is coming every few seconds from the Arduino.

There is another program in the package called mosquitto_pub.exe – and it can publish

image

In this example assuming the service is at 192.168.0.17, it publishes “ooh” to topic test and sure enough the other window sees the output.

There is a client for Android called MYMQTT but be warned it drops connections and is not very good – but once you understand it – it’s ok for testing. For PROCESSING environment there is some stuff out there but I’ve yet to figure out how to use it.

So – how do we make use of this on the Arduino…. we found code for the WIZNET card but to end up paying probably £20-40 for a temperature monitor – erm, no…

So we set about looking for some code to run on the ENC28J60 cards as they are CHEAP. We found that there is a library which relies on the ETHERNET drivers for NANODE which uses the ENC chip – but in fact none of it relies on the NANODE – it will work with a normal Arduino or clone…. apart from the MAC address setup – which it turns out you can just do manually. See code.

So you need as well as your normal Arduino, the ENC board (pin 8 for CE),  this https://github.com/sde1000/NanodeUIP and this http://knolleary.net/arduino-client-for-mqtt/

So – off we went to try that – super – except the example would not start reliably – we figured it was something to do with the ENC board reset and went about putting in manual resets and delays – turns out nothing of the sort – the client for MQTT connection software is just maybe a little lacking in checking that it is actually connected. Also – we found once it was all working that if you temporarily lost your connection or the MOSQUITTO service stopped and then restarted – you were stuffed. So the normal initialisation you might find in SETUP we brought into the main loop with a check in the loop to see if you’re connected and if not – connect – and retry until you do.  Result – up to now utterly perfect.

Note the topic “test” can be anything – in my simple example I can turn a light on but then sending a message kills it – read the code – it will be obvious which in short order… Clearly you can’t send binary zeros in this so you need to ascii encode any binary stuff – and the limit in size thanks to the ENC chip is 127 characters. you can control a LOT of inputs and outputs with that much message!!

Here’s the setup for this test using a topic arbitrarily called “test” – the ENC board uses pin 8 for chip select… connect power (3v3 – I use 2 dropper diodes from 5v), ground, SI,SO, SCK, CS and reset.

image

There’s a lot of serial stuff in there for testing only….note the callback routine that handles incoming packages…. and then below,  the loop itself – including making the connection (and retrying immediately or after a failure)

image

Ok so why bother with any of this – well, it’s CHEAP – you can make a board that will say return temperature to you and let you control bits and bobs for under a tenner if you make it yourself or get an Arduino clone and ENC board from China…. and if you need to log – well that’s handled by the service – I’ll leave you to investigate other services out there but really, testing with a local service is the easiest start.