ESP8266 and MQTT A Marriage Made in Heaven?

It’s been a very busy day – and a very frustrating day. I’ve had to make a start at figuring out how MAKEFILES work because of a fellow who implemented MQTT for ESP8266 so I could seamlessly compile his code and blow ESP-01 chips with the Eclipse environment in Windows.

It all started a couple of days ago when I utterly accidentally stumbled on a project to implement MQTT onto an ESP8266. I don’t know why I went for this but I’d just gotten to grips with the Eclipse setup so I was now a fully fledged Windows developer of ESP8266 software.. by that I mean I’d figured out how to alter code to get a light to flash on and off in the AT command software.

I remember playing with MQTT a while ago and I mentioned it in here or on the ESP8266WIFI Facebook page. Anyway I sat down with a large glass of wine and started to tinker. Essentially, MQTT is a lightweight protocol for sending messages around a network – apparently Facebook Messenger uses it.  You have a “broker” – call it a server if you are a techie… so this is free software like for example MOSQUITTO which you can run hosted, on your PC or on a Raspberry Pi. you can make an account with it – which might be open, have a simple password or have something far more complicated.

One of my fears of controlling the house remotely has always been security and I’ve made some primitive passwording which is really not worth a light.. so this caught my interest. My other issue as regular readers will know was that I was getting utterly hacked off with the ESP8266 as it can’t really do server and client at exactly the same time – so you would have trouble making a network of them – you CAN do it and there’s a slow moving project on the subject… but right now, not a viable solution – that might change.  The instant I saw MQTT my heart skipped a beat as I thought… can’t one device send a message to another?

I put IBM’s free MOSQUITTO on my PC (powers up with no passwords etc, doddle to use even if the command line environment makes some people feel physically sick – including me). IT uses a single port which you can change so it’s easy to make available outside via a little port mapping on your router.

I blew the code for this onto one of my ESP8266 boards – and left that connected to a serial monitor. Sure enough, it connected first time to MOSQUITTO. It SUBSCRIBED to a couple of messages (utterly arbitrary and they don’t need to exist at the time of subscription).

I grabbed a free but not very good program for Android called MyMQTT. IT’s not seen an update in AGES but I remembered it worked ok. I told it about my MOSQUITTO setup (ie the IP address of my computer) and that was that. I subscribed on the mobile APP to the same “topics” my ESP8266 had. I sent a message to that topic… sure enough instantly on the phone, the topic I’d replied to came back with a message – the same one I’d sent out… simultaneously the same message came into the ESP8266. We’re talking INSTANT here – no visible delays.

Immediately I grasped the possibilities. I modified the MQTT code (in the new sparkly ECLIPSE environment) on the ESP8266 to check the content of one of the topics it had subscribed to… a quick STRSTR on the zero-terminated incoming string… I made 2 comparisons – one for a message “GPIO0 ON” and another for “GPIO0 OFF”. I borrowed the heater and port info from a BLINKY sketch (all of 3 lines of code for everything)… and compiled away. I sat in the corner of the room and gingerly entered “GPIO ON” into my mobile phone app… and… BINGO the light came on.

Step 2. When the light comes on – SEND a message to the same topic but a different device number – I should say the author engineered auto device numbers based I think on the mac codes – they’re all unique…   /0033342342/data as it happens – I sent a hello.

Sat back with the phone, put my GPIO message in again, the light came on and IMMEDIATELY the message appeared on the SECOND ESP8266 as expected.

So with the minor inconvenience of a piece of “broker” software minding it’s own business on a minimal PC, a Raspberry Pi or some external server (your choice) the ESP8266 boards can now talk to each other without another processor. Not only that, I reasoned but if the server machine also had a client, it should publish a “TIME” message every 10 minutes or so…  the boards would IF APPROPRIATE be programmed to subscribe to this channel and LO  – ESP8266 boards which can turn things on and off – and also know what time it is. But what of the power of an Arduino – what about all those extra pins? Another project implements I2c on the ESP8266 and when my skills are ready I’m adding THAT into a mix – which means any number of peripherals to do things or return info using MQTT.

It’s a grand total of £40 or so investment to get yourself into the Android App trivial programming brigade using B4A.  Mine of out of date, out of warranty and so I’ll ahve to start again. For £10 a guy has written a library for MQTT and this afternoon on a friend’s PC – someone who has the full B4A, we wrote a simple messenger program that will send and receive MQTT messages. The results – again absolutely instant message passing back and forth.

Imagine – you have a button on the phone – you press it and it does two things.. it turns a little indicator called kitchen_lamp to grey then sends a message to /kitchen/lamp which it does not subscribe to – the message is ON… the kitchen lamp ESP8266 box which features nothing more than an ESP-01, a 3v mains PSU and a relay, gets the message and turns off GPIO0 which turns on the relay (there’s a good reason to do it backwards like that). It then sends a message back to another topic which it does not subscribe to called /kitchenresponse/lamp. THe phone however DOES respond to that – and when it sees an incoming message to that topic, it turns that same indicator green or red depending on the message.  So the lamp is off – you press the on button and your indicator on the phone almost immediately turns green so you know for a FACT that the kitchen light came on.

You might take that a stage further – because you’ve written the client on your PC to send messages like TIME, the kitchen lamp knows what time it is because it also subscribes to TIME.  if no-one turns that lamp off, at midnight it turns it off itself.

Multiply this by dozens of similar gadgets, some DO stuff like turning things on and off, others merely sit there, read the temperature and report it back – either every now and then – or on demand… same technique..

An old Android tablet sits on the wall, it has a program you wrote which subscribes to as many or little of the topics as you want and reports in pretty graphical format the various statuses-  it might even have a large round touch control reminiscent of a a certain apple-esque temperature controller…. and everyone talks to each other securely over WIFI..

BUT you might say, I really want to use Arduinos – ok you have the little ESP8266 do all the hard work of handling messages incoming messages it sends out of the serial port and if you want to send a message that could come in through the serial port… so now you have an Arduino that has to do nothing but take in serial messages and send them out – polling is easy – and you can of course wrap messages in any control character or sequence you want.

But hang on – isn’t Arduino serial interrupt driven? YOU BET – which means if you go in and tinker with that you could have that look for the starting sequence, start storing bytes until the ending sequence occurs, copy that temporary buffer to another, set a flag to say there is a massage – and simultaneously go back to looking for the next message – all using existing resources that are used in the serial handling anyway.  The Arduino ends up with a few bytes less than a full pack and can now receive serial messages and act on them with virtually no extra overhead which means the simple 328 has LOTS of room to DO stuff.

That enough for now? It certainly has me excited but I’ve had enough for one day… I need to turn most of this from thought processes to reality. The MQTT project is apparently in it’s early stages but I can assure you it sends messages with password protection as reliably as it receives them. Even if the designer does nothing else you have this at your disposal – links in earlier entries in this blog. Go get ‘em.

Now if anyone’s up for some collaboration I’d much rather someone else write the Android APP – a decent replacement for the promising but dead in the water myMQTT..  All in good time – good night all.