MQTT for ESP8266 not QUITE but nearly

Minh Tuan is doing an excellent job with MQTT – he has it working stand alone – and in the LUA code (not yet brought into the main code)… but..

Before you go off getting annoyed if things don’t go quite to plan, there still seems to be a tiny but important bug  -I’ve reported it today and Minh will look at it tomorrow. I’m working on code from just an hour ago here (late after noon UK)

I set up a demo, subscribing to 7 short topics. With the exception of the first, all the topic names are short… and all responses are short…. I’ve changed the broker name below to protect the innocent (that would be me).

MQTT: subscribe, topic"home/openHAB/out/Light_FF_Bath_Ceiling/command" at broker xxx.yyy.zzz:1884
MQTT: subscribe, topic"myleds" at broker  xxx.yyy.zzz:1884
MQTT: subscribe, topic"mytime" at broker
MQTT: subscribe, topic"time" at broker xxx.yyy.zzz:1884
MQTT: subscribe, topic"dusk" at broker xxx.yyy.zzz:1884
MQTT: subscribe, topic"dawn" at broker hoxxx.yyy.zzz:1884
MQTT: subscribe, topic"timestring" at broker xxx.yyy.zzz:1884

As you can see all of them subscribed successfully or so it would seem.

Ah, but…

4 of them are broadcast every minute from a page I demonstrated in a previous blog or it’s notes and over on another computer I have MQTT spy watching… time, dawn, dusk and timestring are send out by the same page once per minute.

But what is being seen by the little ESP8266 program is….

TCP: data received
MQTT topic: time, data: {1484956800}
TCP: data received
MQTT topic: timestring, data: {15:15 Saturday 17-01-15}
TCP: data received
MQTT topic: time, data: {1484956800}
TCP: data received
MQTT topic: timestring, data: {15:16 Saturday 17-01-15}

As you can see, dawn and dusk have “disappeared into the dusk”.

Similar happens with the Lua/MQTT code – I thought at first that might be due to the callback not being done before being called again but seeing this I’m thinking… no.

As soon as this one is resolved I’ll report back. It is important that the system can handle several simultaneous incoming messages and that it it reaches it’s limit, something obvious tells us that.   The info above is the minimum I would be sending to any module –ok of course I could merge them into one – but that’s not the point – I could not do that if they were coming from different sources.

Suggest looking in again tomorrow for any fixes…

If I go quiet, I’m off to Brussels for a couple of days for some EU business.

And there’s more… I reduced the number of subscriptions to 3..

MQTT: subscribe, topic"dusk" at broker xxx.yyy.zzz:1884
MQTT: subscribe, topic"dawn" at broker xxx.yyy.zzz:1884
MQTT: subscribe, topic"timestring" at broker
MQTT: Sending..type: 8,qos: 4
TCP: Sent
TCP: data received
MQTT: Sending..type: 8,qos: 4
TCP: Sent
TCP: data received
MQTT: Sending..type: 8,qos: 4
TCP: Sent
TCP: data received
TCP: data received
MQTT topic: timestring, data: {16:43 Saturday 17-01-15}
TCP: data received
MQTT topic: dawn, data: {27208}
TCP: data received
MQTT topic: timestring, data: {16:44 Saturday 17-01-15}
TCP: data received


Couple of things of concern – firstly again it’s SAYING it has subscribed to 3 but it only taking two of them – the last 2 (that’s not a clue, wasn’t last 2 last time) but the messages are also a concern… “Sending..type:8 qos:4”   there ISN’T a qos 4 !!! Don’t know what that is about…

24 thoughts on “MQTT for ESP8266 not QUITE but nearly

  1. Hi Peter

    I have loaded latest esp_mqtt into Eclipse. It builds OK, but when I come to flash the device I get “Makefile:144: recipe for target ‘flash’ failed”

    I have the correct COM port selected in the makefile and the device flashes OK with hello_word etc.

    Have you come across such a problem?


    • Compare the working make file from hello_world with the make file from esp_mqtt; that should highlight the differences.

      Myself, i like to separate the build from the flash step, so I use .bat files to flash. Below is my file ‘flashme.bat’ inside the esp_mqtt directory. I hardcoded my com port into Ensure the following is all on one line.

      C:\Espressif\utils\ write_flash 0x00000 firmware/0x00000.bin 0x3C000 ../../ESP8266_SDK/bin/blank.bin 0x40000 firmware/0x40000.bin

      • Ken

        I have just realised that the firmware is being created in my git\esp_mqtt folder and not the espressif\examples folder.

        I assume I must have missed a step when I imported esp_mqtt into Eclipse! I wonder what!

  2. Travelling for a couple of days so no compiling for me. Just had an email from Minh… He reckons he has fixed Mqtt. What service! Best grab latest code from his github. Link elsewhere (I’m doing this on a mobile phone). High hopes for midweek compile….

    • I’ve tested the latest, and it works fine for me 😀 subscribing and publishing to multiple channels.

      One thing I did have an issue with is the deep sleep, I originally had it in the mqttPublishedCb as a direct call, but this proved unreliable with wdt resets. I ended up doing the following.

      int pubCount = 0;

      LOCAL void ICACHE_FLASH_ATTR rtc_cb(void *arg)

      void mqttConnectedCb(uint32_t *args)
      MQTT_Client* client = (MQTT_Client*)args;
      INFO(“MQTT: Connected\r\n”);
      //MQTT_Subscribe(client, “/mqtt/topic/0”, 0);

      MQTT_Publish(client, “/mqtt/topic/0”, “Temp:0”, 6, 0, 0);
      MQTT_Publish(client, “/mqtt/topic/1”, “Humidity:0”, 10, 0, 0);
      MQTT_Publish(client, “/mqtt/topic/2”, “Battery:0”, 9, 0, 0);

      void mqttPublishedCb(uint32_t *args)
      MQTT_Client* client = (MQTT_Client*)args;
      INFO(“MQTT: Published\r\n”);
      if(pubCount == 3)
      os_timer_setfn(&rtc_timer, (os_timer_func_t *)rtc_cb, (void *)0);
      os_timer_arm(&rtc_timer, 1000, 0);

      and in user_init

      pubCount = 0;

      Obviously I don’t have the dht22 code in yet, but the 3 messages are sent every 20 seconds (currently) and all get through without any issues. The device drops into sleep mode in between sends to save battery power.

      I have soldered the wire into the esp-01 to allow waking from deep sleep.


      • Hi Glen

        My issue and I think it is fixed but can’t test.. I have several sub’s which come in at exactly the same time. ie from the same source. I guess this simulates asynchronous messages which happen to come in at the same instant. Minh has access to my setup and wrote to say it is fixed. I hope to test Tuesday night. Things yet to consider… With a limited HARDWARE buffer will doing things like sending serial bursts to WS2812B LEDs which of necessity must be done with interrupts temporarily off, adversely affect the incoming MQTT stream and will QOS 2 be a way around that…OR do we resign ourselves to doing intense time sensitive stuff like that with an external processor such as an Arduino.

      • Hi Interesting code I must try on my return!!!! I would expect your messages to go out with sleep…. That is useful… But do you know if QOS2 messages come IN reliably after sleep. That would be VERY useful to know. Thanks for sharing. Also have you done the maths…. With your big sleep what is your average current overall??


  3. As of yesterday, we now have an MQTT debug terminal available, courtesy of “EAD Fritz”:-

    You’re not going to get far with this one with your ESP-01’s Pete, but 03’s and 12’s should be okay.
    The project is an ESP8266 with a Nokia-5110 display hooked up to the GPIO pins and MQTT running on it. Interesting use of “subtrees” on Github to bring together two separate projects, too.


    • I am using i2c display with mqtt following Nathan’s guide here:-

      • Tony,
        Thanks for that reminder. I’ve actually done Nathan a double injustice, not only forgetting
        to mention his project first, but also for being cheeky enough to have used a git clone
        of his code since (according to the timestamp on the files) January the 2nd, as the base
        building block for my own (pathetic) experiments. In my own defence I have to say that,
        being of exactly the same vintage as our host, Pete, my eyes can’t cope with those tiny
        OLED displays any longer and I don’t have any in my junkbox (whereas I do have a
        couple of the Nokia 5110’s).


  4. Please don’t forget to try something with Arduino (and not using the esp8266)!
    I understood (from a previous post of you) that we can wrap the MQTT message in special characters like { } and put that into the Serial (how we can do that with the esp8266? :D) but the hardest thing is subscribe and unsubscribe topics from Arduino and let Arduino send messaged to other MQTT devices…

    So, maybe in the future, a full esp8266 + Arduino demo would be very very useful to me! Thanks! 🙂

  5. In the ESP8266 output before it goes to serial you can wrap message in anytihng you want – makes it easier to keep useful debug output while making life easier for Arduino wanting to use info. For me to – I want the Arduino to be able to tell the ESP8266 about AP, about the broker and about which subjects to subscribe to… so I don’t have to keep recompiling the ESP8266 which I see as a communication black box. But last thing to do is test Tuans latest fixes – once it all WORKS then we look at how to use serial both ways rather than having fixed ip and broker programmed into the ESP8266… once step at a time 🙂

    • So, i successfully managed to get arduino send and receive mqtt messages through the esp8266!
      Now I really miss the encryption feature of the mqtt protocol…do you you know if the nodemcu is going to get this feature soon?
      Since you know the developer you could ask him 😛

      • Use esp8266 you can use https: and username and password… and pass info back to Arduino – right now what you cannot do with this setup is have that security and pass instructions from Arduino to ESP8266 – HOWEVER as soon as I’m happy the ESP8266 software if near enough perfect- the next stage is to encourage people to work on rotary buffer serial in and use that to send control messages to the ESP*266 to implement change of Access point, Broker, subscribes etc…

  6. Could anyone provide some info on how to import this into the Windows Eclipse SDK? I have successfully opened other projects but having no luck with this, just flashing the bins for now. A link to a converted project (zip) would be great, but some pointers will also do ;).
    Many thanks, and keep blogging Peter!

    • Aaron… Take Tuans code – dump the directory in espressif\examples… import – change the port – that’s it – needs no customisation at all to fit into the Windows Eclipse setup – that’s where I program mine all the time (don’t ask for practical demo – I’m overseas for a couple of days).

  7. Well my eyes still work… I too find the oled displays a little tiny and the ones I really love are the 160*240 full colour LCD ones, dirt cheap from Bangood in China and needing 5 control lines. I’ve seen nothing for ESP8266 yet which is a shame as it is potentially a much faster drive solution than Arduino.

  8. Incidentally I can confirm the issue with multiple subscriptions is history. I left the unit on not only grabbing 6 subscriptions 4 of which come in simultaneously but also serving dht22 temperature and humidity on deman and 24 hours later it is working a treat. Next stop 1wire reading as I need to bury a temperature sensor in a cave wall.

Leave a Reply to Scargill Cancel reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s