NODE-RED FUN

This is fun – it’s a diversion but it’s fun – no… REALLY. Last night I was having a play with OpenHab.  I have all my own home control code and several working installations – but it never hurts to have a look and see what others are up to – especially having now pretty much decided to go down the MQTT route thanks purely to the fact that it works so well – and with passwords, in the ESP8266 hence solving a long standing security worry for me. I’m usually fairly good at spotting trends and as long as sending and retrieving MQTT OpenHabmessages in the ESP8266 nodes can be made rock-solid reliable – I cannot see any reason NOT to use the protocol – no worries about networks etc, just have the board respond to incoming topics and send out to reply topics as and when needed.   I can see someone coming up with a Windows interface for this (setting upt the router, setting up incoming topics etc  – and becoming a really popular person.

Anyway, I’ve been trying to get to grips with Openhab – trust me – if you’re new to it – you are going to need LOTS and LOTS of coffee.

The upshot is – it is possible despite the many many interfaces that OpenHab supports, to put 3 lines of code in there and have it send all it’s stuff to MQTTand respond accordingly.  So looking at the screen above, I can turn on the bathroom mirror (that’s the demo almost unaltered – I didn’t write this) and that will send an MQTT message out.  I have my little board responding to that. I can even have the board send a response BACK to say what the status of the mirror is (ON or OFF)  (I have no idea how to make that response happen on request – i.e. when the web page first opens).

ANYWAY, my little ESP8266 board can make the yellow bulb turn on and off but the blue switch sadly toggles regardless of any feedback – making you feel you’ve done something when in fact your little board could be in flames….. hopefully someone will help me there PLEASE. I turned off the yellow bulb automatic update but I can’t stop the blue button toggling yet.

SO – I got up early this morning and was waiting for someone to help me with my little Openhab issue when a fellow wrote in to suggest I have a look at CODE-RED. Well, what a fiasco – there’s a lot of LINUX-ASSUMPTION in the setup – just install it – no problem they say!  Well also install NODE which to be fair is also easy… but when the installation cran.. I ended up with a TON of guff coming up and telling me I needed to install VISUAL STUDIO…. I was THAT far off scrapping the idea but I thought I’d give it a go and despite the warnings it seem to work.

So what’s NODE-RED – I have NO idea if it is of any use but I CAN tell you it’s a HELL of a lot of fun – I mean – you HAVE to try this.

node-redSo if you’re bored and want some fun…  go here and get node.js -  installation on Windows is a breeze.  NOW having done that,  go off and get the NODE-RED installation.. http://nodered.org/  -  I just dumped the directory on my desktop.  Open a DOS BOX (they don’t tell you that) and navigate to the new node-redxxxx directory in your dos box.  Now type npm install –production

Ok now in the same DOS BOX – type node red.js and this still start up red.js – it will also likely complain that it has failed to register 9 node types – I have NO IDEA why a fresh installation would do this – and at that point I was ready to throw in the towel  -but it works anyway!!! If anyone has a kiddies guide to fixing this please do write in.

So – you will get to see something like this…

C:\Users\Peter\Desktop\node-red-0.9.1>node red.js

Welcome to Node-RED
===================

9 Jan 07:05:17 – [red] Version: 0.9.1
9 Jan 07:05:17 – [red] Loading palette nodes
9 Jan 07:05:17 – ——————————————
9 Jan 07:05:17 – [red] Failed to register 9 node types
9 Jan 07:05:17 – [red] Run with -v for details
9 Jan 07:05:17 – ——————————————
9 Jan 07:05:17 – [red] Server now running at http://127.0.0.1:1880/
9 Jan 07:05:17 – [red] Loading flows : flows_Monster.json
9 Jan 07:05:17 – [red] Starting flows

It might now look exactly like above….

I left the DOS box open – and pulled up my (Chrome) browser and ran the address and port provided.

A VERY pretty screen came up..

node-red[6]

Actually if you don’t get this far, go back, to a little reading – I might’ve missed something – it really is worth getting this far..

You back? Right – see those buttons on the left – which kind of make you feel this could be tantalisingly easy? Well, it IS. AT this point I have to make some assumptions – you have access to an MQTT server somewhere – and you have a Google email address – and a Twitter account or some of that.

So here’s what I did – I dropped an MQTT input and a DEBUG output on screen – drag and drop. I drew a line connecting them.

mqtt

I double-clicked the MQTT input (see it has the speaker-thing on the left as against the right) – and filled in the name of a subscription.. and my MQTT broker details.  I pressed DEPLOY.

Excited yet?

mqtt-debug

Using MQTT-SPY I sent a message to /test/ and …

mqtt-test2

By this point my toes were tingling… I wonder if I can have it send a message back as well?

mqtt-reply3

Yup… but.. but… but… how about having tweets come in and go to the debug AND my /testreply/ topic? I eagerly dropped the TWITTER box in and authorised it (double-click – absolute doddle) – I put in some hashtags (without the tags it seems)…

twitter

 

and drew a couple of lines – hit deploy and…

twitter mqtt

Unbelievably, tweets started coming into the debug window – AND going out to my /testreply/ – no problem.

Let’s try an email I thought – could it send me an email as well – and what about the contents of a test button going to all 3!!

test

Sure enough… I pressed the test button – the debug showed the message – my MQTT /testreply monitor saw the same thing and an email came into my inbox.

That’s it – I’m going to leave the rest to you… the visual interface (browser) does not need to be running for this to work (well apart from the test button) – and I’m not sure what kind of kit you need to run to keep it going either – or how it would respond to power loss etc, but at least ON THE SURFACE OF IT this as something special! I messed around for DAYS  trying to get a crappy Arduino to send Tweets only to discover most of the code out there is for BEFORE Twitter changed the API – and here – I can have a £2 ESP-01 send tweets AND emails without any effort at all!!!

Back to my Openhab work for an hour then I have some real work to do.

Incidentally, I have a SYNOLOGY disk-station and thanks to helpful people in here I now have that running MOSQUITTO for the MQTT server and thanks to the incredible help from the fellow who put together MQTT-SPY and made loads of changes for me – I can keep a test rig open to test my MQTT. I’ve even sent 1000 messages a second to the broker and had them come back in within a second – so there is no doubt in my mind that this is the correct route … it’s looking more and more like there are a ton of useful toys like OpenHab and Node-red which will help make this a very satisfying project. It looks like I could put Openhab into the Diskstation as well (in case you don’t know the Synology Diskstation is one of these black boxes with 2 disks for storing your media – because they use RAID they are almost immune to failure (one disk fails the other takes over) and unlike the READYNAS before them – they have some power).  IDEAL tool for running stuff like this as they’re on all the time anyway.

Pete.

Advertisements

21 thoughts on “NODE-RED FUN

  1. Can you point me to the firmware i have to put on my esp8266 to get this running ?
    Especially when the esp is connected to an arduino?
    because of all of the things you are testing a got lost in the firmware i need to use on the esp
    i want to do the same thing and as it seems you got things working now really good .

  2. Great articles by the way, and Node-Red is definitely a great home automation tool! I started with the ESP8266 and trying to make it work with MQTT, but not having much success. I got tuanpm’s MQTT firmware, but am having some difficulties getting it to respond to topics and/or controlling GPIOs based on the input.

    Given your experience, maybe worth doing an article about how you tackled this?

    • Well, I lost a LOT of time until I got it into a decent environment like ECLIPSE – and that makes life easier. The subject is BIG – you should be well familiar with using a desktop tool like mqtt-spy to talk to your broker – once that is working a treat – then you need to be able to go into the /include/userconfig.h file and put your details in there.

      Here for reference are mine – minus my password of course.

      #ifndef _USER_CONFIG_H_
      #define _USER_CONFIG_H_
      #include “user_interface.h”

      #define CFG_HOLDER 0x00FF55A5
      #define CFG_LOCATION 0x3C /* Please don’t change or if you know what you doing */

      /*DEFAULT CONFIGURATIONS*/

      #define MQTT_HOST “home.scargill.org”
      #define MQTT_PORT 1884
      #define MQTT_BUF_SIZE 1024
      #define MQTT_KEEPALIVE 120 /*second*/

      #define MQTT_CLIENT_ID “PS_%08X”
      #define MQTT_USER “admin”
      #define MQTT_PASS “xxxxxx”

      #define STA_SSID “loft-east”
      #define STA_PASS “xxxxxxxxxx”
      #define STA_TYPE AUTH_WPA2_PSK

      #define MQTT_RECONNECT_TIMEOUT 5 /*second*/
      #define MQTT_CONNTECT_TIMER 5 /**/

      #define CLIENT_SSL_ENABLE
      #endif

      That works a treat and it will now respond in the serial debug – in /user/user_main.c you’ll see the subjects it subscribes to (wifiConnectCB) and in mqttDataCb you can trap any incoming messages, read them and even send outgoing messages..

      This is just my example

      I put this at the top of user_main.c

      #define LED_GPIO 2
      #define LED_GPIO_MUX PERIPHS_IO_MUX_GPIO2_U
      #define LED_GPIO_FUNC FUNC_GPIO2

      and then doctored the routine below.. should be reasonably obvious

      void mqttDataCb(uint32_t *args, const char* topic, uint32_t topic_len, const char *data, uint32_t data_len)
      {
      char topicBuf[64], dataBuf[64], showBuf[64];
      MQTT_Client* client = (MQTT_Client*)args;

      os_memcpy(topicBuf, topic, topic_len);
      topicBuf[topic_len] = 0;

      os_memcpy(dataBuf, data, data_len);
      dataBuf[data_len] = 0;

      os_sprintf(showBuf, “{%s}”, dataBuf);

      INFO(“MQTT topic: %s, data: %s \r\n”, topicBuf, showBuf);

      if (strcmp(topicBuf,”home/openHAB/out/Light_FF_Bath_Mirror/command”)==0)
      {
      if (strcmp(dataBuf,”ON”)==0)
      GPIO_OUTPUT_SET(LED_GPIO, 1); else GPIO_OUTPUT_SET(LED_GPIO, 0);
      }
      /* Echo back to /echo channel*/
      MQTT_Publish(client, “/echo”, dataBuf, data_len, 0, 0);
      }

      • You don’t call it out in your example but where are you putting the Pin setup code
        PIN_FUNC_SELECT(LED_GPIO_MUX, LED_GPIO_FUNC);
        (I assume you are using it since you added the related #define statements

        I’m not sure why but when I add it in user_init(), the off command doesn’t seem to work. Once I moved it to inside the loop evaluating the received topic (right before the GPIO_OUTPUT_SET lines it works correctly).

        The only thing I can think of is when added to user_init it briefly sets it as an output but then something else in the MQTT code resets it so when it hits the GPIO_OUTPUT_SET they don’t work.

      • Well, I would just whizz over to my open project and confirm but the sodding power went out last night and I have 3 virgin screens in front of me… hang on….

        Yes in user_main.c right at the start..

        void user_init(void)
        {
        uart_init(BIT_RATE_115200, BIT_RATE_115200);
        os_delay_us(1000000);

        CFG_Load();

        PIN_FUNC_SELECT(LED_GPIO_MUX, LED_GPIO_FUNC);

        Someone needs to write a simple set of macros we can put at the start of our projects- I’ve never seen such complicated nonsense.. what’s wrong with..

        pinMode(2, OUTPUT);

        🙂

      • I’ll answer my own question from earlier, unless you modify the code if you are using the stock MQTT firmware (version as of 1/9) you need to add the
        PIN_FUNC_SELECT(LED_GPIO_MUX, LED_GPIO_FUNC);
        line after
        uart_init(BIT_RATE_115200, BIT_RATE_115200);

        In the UART function it initializes UART1 which maps GPIO2.
        I’m sure you know this since you have yours working but in case anyone else struggles with this I wanted to post a note.

        Thanks for all the great info by the way! Very much enjoy the blog.

      • And thank you for that. Next stop – adding in DHT22 code to grab temperature.

        I know the ESP-01 is cheap and easy but this lack of IO is starting to grate a little.

  3. Hi Peter,
    All good fun and manic times again:) Keeps the cells going, although having a few years on you, not so many of them to use!
    Just for the record, I started with mosquitto on an RPi, and as you had made a note of, it couldn’t keep up with the 50mS stream from Spy. Lost every other one. Then it all started playing up, so I moved on to the really small broker. Spy didn’t work too well there, being rejected for sending unknown versions of mqtt. I then got an old laptop up with mosquito, (vista believe it or not), and it has been good as gold so far. Mosquitto will install as a service, so that was handy.
    I did the 50mS runs, and put some messages from phones in at the same time. The log showed all were received.
    Having a look at mqtt-warn at the moment.
    As far as using ssr’s are concerned, my concern would be to only use them where it would be an inconvenience if they failed, as they will normally fail short circuit. (if / when they fail). Electromechanical relays are not the ideal components to have to use, but at least they do provide complete isolation, and very rarely fail closed. (welded contacts / arcing etc).

    You are giving a lot of inspiration with this blog and the detail in it, so just a thanks from me for the hours you spend keeping it up.

    Dave

  4. Thanks for the kind words, Dave..

    Some comments on the above.. I’ve now seen at least 4 items that “Suggest” that the PI is not up to the job – well, it’s putting me off buying one. I can manage with Mosquitto and MQTT-SPY just under 1m repeats – in other words 1000 round trips a second – actually I could probably manage more but the test script has a ms timer 🙂 I can get that just as easily from Really Small Message Broker – which I was VERY happy with – but… I have this DISKSTATION and Mosquitto – so from a reliability point of view that clearly is my choice – but I had NO problems with RSMB on a PC acting as a service. So a couple of options for people there.

    What is will say is that the RSMB instructions are a hell of a lot nicer to humans than the documentation for Mosquitto.

    SSRs – good point – I’d forgotten about that – I used to do home control years ago as part of our business and we developed some little boards using a PIC – early attempts at using TRIACs with compact fluorescents were disastrous and indeed we ended up with shorts. Ultimately our boards used the PIC to feed a MOC20 – which fed the TRIAC and we put a 400v VDR across the (load or triac, can’t remember) – all the problems went away.

    If course today’s solid state relays I have no idea what’s inside them – your point about mechanical relays are well taken – I bought some SSRs because of their 3v operation – ie ESP8266, SSR, 3v3 supply and nothing else… time will tell.

    Pete.

  5. Wow – Node Red…nice find! Thanks! I’ve been resisting learning Node.js cos I’m not the biggest fan of Javascript, but this might send me over the edge.

    The new SSRs are pretty good, and they’re used alot in industrial control. I wouldn’t be afraid to use a good one. But they are still significantly more expensive than comparable relays these days, especially when you can get those relay + driver boards from China at under $2 per relay.

    Raspberry Pi’s are awesome little network workhorses, easy to use, and only $35, but they have limitations. I wonder if the detractors have tried running it headless (no GUI) when it’s acting as a server. Anyway, the Pi is just a gateway drug to some of the newer more powerful single board computers that are available (eg Beaglebone Black, Intel Edison, and many others)

  6. It appears that you’ve got things sorted out, but for anyone interested in Node-Red with minimal configuration talent, take a look at http://thethingbox.io/. They have put together an image for the Raspberry Pi that integrates every thing you need to get started playing. Copy the image to a SD card and boot the pi. Log into Node Red. Pretty simple to get started!

    Between your ESP8266 work, openHAB, and Node-Red there is a pretty awesome capability available to the home tinkerer! Thanks for all your good work!

  7. I’m similar with node – I didn’t understand what it was – I’m still not happy about it needing an installation on every pc to be used.. but… the results in node red – speak for themselves – that’s a clever bit of programming and I’ve only just started to get into it. More over the weekend no doubt.

    • Note Red need not be installed everywhere. If your logic isn’t too time sensitive, one installation can subscribe to multiple MQTT brokers, take appropriate actions for each, and post back to a local or single consolidation broker. If you use reliable messaging, network latency or occasional dropouts would be covered. If reliability and speed ARE an issue, you’d probably want Node Red running somewhere in your local subnet anyway.

  8. Peter, thank you for all your work on ESP8266! I found this site around Christmas and have been glued to it ever since. I appreciate that it is all about Windows installations, too!

    So I have the latest versions of Mosquitto, Node Red, and MQTT Spy all working properly sending messages, but for whatever reason, I continually fail at getting Node Red to send out e-mails. I tried sending to 3 different e-mail accounts that I have (including Gmail as you show in your write-up) but no success. Would you please provide any hints/settings/screenshots that might help? I’m running a Win7 64-bit machine with JRE 1.8.0_25 but suspect a Java issue with Node Red due to warnings in bold that I see on their homepage.

    https://code.google.com/p/mqtt-spy/

    Thanks again and keep up the excellent work!

      • Thank you for the quick reply. I guess I am misunderstanding your 9 Jan blog statement below:

        “Sure enough… I pressed the test button – the debug showed the message – my MQTT /testreply monitor saw the same thing and an email came into my inbox.”

        I will await your session on Code Red sending emails. Kee[ up the great work.

  9. I know I started this one off – but I’m sitting here playing with node-red – and as an output I’d added a log file – no matter where I put it or what I call it I cannot find the file it is supposed to have created. So if I make a file c:\myfile.log it should be sitting in the root – nothing – anyone come across this?

  10. What kind of projects do you think Node-Red lends itself to? Any microcontroller with network capabilities?

    As someone looking for a visual way of programming (like labview or simulink) I seems like it can be a good tool for general learning.. but perhaps best used for only home automation (openHAB) using ESP8266?

    • Ok so from what I’ve seen so far by playing with node-red it’s kind of gee-whizz – look I can take one incoming form of data and transfer it to another easily… it looks great but at the end of the day – does it need to run on a PC, or is a £25 Raspberry Pi fast enough to do a decent job… at least with Mosquitto I’ve managed to get it running on an ultra-reliable DISKSTATION NAS box… not sure I’d want to chance a multi-property installation on a PC running node-red? Lots more research to do on this before I can pass sensible comment – but as a learning tool it’s a cracker. I’m personally thinking OpenHab – I just have some mental blocks on the subject. Someone needs to write an “Idiot’s guide to OpenHab” – sadly there are not too many of those around and the online doc is for people with lots of time…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s