Arduino Home Control Part 1

Background
A little background before I launch into a project I’m working on..

In the closing decades of the last century, just as the Internet age was about to start, I was one of two directors in a small company who innovated in electronics – we did it in a variety of ways, some projects were successful, some before their time and just occasionally came the odd project that was not completely thought out.

Home Control last-century style
Home control fell into the latter category. We were into using the PIC chips (from Microchip) at the time and we ended up with a bee in our bonnet about home control – wouldn’t it be nice to have little boards all over the house controlling stuff.  We developed the product before too deeply considering the market – but what a product it was for a time.

Essentially a small board with a PIC, a TRIAC output (for controlling lights and heaters – and yes we could handle compact florescent lamps without issue thanks to a VDR), a digital temperature sensor (Dallas), a couple of inputs and a simply transistor switch to control PWM sound output. The board was smaller than a pack of cigarettes, cheap, ran on 12v and had a unique ID number.  We could connect maybe 128 of these along a 4-core telephone wire (normal white twisted pair stuff commonly in use today).  We could manage maybe 100 metres.  At the end we had a little board which provided 12v power and interfaced by RS232 with a PC. A program on the PC showed a plan of the house and the boards showed as little coloured block superimposed on the map of the house. The colours changed depending on the input and output status of the boards in real time – and you could write a set of rules to control what the boards did. The 4th wire ran simple PWM sound from the PC speaker around and a transistor could be programmed on any board to turn on – hence enabling sound routing to any board.

Application Control
We called the system “Appcon” which immediately got us into trouble with someone in the apparel industry and we had to promise not sell competing products!! To give you an idea of how incredibly powerful this system was, in my home which had 40+ of these units, I had wall-mounted PIR movement sensors hooked to inputs, heating, lighting and alarm hooked to outputs as well as door magnetic sensors.

I could route cricket or bird sounds to speakers in the garden, I could detect open doors and turn the heating up depending on the time of year, with a closed switch I could enable an alarm system (all done by rules using nothing more than the basic devices and inputs – and my PC) and of course enable PIR triggered yard lighting only during certain hours of the day.

The Automated Home
A magical system and make no mistake, it  was successful in a small-business kind of way, we sold many hundreds of the units, we got in major PC magazines – for example PC-Answers May 1984 – “Wired Tales” heavily featured the system as did many others. We got the hobbyist market and that was it and really there was no way to make a fortune out of this – but we had great fun developing it, making new friends who were equally fanatical about home control – but then the problem was wires…

As I’ve found out in my cottage here in the country – with solid floors and 18-inch thick walls – you just can’t go putting wires all over the place – partners get really fidgety about this stuff.. but then the alternative, inexpensive radio doesn’t work too well through solid walls either and over-the mains control – well that tends to be limiting and expensive.

There are solutions out there today, lots of them, but they have severe limits and are relatively expensive. Somehow the idea of paying £20 to £30 for a board to control a light seems ludicrous to me and the cheaper systems are inflexible to say the least.

Moving ahead 28 years…

I’m into Arduino right now – which is merely a modern-day PIC (well, the PICs are still around and surprisingly little has changed there) and for some time I’ve been thinking of home control – the problem is that wiring just won’t work for me here… and radio modules generally are just too expensive to incorporate into boards – well, unless you want to throw away £30-40 all in just to control a lamp!!

Cheap Radio Solution
I recently discovered the NRF24L01 boards, available from China on Ebay for less than £2 (now THAT’s more like it) and if you’re into Arduino you’ll know that given the chip (the most expensive part and some suppliers really rip people off) you can put together a board for WELL under a tenner.

Line of Sight Radio
Ok, that’s a start, I looked up libraries to handle the radio chip and one, RF24, does a cracking job of one-to-one communication. Sending a small package of data from one device to another reliably is easy with these boards and the library. But we need to be realistic – the range is limited.

My pal and I did some tests a while ago – out in the open – MAYBE 60 metres on a nice day.. we tried the really cheap units and the ones costing 3 times as much with a nice stubby aerial – not that much difference amazingly so we stuck with the cheap units.  What does make a difference is speed – you can run them at up to 2Mbps – but drop that down to 256Kbps – and your range goes up to maybe 80 metres line of sight.

That’s open air – what about indoors? – OH DEAR ME… thick stone walls just about cripple the signal  – in my cottage I can just about manage to get through an 18” thick stone wall, you’re talking 20ft IF YOU ARE LUCKY. In other homes with thin internal walls  I’ve seen 40ft or more. It all depends on wall thickness and materials, on what other radio devices are operating etc. I was about to give up – but then started to think about networking – what if you could bounce the signal around as in networks – cellnet, Internet etc.. I thought I was asking too much of a small, cheap radio but the chips do have the basics of networking in them – they can LISTEN to up to 6 channels at once (but can’t listen AND send at the same time)  – maybe there was a future in this?

Networked NRF24L01?
I scoured the web, lots of half-arsed projects that went no-where and ONE project that looked like a starting point – it’s called RF24NETWORK.  It’s not a true network in so far as plugging in arbitrary units and having them relay stuff all over the place – but it’s a start and enough to make a practical system albeit for those who didn’t utterly fail maths.

I started playing with the library and with examples and had some teething problems – some of the examples would not work on the latest Arduino development environment (IDE) and some of the stuff was not too clear. I wrote to the developer. Nothing. I went to his website and wrote there – nothing. He’s either on holiday, or busy or dead – either way it looks like for now the one and only simple network is unsupported – but the good thing is – it works (there’s a caveat later). I wrote an article on the NRF24L01 and the RF24 library here originally.

Limited Network
I need to clarify… the NRF24L01 chips can listen to 5 other devices at once… that’s it. So how do you make use of that? The way RF24NETWORK code does this is to ask you to regularly call a function on all of your boards – this function transparently handles the networking. You have to number your boards. The boards are numbered in Octal (don’t let that put you off – the designer has used this to make best use of the 5-byte addressing) and Octal in Arduino is expressed with a leading 0. So, to express the Octal “222” you simply preface it with a zero – 0222.

This incredibly easy online convertor makes it all clear. 0222 is 146 in decimal for example. This is irrelevant of course but as this caught me out at first I thought I’d better explain why I’m prefacing numbers with zeros!!

There are numbers, then there are numbers
So, given that the boards can talk to 5 others – it makes sense to build a tree structure – any device can talk to a device further up the tree – and 5 devices below it. Device 1,2,3,4,5 can all talk to device zero (0) – and 0 can talk back to them – directly.  Moving outwards from there involves some hopping – BUT FEAR NOT as the software does that for you automatically – you just have to think about your numbering. In theory such a simple network should be easy to build but I can’t get my head around the acknowledge signals travelling up and down (how can you have unique packet numbering to ensure packets get there – when you can’t talk directly to the root!)… so for now, RF24NETWORK gets the job until someone comes up with a better solution.

It’s getting too complicated
The numbering system is well described in the RF24NETWORK pages so we’ll not go into that here. Suffice it to say you can attach thousands of units together – but I was thinking more of 2-100! I’m sitting here with device 0 talking to device 02 which is acting as a gateway to device 022.

Relax – you don’t need to know any more
So we’re talking hundreds of potential devices. Now it’s important to realise you just need to understand the numbers – nothing else as the networking software handles the rest.

If course all of this is ok but as units can’t talk and listen at the same time – I figured a controller was needed… and then what – you need to DO something with all of this… and I started a series of brainwaves which brings us to where I am now – developing a working home control system based on nothing more than cheap Arduino-type boards.

And that is the end of part 1 – in part 2 we’ll look at putting some meat on the bones – and yes, I’m sitting in front of a basic working system here, this isn’t going to be merely an academic effort – but there are some issues – read on.

See part 2.

Arduino Home Control Part 2

In part one I gave you a little background as to why you might  be interested in home control – and some information on the RF24 and RF24 libraries.  If you’ve not read the earlier article – I suggest going there first.

Putting some meat on the bones
With my background in home control I’m looking for low cost and simplicity – but don’t take that to mean simplistic. I want powerful control over a range of devices, inputs and outputs. The important thing is cost and reliability – it has to cost very little – and sit there and just WORK once set up.

The design explained below is based on the notion of a controller and a bunch of slaves -  in each case just a simple ATMEL 328-based board put together using the Arduino IDE (1.1 as it happens).

So let’s take the “slave” units first. Starting at the beginning, they need inputs and outputs. In my version those inputs will include temperature and humidity (don’t worry that’s one cheap single-wire device from China) and an analog input (thanks to the Arduino 328 chip) which could be used for example with a cheap light sensor. They’ll have some digital outputs (including a relay) and 3 analog (PWM) outputs.

So how do they communicate?
“In the beginning”, each slave broadcasts a message – every few seconds to say “I’m here” until it is contacted. The delay is fixed but SLIGHTLY different for each one – merely incorporating the ID number into the delay – just to make sure you don’t get a boatload of units trying to sign on at the same time. This constant sign-on ONLY happens until they make contact with the master or if they LOSE contact with the master.

The master maintains a list of slaves it should be contacting regularly. Once slaves get confirmation that they’ve sent something (i.e. some information is sent back to them from the master – they STOP sending this regular heartbeat – instead they reply immediately and once only, every time unit 0 talks to them. So unit 0 sends a package (outputs) – the units program their pins respectively, read their inputs and send the package back. If they hear nothing back for a few seconds – they go back to their constant sending without waiting to hear anything.

And now to the master. The master initialises by clearing out a table of working units…. and sits in a loop looking at that empty table while checking inputs.

If a communication comes in from any unit, the table is updated with that unit and it’s data. From now on the loop sends data to that unit and any others constantly in a loop. If that communication fails, a flag is set to say that unit is not responding (the OK flag – merely the top bit in the address field)..

As more units sign on – the table fills – and the master loop talks to each one in turn.

After each of these communications before going to the next item in the loop – the software checks for incoming data – if it gets a package it either ADDS this unit to the table or updates the table – resetting the OK flag high even if it was high already.

And that’s about it for communications.

In the slaves you may have all units active or some may merely be acting as bridges (device 2 for example) to get through a thick wall or extend range generally. An INPUT pin on these units, if pulled LOW, tells the unit not to check inputs, set outputs or process the rules – and not to communicate at all but simply to call that network function so that it may pass information back and forth. With current consumption of radio+chip of less than 20ma, one could consider making this solar powered even. The easy way however is a simply USB plug-in-the-wall adaptor – these are cheap and efficient (sub-£2 from China) and should not stress the electricity bill.

So that’s nice – a fully functional, massively expandable totally brain-dead home control system capable of reading temperature, humidity, analog inputs, digital inputs and setting digital and PWM outputs all from a simple table stored in memory in device zero – but with no brain to do ANYTHING useful.

Talk for miles?
Incidentally all of the above WORKS – I have unit 0 and unit 022 sitting in front of me – not talking to each other but talking via unit 02 which is on the other side of a very thick wall… so immediately I’m looking at open air range of maybe 80+80 metres. I could be looking at 80+80+80+80 or more. If you want to get REALLY large range – I’m prepared to guess that if you stuck 2 units into the LNB holder of a typical satellite dish and pointed them at each other you could get DRAMATICALLY more range (again think perhaps of solar power+battery here if you live in the country). Not tried it yet and you’d have to fiddle a bit but that’s what LED indicators are for!

Brain Dead Home Control

I woke up in the early hours over the weekend and I guess my history of home control (see, there’s a reason I put the history up there) is coming into play here as I have a plan.

So – device 0 has this table – ID numbers, input status, output status, analog values…

table entry

and we might have lots of these – remember the top bit (16 bit number) of that node address will be set if the device is THERE and ACTIVE – if there is anything there other than a zero you would assume that the device is either sleeping (I’ve not thoroughly thought that out yet) or just on the edge of range – so basically if the value is not zero you’d use this info.

The data being passed back and forth is even simpler…

data

So how do we take this basic info and give it a brain.

SD memory seemed to be the simple answer…a series of questions and answers that loop and together make the system DO something. I had a vision of a little slot in the master unit that accepted SD memory and you’d simply program something up on your PC and populate the SD. But then after a few coffees I remembered that I’ve a small library for handling 24C65 chips – a tiny 8-pin, widely available flash memory chip that is cheap and easy to use. The current plan is to put instructions in here from any PC via something like comma-delimited serial of even simply XML – the format isn’t important – it’s how to create the content and get it sent over that’s important – but that comes later.

Simple… it can’t be THAT simple can it? Well yes – read on… I hit on the idea of a simple sequence of numbers to control the thing – based on 5 bytes at a time.

blocks

Bear with me – this will become obvious.

In order to actually DO something you need to have rules – ie conditions and actions.

I came up with the format you see above – rule (8 bits), device address (16 bits), a value (8 bits) and the next command (8 bits) – this is incredibly easy for a simple processor to handle.

So the rules may or may not be conditions or actions and may or may not use the device address and value.

So for example given that each unit has a couple of inputs.. some suitable conditions might be…

If input 1 on device 022 is high

If input 1 on device 022 is not high

For the sake of it I’ve given these one-bye conditions a name…  IF_IN_1  and IF_NIN-1 – but you could call them any thing you like – they need the device address only – they ignore the value…

If you’re checking temperature you might use the value..

IF_TEMP_GT (i.e. if temperature reading on a device is greater than)…   oh, how can you store temperature in one byte? Easy – assuming you’re only interested in a practical temperature range and only to the nearest 0.5 degrees C, it’s easy to fit that into a byte and saves messing with floating points etc.

You might expect the INSTRUCTION at the end to be “THEN” – and you’d be right most of the time but I’ve implemented AND and OR ……

AND is also implemented in the ACTIONS….. turn on output 2 on device 022 AND turn off output 1 on device 023…

After actions are done the instruction END finished off that rule nicely and a rule starting with ENDRULES finishes the table.

Here’s an example – for now I have this stuff stored in memory  – adding the 24C65 comes last.

tmp457

rulesWhat you see above is a simple set of rules…each complete rule taking up maybe 10 bytes,  talking to device 18 (022) in this case and bearing in mind we’re looking at an 8KBytes chip – you could put a LOT of rules in there and there is no reason to make such an arbitrary limit.

You can imagine needing a range of commands to cover all the inputs…oh and some general purpose flags – (use the device address as a flag address but given we’re talking Arduino we’d not be talking thousands of flags – maybe a few dozen). Perhaps a few automatically-decrementing counters… I plan all of that.

With the ability to check inputs and flags and then set outputs and flags the power of such a simple, cheap system is down to your imagination… I’ve also added time and date commands and a random function …I did think that I’d have to make device address more than 16 bits to handle time – but then you’d really not need time as such more like day of week, hour of day, month etc.. i.e. if month > October OR month < March do something with the heating… etc.…

So many things you could do but for now I have implemented this lot.

So what are the catches.. right now I could do with someone to cooperate on this – there does not seem to be any development going on with RF24NETWORK– the comms is giving me grief – my coding should allow constant communications yet every now and then at high speed, communications stops for a second or so and I have no idea why – it’s reliable – it always comes back but it is failing – and the requirement to constantly poll is getting to be a show-stopper. In the network software as it stands, using RF24NETWORK can stop if a unit acting as a relay fails – all the units after that will fail – 328-based boards can be made INCREDIBLY reliable but it would be nice to have a better networking layer if that is possible. Having a better understanding of RF24NETWORK library would be good as would figuring out how to get a public function out of it to indicate when it is relaying information (to put on a LED to help with range checking). What is REALLY needed is to get RF24NETWORK running reliably under interrupts – one guy did start it but there were concerns over the code – and try as I might I cannot contact him or see any related development on the web. Also while I envisage using something simple like a spreadsheet to create more complex rules it might be nice if someone could look at that angle with a view to a better way.

However, as you can see, I’m off to a flying start, the rule example you see above is working right now on my desk  reliably aside from network reliability.  If anyone has any experience in using interrupts with RF24Network I would be more than happy to hear from them.

Update May 17th 2013 – it has come to my attention that a gentleman by the name of G Copeland has a fork of the RF24 library which he claims is more reliably while retaining compatibility. In all of this time I have not been able to get the code to work reliably – the radio just does not seem to run smoothly – with caps of up to a second in transmission…  So I’ve replaced the original RF24 library with this one…  I’m experimenting with it right now – it worked without any issues – but I’m still not seeing completely smooth transmission. With devices 0, 02 and 022 I’m getting results back (022 can’t work without 02 being on as 0 can’t address it directly) and there are still gaps of no transmission – but I’ll keep working on this and update this blog – meanwhile if anyone has any ideas do feel free to contact me at pete@scargill.org

See video demo of the RF24Network running with my rule set here

Update June 23rd 2013 – I’ve not taken the rule system further – however this article has now been superseded by a more practical development which is currently near fruition – a slightly different route however I think you’ll agree when you see it – something that is easily put into practice – I encourage you to follow this later article on home control over the Internet… I have the RF24 Network working reliably now along with Ethernet control. Worth a read.

 

Regards

Peter Scargill

Networking the NRF24L01+

tmp97BDThe tiny NRF24L01 modules you see in the photo to the right (the module handing off the board to the left) make great short distance communication units for Arduino projects – using the RF24 library. The main attraction of these radio units is their cost – under £2 each module from China (Ebay). The downside is they are short range, maybe 80 metres or so in a straight line but dramatically less through walls.  The main board you see is something I put together as a general purpose Arduino-compatible board, soon to be revised.

I’ve done a fair bit of experimenting with these radio modules as they make really cheap radio communications possible – the RF24 library works ok but the first thing to do is change the default 1Mbps data rate to 250Kbps. Still quite fast, considering you only send short packages – but this adds maybe 50% to the range depending on obstacles. I found the variations with a better aerial were only marginally better than these green10-pin modules.

http://maniacbug.github.com/RF24Network/index.html

The real breakthrough is to use these in a network – I have scoured the web and found almost nothing out there to do the job –some academic half-finished projects… but there is one… RF24NETWORK. I had to go into the main library file and change the speed (one location, easy to find) and in the test file do the address assignment in code (the example accepts a single digit from the serial input – if you don’t have anything attached to the input and leave it floating,  you’re asking for trouble as the format is too simple – but then I am just talking about an example – one would assume you’d burn in the address by another method in production.

The fellow has documented everything well but beginners beware – if you see a leading zero, he’s using OCTAL (base 8, ie 0 1 2 3 4 5 6 7 10 11 etc.) numbers which threw me off for a few minutes…

So the idea is you connect units in a tree structure… the root is device ID 0… then at the first level up you have devices 1,2,3,4,5.  The modules can listen to 5 channels at once so that’s how he’s based his network – here’s where the octal comes in  -siblings of device 2 would be 012,022,032,042,052 (they are Octal numbers). Any of those 5 can listen to 5 more etc.  It is possible to put thousands of modules in this network (though I would not recommend it) and I’ve already proven to myself that it works by making 3 units (the photo above is a board I designed some time ago as a general-purpose Arduino board) and spacing them so that the last board could not possibly communicate with the first one by sheer distance.

Works a treat – well worth investigating. Here is a conversation (with debugging turned on) and you’ll see the messages being passed around. Note line 8014. This log is coming from device 0.

Update May 2013: This article was originally written in 2012. Note that I have written a later article here – there are still issues with the networking software as you will see.

tmpE7E0