All in One Ethernet Arduino

Just as I was getting all upset about my ESP866 woes, the postman arrived. The nice people at Freetronics sent me a couple of samples to play with – and here’s the first one.

tmp1AFD

I’ll confess I’m not a great one for using the Atmega328 as found in Arduinos – it has too little memory for my liking – but there’s no doubt it’s very popular and so an all in one board containing the processor AND the Ethernet card has to be a good idea. The on-board Ethernet is NOT the cheaper ethernet card which I wasted months on but is the full official Wiznet version – so this board,, which has a micro-usb interface, power socket, Ethernet socket and comes complete with lead and instructions, looks ideal for the more ambitious projects. There’s an SD card interface as well as a pair of half-decent looking 3v3 regulators – actually the 3v3 regulator looks man enough to handle the likes of various radio boards which need 3v3 so that’s a plus.   I’ll write some more on this when I can fit it into my testing schedule but right now this looks like a nice, solid winner. You have the link above for Freetronics.

Advertisements

MQTT and Arduino

Today I had my pal Jonathan over and we took a diversion from my current network on NETIO and Arduino-based home control – to look at something called MQTT which is a simple protocol for sending messages back and forth across the web from and to the likes of Arduino.

I say diversion because we came across some brilliant software which I’ll describe – but the documentation out there is designed for nerds – it is certainly not designed for real people with other things to do and the services are not entirely free so this certainly won’t replace what I’m working on – but for some monitoring and control purposes where COST is an issue – read on…

So – let’s get straight to it – you have an Arduino and you want to send commands to it over the Internet – or you may want IT to send out, say sensor data. You want to be able to control this remotely or monitor remotely – and you may want to LOG the data.

On the surface of it – easy – get an ETHERNET card for your Arduino, dump some software in there – you’re up and running. But serving web pages on an Arduino is VERY limited as is space for storage – by the time you add in SD card software you’re onto the next level up of board to get more memory etc etc… not necessary – read on.

To send info via MQTT and have it logged somewhere, you need an MQTT service – and this is where it gets complicated – some are free – all the ones we found come with severe service limitations unless you cough up. Can anyone show us otherwise? The service acts as a broker between the data you want to send – and wherever you want to send it – logging it in the process. This is all sub-second turnaround ideal for interactive stuff.

So for the purposes of explanation and testing, we found a free MQTT service that runs on your computer. This is ideal for testing but clearly a high resiliance external service would be better as for this you need a computer permanently connected to the Internet and left running.

It is called MOSQUITTO. Yes, the TT is correct. When you install Mosquitto on a PC, you run the Mosquitto.exe and it pops up a black box. That’s it running. It can also run as a service.

The idea being that your Arduino talks to this service – and subscribes to various topics (you make the names up and they are unique to your account – which is generally free but you’ll pay to connect more than a few devices and more than a bit of data. You can SEND some info to a topic or retrieve info from a topic. Elsewhere, a PC client or Android client may also be connected to Mosquitto and send information to the same topic (in which case the subscribing Arduino will receive that info) or simply poll the topic – in which case it will see the info the Arduino is putting out.

Think of it as a BROKER – an intermediate service.  You can add security – you can add encrypting and you can LOGS… taking most of the work away from the Arduino. Sadly the software was originally written for Linux and could we HELL figure out how to turn the log on. Advice welcome.

Anyway, here’s the link. http://mosquitto.org/

So you have that window open and running on your PC – that’s the service running – runs out of the box  – you open another window…. and run mosquitto_sub.exe -t test   – that window is now subscribed to a topic TEST – and will wait to show you anything coming in… your Arduino can publish to TEST and you’ll see the results on your PC.

image

Here you see after subscribing to TEST  (-t is TOPIC) the phrase “Hello World again!” is coming every few seconds from the Arduino.

There is another program in the package called mosquitto_pub.exe – and it can publish

image

In this example assuming the service is at 192.168.0.17, it publishes “ooh” to topic test and sure enough the other window sees the output.

There is a client for Android called MYMQTT but be warned it drops connections and is not very good – but once you understand it – it’s ok for testing. For PROCESSING environment there is some stuff out there but I’ve yet to figure out how to use it.

So – how do we make use of this on the Arduino…. we found code for the WIZNET card but to end up paying probably £20-40 for a temperature monitor – erm, no…

So we set about looking for some code to run on the ENC28J60 cards as they are CHEAP. We found that there is a library which relies on the ETHERNET drivers for NANODE which uses the ENC chip – but in fact none of it relies on the NANODE – it will work with a normal Arduino or clone…. apart from the MAC address setup – which it turns out you can just do manually. See code.

So you need as well as your normal Arduino, the ENC board (pin 8 for CE),  this https://github.com/sde1000/NanodeUIP and this http://knolleary.net/arduino-client-for-mqtt/

So – off we went to try that – super – except the example would not start reliably – we figured it was something to do with the ENC board reset and went about putting in manual resets and delays – turns out nothing of the sort – the client for MQTT connection software is just maybe a little lacking in checking that it is actually connected. Also – we found once it was all working that if you temporarily lost your connection or the MOSQUITTO service stopped and then restarted – you were stuffed. So the normal initialisation you might find in SETUP we brought into the main loop with a check in the loop to see if you’re connected and if not – connect – and retry until you do.  Result – up to now utterly perfect.

Note the topic “test” can be anything – in my simple example I can turn a light on but then sending a message kills it – read the code – it will be obvious which in short order… Clearly you can’t send binary zeros in this so you need to ascii encode any binary stuff – and the limit in size thanks to the ENC chip is 127 characters. you can control a LOT of inputs and outputs with that much message!!

Here’s the setup for this test using a topic arbitrarily called “test” – the ENC board uses pin 8 for chip select… connect power (3v3 – I use 2 dropper diodes from 5v), ground, SI,SO, SCK, CS and reset.

image

There’s a lot of serial stuff in there for testing only….note the callback routine that handles incoming packages…. and then below,  the loop itself – including making the connection (and retrying immediately or after a failure)

image

Ok so why bother with any of this – well, it’s CHEAP – you can make a board that will say return temperature to you and let you control bits and bobs for under a tenner if you make it yourself or get an Arduino clone and ENC board from China…. and if you need to log – well that’s handled by the service – I’ll leave you to investigate other services out there but really, testing with a local service is the easiest start.

Google Drive and Bandwidth Limiting

As many of you will know, in the UK at least we have an issue with broadband in that it our upload speed is no-where near as fast as our download speed.  In my case, not untypically I can download (i.e. play videos, grab music,  files and web pages) at something like 6Mbps. However the less obvious UPLOAD speed is around 600Kbps (i.e. 10 times less). This is often disregarded and yet it is very important.  Some folk have as low as 300Kbps and this is very limiting – especially if you want to run something like Skype.

Where web pages are mainly about downloading (i.e. grabbing content) Skype is mainly about uploading – i.e. you are SENDING (uploading) your voice and video to the other party.  As uploading is the slower of the two, this then becomes the limiting factor for quality and reliability for Skype and other forms of conferencing software.

If your connection was doing nothing else, this would likely not be an issue – but even if you only have ONE PC on the network connection (and many homes and offices have several) you might still have issues thanks to OTHER processes/programs using up your limited upload capability.

Examples might be Google Drive, Dropbox, Box and other backup/sync solutions.  In the background these programs check for changes in files (perhaps a WORD document you may be editing) – and if there is a change this triggers an upload to the cloud. It happens automatically and in the background.

Thankfully the people at Dropbox put a LIMIT as to how much of your bandwidth the program can use – hence allowing it to work while you’re Skyping without bringing Skype to a grinding halt.

The people at Google are no-where near as thoughtful despite having (arguably) a better product and yet despite many discussions and gripes in Google forums – right now there is nothing to stop Google Drive eating up all of your available bandwidth. Similarly programs like LiveDrive may do the same though in this case you can, if you know what to do, limit the upload bandwidth.

But the real problem is actually knowing if this is happening. Let’s say you’re right in the middle of a Skype conversation and the quality degrades – is this your broadband connection or did you just update/overwrite something that’s now being backed up – what about the kids in the next room…how would you know if something else is using up your bandwidth – and if you DID know what would you do about it?

To the (possible) rescue comes NetBalancer. This is not free – though a limited version IS – go get the free download and see for yourself (I’m assuming you are on a PC, not a MAC – and I’m using Windows 7 64-bit – which works just fine).

image

The program once installed shows you (live) which processes are using what bandwidth – both for upload and download (*see up rate and down rate above and take note if the B in KB is upper-case that’s kilo-BYTES as against kilo-bits – roughly a kiloByte is 10 kiloBITS).  Note in the example here, I simply right-clicked GOOGLEDRIVESYNC and set an upload limit of 20KB/s i.e. that process is not allowed to upload at a speed of more than 200 kilobits a second no matter what… Google Drive left to it’s own devices would consume ALL of my upload bandwidth.

This program in the unregistered form will let you control up to any 3 processes. If you choose to pay them something like £20 ($29.95) you can control as many as you want. In my case, this will do just fine. In the example GOOGLEDRIVESYNC is the ONLY process I’ve limited. In addition (right click) I’m giving SKYPE HIGH upload and download priority!

Something I did notice- occasionally for no obvious reason – OUTLOOK does significant uploads in the background….. like 300Kbps significant – just briefly but enough to mess up a conversation.

I hope this is useful to you – if nothing else you now know where to look when having problems with upload bandwidth. For this to be effective you’d need this on any and all machines connected to your broadband as you never know what might be happening in the kid’s room when you’re having an important Skype conversation!

The SAD Light lives

After more hours than I care to think about – the SAD light nears conclusion – just need a decent socket arrangement for the back and a few last minute tweaks.

The little box you see has no less than 2 micros in it (as you can’t get 10-bit PWM from one – so a second controls a chip designed for the job and talks to the first over i2c) – the 4 rolls of LED strip take in total more than 50 watts if all on at once and look deceptively dim because the 4 metres of strip in each case is wrapped up in a roll (and the white is almost off).

Just got some last minute software tidy up to do, this small unit has a display showing time, date, alarm time and other info and is controlled by an infra-red remote. In it’s day job it controls 3-colours of LED to produce soft or intense light in any of a wide range of colours – but it’s early morning job, over a period of 20 minutes, is to take the room – or more specifically the top of the bed, from darkness through soft early morning light – to bright, white powerful light – hence the SAD light!

One project finished 2 more on the go – the Ethernet thermostat has had 3 weeks without a single failure (Ethernet based control unit with serial data to LCD head unit soon to acquire ultrasonic presence detection)… and the 8-relay Ethernet controlled board (with short-range radio controlled 2-relay extension) has equally been trialling without issue for 2 weeks – not quite sorted a case for that yet.

SAD light