Investigating the ESP8266–Serial Killing

The ESP8266 is a potentially revolutionary new, small WIFI module costing under £3 ($4). Revolutionary because it will allow small microcontrollers to interface with the web without a) a costly ETHERNET card, b) a more costly traditional WIFI unit and c) without a costly (in terms of storage) library for Ethernet – which can take up half of, say an Arduino UNO’s FLASH memory.

The ESP8266 units work serially – i.e. you talk to them via serial in and they talk back via serial out. They require +3.3v and ground. Regardless of whether you are using a 5V processor or 3v3, the serial output will work with both – but you may consider a level shifter to talk from the processor to the WIFI board if the former uses 5v. This can be as simple as a resistive divider. I am using just in fact a series resistor – but then – my one and only board doesn’t work properly.

Initial translations and blogs out there suggest you need serial in and out – and SOME of tem suggest you also need to take a pin called  CH_PD to +3.3v. The problem is that there are identical looking boards some of which have that pin and NEED it connecting high – others don’t have that pin.  Check out here and here.

So – most of the common code you see will instruct you that the units need 115,200 baud data – and that you start the ball rolling with a reset command. They usually show the use of the Arduino serial port – sending out the command – and an option I’d never used before to analyse the result.. i.e. Serial.find()

So – armed with my new chip I went off and tried this – everyone in the web is using an Arduino with only one serial port – I prefer the Atmega1284 which is the same but better and has 2 serial ports – so when you see me refer to Serial1.find() you’ll realise I’m talking about the second serial port.

So to start the ball rolling you send this…

Serial1.println(“AT+RST”);

And you get (or I get) this rubbish back if you care to look at it.

OK

Eh? What’s this – that’s not what I SAW – it’s what pasted into this blog. What I SAW was this..

OK

  ets Jan  8 2013,rst cause:4, boot mode:(3,7)tail 12
chksum 0xe0
ho 0 tail 12 room 4
load 0x3ffe8000, len 3168, room 12
tail 4
chksum 0x93
load 0x3ffe8c60, len 4956, room 4
tail 8
chksum 0xbd
csum 0xbd

ready

How can this be? Why can’t I paste the lot?  This happened over and over – the copy and paste from the COM window to Windows Live Writer would not work – I ignored it – software bug.

But herein lies the rub…here’s the code I used…

Serial1.setTimeout(5000);
Serial1.begin(115200); // for debugging
delay(1000);

Serial1.println(“AT+RST”); // reset and test if module is ready
my=millis()+2000;
while (my>millis()) { if (Serial1.available()>0){  Serial.print((char)Serial1.read()); my=millis()+2000; } }

Why on EARTH would the result be spit up like that above…

So – here’s what everyone else is using with a variation, I’m looking for (OK) – others look for “ready” – bear with me..… we’ll not repeat the initialisation code here.

Serial1.println(“AT+RST”); // reset and test if module is ready
if (Serial1.find(“OK”)) Serial.println(“Got OK”);

I’d never used this before.. send the data out and wait 5 seconds OR return if you get the sequence OK.

But – what of the REST of the result.. all that crap that comes after ok? SURELY the find() function must wait SOME time and get rid of it – or is it  going to be sitting there waiting to ambush future checks?

And it gets worse.. because the ABOVE works! But this which others are using, doesn’t.

Serial1.println(“AT+RST”); // reset and test if module is ready
if (Serial1.find(“ready”)) Serial.println(“Got ready”);

How can that be – everyone else couldn’t possibly be wrong – how can they find “ready”?  So I made this combination..

Serial1.println(“AT+RST”); // restet and test if module is redy
if (Serial.find(“OK”)) Serial.println(“Got OK”);

my=millis()+2000;
while (my>millis()) { if (Serial1.available()>0){  Serial.print((char)Serial1.read()); my=millis()+2000; } }

So this time I’d trap OK – and then look to see if there was anything left… and little surprise there. All the stuff following OK came flying into view!!

So I tried this – as the data all comes within a second or so – surely a delay after sending would ensure the whole lot was in the buffer ready for checking?

Serial1.println(“AT+RST”); // reset and test if module is ready
delay(3000);
if (Serial1.find(“OK”)) Serial.println(“Got OK”);

my=millis()+2000;
while (my>millis()) { if (Serial1.available()>0){  Serial.print((char)Serial1.read()); my=millis()+2000; } }

Erm, NO!!

Got OK

ets Jan  8 2013,rst cause:4, boot mode:(3,7

Exactly – by now I was thinking of taking up woodwork… the OK was trapped but not only was some of the stuff left in the buffer – but the text “ready” was missing off the end!! A light bulb turned on… Could some of this  have to do with fact that the Serial buffer in Arduino is only 64 bytes long?? There’s more than 64 bytes here…  So presumably by waiting, I’d LOST the remainder.

Then I twigged – if you’re waiting for “ready” you might never get it if the buffer fills up and does not discard what’s there… and that would account for the result above – how on EARTH are others getting this to work?

Then I remembered something about increasing the default buffer size for serial.

In HardwareSerial.cpp – which is located at  hardware/cores/standard  or in my case as I’m using the 1284 chip and have a special setup for it the file is located under my documents at hardware\mighty-1284p\cores\standard

Sure enough – I increased the size of the SERIAL_BUFFER_SIZE from 64 to 512 bytes (I have 16K Ram on the 1284) and the LOST DATA problem above went away.

But it STILL would not find “ready” – at which point I thought – you know – I’ve invested 512 bytes of RAM and this is getting WAY too messy – I’ve never used .find() before – so it won’t be missed  – I could only assume there must be a special character in there somewhere stopping me finding “ready”

And sure enough – it would find OK but not reset or ANY word after that OK area – maybe a NULL in there somewhere??  I could have checked but life is short.  There had to be a better way. SO I wrote my own… I wanted something that would search for a phrase, waiting a certain length of time – but also waiting after anything ELSE came through until the buffer was truly empty..

Serial1.println(“AT+RST”); // reset and test if module is ready
if (SerialFinder(Serial1,”ready”,3000,300)) Serial.println(“GOT IT!!”);

Spot on!  Now, that still does not account for why my WIFI module won’t work  properly – that may well need another module purchasing to compare with – but I hope that helps others who may well be struggling with this very issue. Here is the function below – just drop it in your code or in a library somewhere  – do what you will with it. It’s simple enough… feed it the string – it will return true or false whether it finds the string or not – but not until the timeout. You could add a parameter to kill the timeout once the string is found – but then – what about anything coming in after then and you can’t just flush the buffer – info may not have yet finished coming in. Seems to me the right option would be to just make the delay “sensible”.  You can of course change from Serial1 to Serial etc (multiple serial does not work if you’re using a humble UNO but the mega has either 2 or 2 UARTS and I DEFINITELY recommend having one for your project and one for monitoring/programming) anyway – here it is.

boolean SerialFinder(HardwareSerial &refSer, char *str,unsigned long howlong,unsigned long timeout)
{
   boolean gotit=false;
   unsigned long mytime;
   char *strtemp;
   char incoming;
   strtemp=str; mytime=millis()+howlong;
   while ((mytime>millis()) || refSer.available()) // if timer demands or there is something there
   {
    if (refSer.available())
      {
       mytime=millis()+timeout; incoming=refSer.read();
       Serial.print(incoming); // for debug
       if (incoming==*strtemp) { strtemp++; if (*strtemp==0) {strtemp=str; gotit=true; } } else strtemp=str;
      }
   }
  return gotit;
}

And so I start again from scratch but at least this time knowing my searches work… and so it begins..

Serial1.begin(115200);// talking to WIFI
Serial.begin(115200); // for monitoring
delay(1000);
Serial.println(“Initiating with AT+RST”);
Serial1.println(“AT+RST”); // restart module
if (SerialFinder(Serial1,”ready”,2500,200)) Serial.println(“Good”); else Serial.println(“Nope!”);

Advertisements

ESP8266 WIFI MODULE – CHEAP WIFI?

tmp17B8I have in front of me what many others have ordered and few others have as yet – an ESP8266 inexpensive WIFI module from China.

This module has an 8 way connector (there are variants with 4) and an onboard antenna (there are variants with a socket).

The unit operates from 3v3 and the connections are ground, 3v3, tx and rx. Yes, at long last a low cost board that hooks into your WIFI and which your Arduino or other projects can talk to.

Make no mistake – the ESP8266  has the potential to be a GAME CHANGER – WIFI has always been ridiculously expensive for small projects – Arduino clone £5 – WIFI unit £40  – just doesn’t make sense. There are 2 chips on the board and the small one is FLASH memory (I checked), the other is the processor with built in WIFI. There have been a couple of attempts before now but nothing in this price range.

The predominently Chinese documentation is slowly being translated and there ARE examples of code – just enough to let you use modem commands like AT etc.… and get a response. Serial speed apparently is 56k baud though mine works at 115200 baud.

And therein lies the rub. I can’t get it to work properly. With power (3v3), ground, input and output connected, it responds.. but the examples don’t work.. for example –  on the web the example code AT_RST should respond with “ready” – mine responds with “OK”. The unit will then sometimes connect to my WIF – or so it seems – but then connecting to a web page either produces a failed response – or something like “busy” in the responding text.

I am using a 5v 1284-based Arduino.  Example here uses Serial to talk to the board and a software serial to talk back to the PC or whatever.. I changed that to talk to the board with SERIAL1 and the normal SERIAL to talk back to the PC – other than that I tried to leave the example as is.  I got nothing back until I realised I was receiving OK at the start!

At first, NONE of this was happening until I happened to connect (via a 1k resistor) one of the spare lines WCHICH YOU SHOULD NOT APPARENTLY NEED TO TOUCH to Vcc – then the output started to behave properly – however, for all I know this might be causing another issue. Without such a connection, the output sits at around 1v and does nothing. With that connection it talks reliably back to my Arduino – but with differences noted above.

I wrote to the Chinese supplier who clearly does not speak English – all I got back was a pointer to a data sheet.

Whether I have an early version I don’t know but I would LOVE to hear from others who have succeeded in getting reliable client code to work with this board.

 

1284p Enlightenment

This is a major update of a post I wrote a year ago.

In a previous post I wrote about the article in which Manicbug talks of the 1284p and Optiboot. It’s been a long learning curve since then, blowing bootloaders into the 1284p chip and then attempting to blow programs (sketches) only to find the programming process unreliable. Various solutions abound – and I’d previously adopted the one involving a series resistor and cap into RX0 in order to round off serial signals going into the chip (for programming) because of some mysterious bug.

Well, enlightenment took some time but here it is. The cap and resistor are not at all necessary. The issues lies in a problem with some – perhaps all 1284 chips – the TX input is right next to the crystal and it is possible for high speed serial data to cause “noise” affecting the crystal – and hence bring the whole programming experience to a grinding halt – somewhat intermittently – some get it – some don’t – some just get it part of the time.

The ANSWER lies in the FUSES. The Optiboot setup – which is in the BOARDS.TXT file – but not your MAIN boards.txt file – but the one as described in the Manicbug article i.e. in your “my documents/arduino/hardware/boards.txt” file. It’s a simple text file readable with Notepad++ etc. – in there you’ll find a line that refers to “.low_fuses” – do a search… and you’ll likely find the fuse set to 0xff.

That was how my file was configured. You see the process of adding a bootloader via the IDE ALSO sets the fuses for the chip – i.e. crystal frequency etc.  The “FF” sets the oscillator into a low power mode – and that’s fine – we all like to save power – but it makes for a VERY WEAK oscillator signal. Changing that value to 0xF7 and then saving the file, powering up your IDE and blowing a new bootloader will make all the problem simply fade away.

Another simple way to do it is if you have something like the Dragon if you happen to have one – simply read the fuses – change the low fuse value from FF to F7 and program… Bob’s your uncle.  But don’t do as I did, waste several hours wondering why your Dragon processor won’t talk to the chip…. if the chip is already set to use an external crystal – and you’ve put a DIP socket on your Dragon – where’s it going to get it’s oscillator from!!! Took me ages to twig to that – simply stick a crystal in the socket with the chip!

So now, I have a bunch of 1284p chips which need no special attention to program beautifully.

Why would I bother with these chips? Well, they’re the largest that Atmel do in the range that are still DIP – i.e. easy to use for hobby/casual use. Not only do they have 128k of FLASH which is 4* that of the venerable 328, but FAR more importantly – they have 16K of RAM – and when you’re looking at large programs, SD memory, Ethernet etc., RAM is always the issue. No more with these chips so it’s worth the effort to learn how to use them properly.

However, support for these is still patchy and even the latest 1.57 BETA release of Arduino has not made matters any better. Firstly AT LEAST the Ethernet library still doesn’t support the chip… well it didn’t until now… implement this fix:

In your arduino/ethernet/utility/w5100.h file you will see this..

private:
#if defined(__AVR_ATmega1280__) || defined(__AVR_ATmega2560__)
inline static void initSS()    { DDRB  |=  _BV(4); };
inline static void setSS()     { PORTB &= ~_BV(4); };
inline static void resetSS()   { PORTB |=  _BV(4); };
#elif defined(__AVR_ATmega32U4__)
inline static void initSS()    { DDRB  |=  _BV(6); };
inline static void setSS()     { PORTB &= ~_BV(6); };
inline static void resetSS()   { PORTB |=  _BV(6); };
#elif defined(__AVR_AT90USB1286__) || defined(__AVR_AT90USB646__) || defined(__AVR_AT90USB162__)
inline static void initSS()    { DDRB  |=  _BV(0); };
inline static void setSS()     { PORTB &= ~_BV(0); };
inline static void resetSS()   { PORTB |=  _BV(0); };
#else
inline static void initSS()    { DDRB  |=  _BV(2); };
inline static void setSS()     { PORTB &= ~_BV(2); };
inline static void resetSS()   { PORTB |=  _BV(2); };
#endif

See that first line I’ve put in BOLD – esssentially it is saying that if you have a MEGA board – ie one using the 2560 chip – B4 is your SS line otherwise for default Arduinos it’s B2. NOPE it’s not… Change that conditional in bold to this…

#if defined(__AVR_ATmega1280__) || defined(__AVR_ATmega2560__) || defined(__AVR_ATmega1284P__)  || defined(__AVR_ATmega1284__) || defined(__AVR_ATmega644P__)  || defined(__AVR_ATmega644__)

Similarly in the otherwise excellent RADIOHEAD library – the interrupts assume the two interrupts of the 328 – or the many interrupts of the 2560. The 1284 has 3 and they’re not on the same pins as the 328 as you’ll see if you study Manigbug’s line – but fear not – by the time you read this they’ve probably incorporated the fix into their radiohead.h file – if not – here’s my latest fix.

#if defined(__AVR_ATmega1280__) || defined(__AVR_ATmega2560__)
// Arduino Mega, Mega ADK, Mega Pro
// 2->0, 3->1, 21->2, 20->3, 19->4, 18->5
#define digitalPinToInterrupt(p) ((p) == 2 ? 0 : ((p) == 3 ? 1 : ((p) >= 18 && (p) <= 21 ? 23 – (p) : NOT_AN_INTERRUPT)))

#elif defined(__AVR_ATmega1284__) || defined(__AVR_ATmega1284P__)
// Arduino 1284 and 1284P – See Manicbug and Optiboot
// 10->0, 11->1, 2->2
#define digitalPinToInterrupt(p) ((p) == 10 ? 0 : ((p) == 11 ? 1 : ((p) == 2 ? 2 : NOT_AN_INTERRUPT)))

The bit in bold is the bit I added and have recommended. This says that on the 1284, the interrupts Digital pins (not physical pins) are 10,11 and 2 for interrupts 0,1 and 2 respectively. In the case for example of using the Si4332 chip which is compatible with the RF22 section, interrupts are used to track incoming data (I only WISH they’d do that with the NRF24L01 chips).  I’ve tested all 3 options with my fix above – works a treat.

Also in the 1.05 Arduino environment, I found that storing the Mighty 1284 board info in the hardware directory of my sketches folder was enough to ensure the chip turned up in the list of chips in the IDE. Not so in the 1.57 release which could not find the chip. I found that by making a folder until that custom hardware folder called AVR and moving the stuff into there – it all worked until it came to compile time and it complained about a missing setup. Turns out a file called platform.txt is needed which I copied straight out of the new arduino/hardware/arduino/avr folder fixes that one – I’m now up and running, compiling for the 1284 on the new beta environment.  Until on the of the new libraries fails due to me forgetting to make changes.

There will be other libraries that need fixing – if you happen to come across any needed fixes – please do let me know.

1284P and SD Cards

Just what is it with the Arduino people – they seem to have it in for the 1284p – one of the best chips out there. WHY is it a good chip? Well, it runs all Arduino projects – but you get 128K of FLASH and 16K of RAM – is that a good enough reason? AND they do a DIP version so it’s a doddle to do prototyping with it – and its’ cheap – and has lots of pins etc.

BUT – some of the libraries pretend the chip doesn’t exist.. SD is one of them –  I just updated the SD library and I happened to try a little test program to put something in SD on the normal Ethernet card… Nothing… I made sure I specified the select line (I chose output 27 – the Ethernet select line is output 4 on these chips normally)…. nothing.. then I remembered I’d updated the library – went on a search and found THIS. See code below – but I suggest ALSO adding for the 1284 – so see latter of the two updates.. this might apply to other libraries if you have any hassle…

Go into this file: C:\Program Files (x86)\arduino-1.0.1\libraries\SD\utility\Sd2PinMap.h
Change this line: #elif defined(__AVR_ATmega644P__) || defined(__AVR_ATmega644__)
to: #elif defined(__AVR_ATmega644P__) || defined(__AVR_ATmega644__) || defined(__AVR_ATmega1284P__)

Go into this file: C:\Program Files (x86)\arduino-1.0.1\libraries\SD\utility\Sd2PinMap.h
Change this line: #elif defined(__AVR_ATmega644P__) || defined(__AVR_ATmega644__)
to: #elif defined(__AVR_ATmega644P__) || defined(__AVR_ATmega644__) || defined(__AVR_ATmega1284P__) || defined(__AVR_ATmega1284__)

Logging the Internet of Things

Updated 18/9/2014: I’ve been making gadgets for “the Internet of Things” since before the phrase was invented.. one of the issues I’ve always had was displaying data. Oh sure you  can set up a website and talk to it but now you have to maintain that site etc and one-man-band graphics are rarely up with the best.

I’ve been looking lately at services that will consume data from my gadgets and display it on mobile phones, PC, laptop etc.

Oh sure you could also send the data serially to your PC and use the power of that machine to process it – but ideally one would want to have each gadget send regular updates all on it’s own. And thanks to the services appearing out there and Arduino – now you can.

This is where I’m up to – having dismissed several blind alleys…(and thanks to all of you for the feedback).

So I have a controller in the house that talks to various sub-controllers – collates the information and makes it available to me on my phone – temperature, humidity etc and lets me control lights etc… what’s missing is logging – the chips I use don’t have a lot of storage and getting that out onto the Internet at speed would not be trivial even if they did – and I can’t be leaving the phone on 24/7 to log stuff – so instead I’ve been looking for services whereby I can send data out at regular intervals, a small package at a time – and have some server somewhere process that data.

For my first stop we’ll look at Grovestreams – https://www.grovestreams.com

Here you use a simple Ethernet client on the Arduino or similar and send a PUT request with some data – seriously – a simple command-line job on a PC which is hardly more complicated on an embedded machine. I send data every 10 minutes on internal, external temperatures and overall humidity from our home in Spain. Been doing that for weeks now. The result?

tmp86C8

As you can see, it’s all been operating flawlessly since the back end of August – when we left it was dry and very hot – today the humidity is a lot higher (green), internal temperatures slowly falling and external temperatures going as low as 14c.  All of that on a free service (as long as you don’t overdo it – check their site for cost details).

What’s nice is the web-based zoom – see the last couple of days above – and the sliders the bottom of the image.. Also they handle the averaging and there are tons of options.

tmp25C6

Neat – so next I tried Carriots – https://cpanel.carriots.com  and this has a free option – I’ve yet to get that working but the day I asked for help they came straight back to confirm they’d tried my code and it works – so I’m sure it’s something minor – more on that later but the site is worth a look.

Right now I’m playing with http://dweet.io and http://freeboard.io

So Dweet is a service that again your micro can call directly without major overhead – it will store your data for up to a day and is an intermediate service. Freeboard consumes that data and…..

tmp8B5C

So this is not in the same league as Grovestreams BUT it’s a dashboard… imagine you had a bunch of gadgets all connected to the Internet and you wanted to see data from them all on one screen – if that’s the case then Freeboard and Carriots might be for you.

And now onto EXOSITES (www.exosites.com) – they have a very nice interface – when I originally wrote this article I was having lots of problems connecting to them – their examples for Arduino used STRINGS which I’ve never trusted (it is SAID that in Arduino string library for 1.5 (beta) they are improved but for now I’m sticking with the official release) and their examples would not send data to their server reliably.  I wrote to technical support – who agreed with me about strings – and they’d also made available a version using character arrays to send string data. I tried that, it still didn’t work – however I have to say after a series of email conversations, last night they put up a version of their code that works! It has the STRANGEST issue in that sending data to them fails every 100th attempt – at this point I’ve no idea if it is their end or mine but I’m more than happy with the technical support up to now and I’ve just sent the entire evening’s logs off to them to look at – with absolutely reliable failure every 100th send… here’s the current library. https://github.com/exosite-garage/arduino_exosite_library

So – what do you do and what do you get with Exosites-  they provide a REALLY simple to use library – basically you have a KEY  – and you send data sets along the lines of X=Y – that’s it really. A mere few lines of simple code – it’s all done in the library. It’s fast and efficient and you get back a copy of what you sent for confirmation!

tmpE5DD

The graphs are not in the same league as Grovestreams nor is the control – BUT there again they don’t put a limit on the amount of data in the free account and the dashboard is nice – and works on mobile phones and tablets – you’ll need to decide for yourself what is best for you. Right now I have high hopes that last tiny niggle of data failing will be resolved and I’ll be more than happy with this.

More as I go on. Someone suggested this morning I take a look at PLOTLY.. at first sight I could not make head nor tail of this – it’s in BETA and no-where near as simple or straightforward as above – but you might want to take a look. https://plot.ly

Oh, along the way, another kind co-respondent pointed out that one of my images (I think Freeboards) uses something called high-charts… now, if you need charts for personal use – this looks THE BUSINESS – http://www.highcharts.com/products/highcharts I had a quick look at you download the library with a TON of examples.. and they really look GOOD. Indeed – all I needed was for some kind soul to put Highcharts together with an Arduino – and SD card.. and VOILA.. http://everettsprojects.com/2012/12/31/arduino-super-graphing-data-logger/  I MUST STRESS I have not tested this (give it time) – I have the sketch in front of me – it looks reasonable – but that doesn’t mean it will work.

Pete.

Samsung S6D02A1-based LCD colour display and Arduino

Samsung S6D02A1-based LCD colour displayThis particular display is available from Banggood and Ebay generally – and looks like the photo on the left. Ignore the larger connector – the small one on the right consists of power, ground, LCD backlight and 5 control signals.

To make these work with Arduino-type boards, you need to hook the power to +5v, the LCD backlight to +5v (or an output – you can effectively control brilliance with a PWM output) and ground – then via 1k series resistors – hook up the 3 control signals – detailed in the various libraries.

The video details how to get this working using software out there with slight modification. I now have an endless source of fonts available to me thanks to the efforts of others and some tweaking. I’d love to hear from anyone who further speeds this up or indeed anyone who manages to find a really decent LCD 7-segment font that works well. The one I have is “ok”.

Anyway, it’s all described in the video below.

Using Samsung S6D02A1-based LCD colour display with Arduino