ESP8266 and the Dallas DS18B20 and DS18B20P

Please note – this information has now been superseded by the info on the new blog at http://tech.scargill.net – please visit the new blog.

Having wasted half the night following up dodgy temperature readings for the DS18B20P only to realise I’d left existing code running for the DHT22 !!!  I managed in the process to reduce the necessary code down to a minimum.

If you look at a previous blog in here I referred to some simplified Arduino code I’d written back in the dark ages and wondering if I could get similar here.  I’m only interested in temperatures to the nearest degree C for thermostatic control and was very concerned about the typical 1 second delay to read these chips.

I reasoned that if you were to read the chip before conversion then start a conversion off, your first reading would be rubbish – but subsequent readings would be fine without the delays associated with the more normal code.

So, here is what I ended up needing for a library – very little as you’ll see… most of it is comments…

 

/*
* Adaptation of Paul Stoffregen’s One wire library to the ESP8266 and
* Necromant’s Frankenstein firmware by Erland Lewin <erland@lewin.nu>
*
* Paul’s original library site:
*   http://www.pjrc.com/teensy/td_libs_OneWire.html
*
* See also http://playground.arduino.cc/Learning/OneWire
*
* Stripped down to bare minimum by Peter Scargill for single DS18B20 or DS18B20P integer read
*/

static int gpioPin;

void ICACHE_FLASH_ATTR ds_init( int gpio )
{
//set gpio2 as gpio pin
PIN_FUNC_SELECT(PERIPHS_IO_MUX_GPIO2_U, FUNC_GPIO2);
//disable pulldown
PIN_PULLDWN_DIS(PERIPHS_IO_MUX_GPIO2_U);
//enable pull up R
PIN_PULLUP_EN(PERIPHS_IO_MUX_GPIO2_U);
// Configure the GPIO with internal pull-up
// PIN_PULLUP_EN( gpio );
GPIO_DIS_OUTPUT( gpio );
gpioPin = gpio;
}

// Perform the onewire reset function.  We will wait up to 250uS for
// the bus to come high, if it doesn’t then it is broken or shorted
// and we return;

void ICACHE_FLASH_ATTR ds_reset(void)
{
//    IO_REG_TYPE mask = bitmask;
//    volatile IO_REG_TYPE *reg IO_REG_ASM = baseReg;
uint8_t retries = 125;
// noInterrupts();
// DIRECT_MODE_INPUT(reg, mask);
GPIO_DIS_OUTPUT( gpioPin );
// interrupts();
// wait until the wire is high… just in case
do {
if (–retries == 0) return;
os_delay_us(2);
} while ( !GPIO_INPUT_GET( gpioPin ));
// noInterrupts();
GPIO_OUTPUT_SET( gpioPin, 0 );
// DIRECT_WRITE_LOW(reg, mask);
// DIRECT_MODE_OUTPUT(reg, mask);    // drive output low
// interrupts();
os_delay_us(480);
// noInterrupts();
GPIO_DIS_OUTPUT( gpioPin );
// DIRECT_MODE_INPUT(reg, mask);    // allow it to float
os_delay_us(70);
// r = !DIRECT_READ(reg, mask);
//r = !GPIO_INPUT_GET( gpioPin );
// interrupts();
os_delay_us(410);
}

//
// Write a bit. Port and bit is used to cut lookup time and provide
// more certain timing.
//
static inline void write_bit( int v )
{
// IO_REG_TYPE mask=bitmask;
//    volatile IO_REG_TYPE *reg IO_REG_ASM = baseReg;
GPIO_OUTPUT_SET( gpioPin, 0 );
if( v ) {
// noInterrupts();
//    DIRECT_WRITE_LOW(reg, mask);
//    DIRECT_MODE_OUTPUT(reg, mask);    // drive output low
os_delay_us(10);
GPIO_OUTPUT_SET( gpioPin, 1 );
// DIRECT_WRITE_HIGH(reg, mask);    // drive output high
// interrupts();
os_delay_us(55);
} else {
// noInterrupts();
//    DIRECT_WRITE_LOW(reg, mask);
//    DIRECT_MODE_OUTPUT(reg, mask);    // drive output low
os_delay_us(65);
GPIO_OUTPUT_SET( gpioPin, 1 );
//    DIRECT_WRITE_HIGH(reg, mask);    // drive output high
//        interrupts();
os_delay_us(5);
}
}

//
// Read a bit. Port and bit is used to cut lookup time and provide
// more certain timing.
//
static inline int read_bit(void)
{
//IO_REG_TYPE mask=bitmask;
//volatile IO_REG_TYPE *reg IO_REG_ASM = baseReg;
int r;
// noInterrupts();
GPIO_OUTPUT_SET( gpioPin, 0 );
// DIRECT_MODE_OUTPUT(reg, mask);
// DIRECT_WRITE_LOW(reg, mask);
os_delay_us(3);
GPIO_DIS_OUTPUT( gpioPin );
// DIRECT_MODE_INPUT(reg, mask);    // let pin float, pull up will raise
os_delay_us(10);
// r = DIRECT_READ(reg, mask);
r = GPIO_INPUT_GET( gpioPin );
// interrupts();
os_delay_us(53);
return r;
}
//
// Write a byte. The writing code uses the active drivers to raise the
// pin high, if you need power after the write (e.g. DS18S20 in
// parasite power mode) then set ‘power’ to 1, otherwise the pin will
// go tri-state at the end of the write to avoid heating in a short or
// other mishap.
//
void ICACHE_FLASH_ATTR  ds_write( uint8_t v, int power ) {
uint8_t bitMask;
for (bitMask = 0x01; bitMask; bitMask <<= 1) {
write_bit( (bitMask & v)?1:0);
}
if ( !power) {
// noInterrupts();
GPIO_DIS_OUTPUT( gpioPin );
GPIO_OUTPUT_SET( gpioPin, 0 );
// DIRECT_MODE_INPUT(baseReg, bitmask);
// DIRECT_WRITE_LOW(baseReg, bitmask);
// interrupts();
}
}
//
// Read a byte
//
uint8_t ICACHE_FLASH_ATTR ds_read() {
uint8_t bitMask;
uint8_t r = 0;
for (bitMask = 0x01; bitMask; bitMask <<= 1) {
if ( read_bit()) r |= bitMask;
}
return r;
}

and here is the code that ended up in my timer callback routine, sending off an MQTT message for the temperature every 5 seconds – this works with both chips and works reliably.

    ds_init(2); // one off for the DS18B20

The above goes in my INIT routine – note that I’ve referred to GPIO2  – this will not work for other port bits as the Port initialisations seems to be partly hard coded – if anyone cares to re-write that so that anyone can easily access any of the usable pins.. would be nice..

I’ve also renamed the  routines, adding ds_ at the start to avoid confusion with other routines.

And now  for the actual code.. I call this every 5 seconds for testing and as you’ll see there’s a little extra code to send this out via MQTT. The extra code is in bold. “Temperature” is an integer defined elsewhere.  As you can see, no search, no checksum, no delay for processing – this works with both the DS18B20 and the DS18B20P chips.

char *tBuf = (char*)os_zalloc(6);
ds_reset();
ds_write(0xcc,1);
ds_write(0xbe,1);

temperature=(int)ds_read();
temperature=temperature+(int)ds_read()*256;
temperature/=16;
if (temperature>100) temperature-=4096;
ds_reset();
ds_write(0xcc,1);
ds_write(0x44,1);
os_sprintf(tBuf, “%d”, temperature);
MQTT_Publish(&mqttClient, “temperature”,tBuf,strlen(tBuf), 0, 0);
os_free(tBuf);

And so how to test all of this. For the life of me I can’t find a simple, no-nonsense program to read MQTT and graph it – you would think there would be loads around – I have asked the writer of MQTT-SPY if he’d like to add something into the program – for what’s out there – you seem to have to sign up and go through a whole learning curve – a little excessive for what I wanted (i.e. MQTT in – graph out), so I hooked up Node-RED to simply store the incoming data in a file.

node-red
And now having that information in a simple text file, it was easy to graph in EXCEL but again a whole FAFF to simply update dynamically until I discovered Live-Graph – not the prettiest but set to read the file every 5 seconds and update the graph –  I have to say it works a treat.

Here is my little sensor, as it happens the P variety I was originally having problems with, happily monitoring the ambient air temperature with a little excitement added on two occasions from my Dust-off air blower which is able to freeze the life out of the sensor on demand!

plot[3]

As you can see, the code is working well – doesn’t even need averaging.

Advertisements

MQTT for ESP8266 not QUITE but nearly

Minh Tuan is doing an excellent job with MQTT – he has it working stand alone – and in the LUA code (not yet brought into the main code)… but..

Before you go off getting annoyed if things don’t go quite to plan, there still seems to be a tiny but important bug  -I’ve reported it today and Minh will look at it tomorrow. I’m working on code from just an hour ago here (late after noon UK)

I set up a demo, subscribing to 7 short topics. With the exception of the first, all the topic names are short… and all responses are short…. I’ve changed the broker name below to protect the innocent (that would be me).

MQTT: subscribe, topic"home/openHAB/out/Light_FF_Bath_Ceiling/command" at broker xxx.yyy.zzz:1884
MQTT: subscribe, topic"myleds" at broker  xxx.yyy.zzz:1884
MQTT: subscribe, topic"mytime" at broker home.scargill.org:1884
MQTT: subscribe, topic"time" at broker xxx.yyy.zzz:1884
MQTT: subscribe, topic"dusk" at broker xxx.yyy.zzz:1884
MQTT: subscribe, topic"dawn" at broker hoxxx.yyy.zzz:1884
MQTT: subscribe, topic"timestring" at broker xxx.yyy.zzz:1884

As you can see all of them subscribed successfully or so it would seem.

Ah, but…

4 of them are broadcast every minute from a page I demonstrated in a previous blog or it’s notes and over on another computer I have MQTT spy watching… time, dawn, dusk and timestring are send out by the same page once per minute.

But what is being seen by the little ESP8266 program is….

TCP: data received
MQTT topic: time, data: {1484956800}
TCP: data received
MQTT topic: timestring, data: {15:15 Saturday 17-01-15}
TCP: data received
MQTT topic: time, data: {1484956800}
TCP: data received
MQTT topic: timestring, data: {15:16 Saturday 17-01-15}

As you can see, dawn and dusk have “disappeared into the dusk”.

Similar happens with the Lua/MQTT code – I thought at first that might be due to the callback not being done before being called again but seeing this I’m thinking… no.

As soon as this one is resolved I’ll report back. It is important that the system can handle several simultaneous incoming messages and that it it reaches it’s limit, something obvious tells us that.   The info above is the minimum I would be sending to any module –ok of course I could merge them into one – but that’s not the point – I could not do that if they were coming from different sources.

Suggest looking in again tomorrow for any fixes…

If I go quiet, I’m off to Brussels for a couple of days for some EU business.

And there’s more… I reduced the number of subscriptions to 3..

MQTT: subscribe, topic"dusk" at broker xxx.yyy.zzz:1884
MQTT: subscribe, topic"dawn" at broker xxx.yyy.zzz:1884
MQTT: subscribe, topic"timestring" at broker xxx.yyy.zzz.org:1884
MQTT: Sending..type: 8,qos: 4
TCP: Sent
TCP: data received
MQTT: Sending..type: 8,qos: 4
TCP: Sent
TCP: data received
MQTT: Sending..type: 8,qos: 4
TCP: Sent
TCP: data received
TCP: data received
deliver_publish
MQTT topic: timestring, data: {16:43 Saturday 17-01-15}
TCP: data received
deliver_publish
MQTT topic: dawn, data: {27208}
TCP: data received
deliver_publish
MQTT topic: timestring, data: {16:44 Saturday 17-01-15}
TCP: data received

 

Couple of things of concern – firstly again it’s SAYING it has subscribed to 3 but it only taking two of them – the last 2 (that’s not a clue, wasn’t last 2 last time) but the messages are also a concern… “Sending..type:8 qos:4”   there ISN’T a qos 4 !!! Don’t know what that is about…

ESP8266 the unstoppable March of Progress

esp01I’ve been following the progress of this little chipset as you know since day one – having spend a year more developing skills at using horrible little NRF24L01 radios with as much indoor range as my legs – and Arduino Ethernet cards (which to be fair are WONDERFUL other than their need for a marriage-endangering WIRE), out of the blue came a little Chinese board with the potential to change everything. The ESP-01 showed much promise but very little reality at first. The instructions were in Chinese, the documentation was in Chinese or missing, people told me “there are these other boards for only a tenner that do so much more”.

And yet here we are today with much of that about to change. The new so called white ESP8266 board with it’s development “kit” for less than a tenner is winging it’s way to me now – and the ESP-12 boards are neat and even have approval stickers to keep people happy not to mention a health range of I/O.

There are now a range of boards, all cheap  (there is a question mark over the ESP-07 – not for the first time I’ve heard there are 2 versions one of which is “suspect”) – and the ESP-01 remaining, I think true to my original statement – the title of one of my blogs “the cheapest computer in the world”. At under £2 tell me I’m wrong. 1 plug in the wall power supply , one ESP-01, one cheap solid state relay and you have a functioning WIFI-controlled device – with the right software that is.

But like all such devices, software makes or breaks this dream and in recent weeks we’ve seen the emergence of the ECLIPSE environment for Windows – I’m sure there are many such environments for Linux – and others will blog about them – I’m not interested in the command line and haven’t been for a long time. Eclipse and the tremendous work by the fellow who set this all up for us – means everyone from expert to complete beginner can “have a go”.

This week however is a bit special – we have in one corner TUAN and his MQTT code – on the other corner we have ZERODAY and his LUA interpreter – ( I really should find out everyone’s proper names) and this weekend – they look like merging.  I’m hoping that the MQTT code will continue to be developed independently of LUA as I think there are uses for both. I DO have code that WORKS – it’s not just a promise. We have some great tools such as Esplorer for testing Lua and the AT command set.

I can’t tell you what is possible with the software and hardware out there – all I can tell you is what I’m doing and why – I hope my blogs up to date have given you all the links you need. For days now I’ve been testing the MQTT software – and as far as I can tell there is only one “bug” left of note and it’s not even a bug – it’s an inconvenience – something to “wrapped up” – essentially the code works – but it’ not too happy about empty messages. I expect right now that is being fixed and I am so grateful to so many authors who have communicated, helped, changed things – in the main for no commercial gain – because they love it.. it’s almost like the days of the first 8-bit micros – the rush of learning.  Meanwhile I got an email this morning to say that MQTT and LUA were merging and a test set of .BIN files was available.

So, despite a head that is spinning like a TOP due to flu, this afternoon I set up a test rig…  let me take you through that quickly… erm… it works but read on as you need the bits that come before it.

The plot so far:

In order to use MQTT as the base of your control system (in my humble version of this vastly diverse Internet of Things) you need an “MQTT broker” running somewhere and it needs to be reliable – ROCK SOLID. In my case I have a Synology Diskstation (the link is an example – mine is not white) and with a little help I got the free MOSQUITTO running on it – you could put this on an old PC, a Linux box – just about anything with a brain – but it needs to be reliable. There are services out there – the free ones they generally warn you not to rely on them – so I’m not going to – I’ll have my own thanks – I’ve talked about several variations elsewhere – at the end of it I went with Mosquitto.  My disk backup DISKSTATION is on 24-7 so it seemed the ideal place for this. As for testing Mosquitto on a PC – you are not going to beat the EXCELLENT MQTT-SPY.

So now a place to fire messages to – and a place to receive them from… and the software tools for ESP8266 to send and receive those messages – the sky is the limit. But what about issues people have had – swapping modes – you might need the time from a time server for your little board or some other info. Well, I found a way around that.   MOSQUITTO is on all the time – what is needed is for it to make available info to the little boards, be it time, lighting up time, perhaps other information.  I found a PHP library – the nice kind – the type you don’t actually have to understand to use.  Some time ago I documented a web page I was using to fire back time and other information – I used it as I found NTP time servers to be slow at times.  I have cheap web space with a provider much as many of you will – with CPANEL – which means I get all the benefits of using a reliable Linux based web service while staying within my pretty coloured boxes visual interface comfort zone (I’m sure there’s an abbreviation for that).

With such a setup you can write PHP pages like one that will send time messages for example and with the ability to run those pages on demand using CRON (doddle) you can make the time and over information available to your toys.

Here’s the deal.

<?php
require("../phpMQTT.php");

$mqtt = new phpMQTT("xxx.xxxx.org", 1884, "somenamePub");
if ($mqtt->connect(TRUE,NULL,"mymqttuser","mymqttpass")) {
            $locn="Europe/London";
            $lon=55;
            $lat=-2;
            date_default_timezone_set($locn);
            if ($_GET[‘loc’]!="") $locn= $_GET[‘loc’];
            if ($_GET[‘lat’]!="") $lat= $_GET[‘lat’];
            if ($_GET[‘lon’]!="") $lon= $_GET[‘lon’];
            $dateTimeZoneLocal = new DateTimeZone($locn);
            $dateTimeLocal = new DateTime("now", $dateTimeZoneLocal);
            $localDateTime = date("H:i l d-m-y", time());
            $localTime=strtotime($localDateTime);
           
            $mqtt->publish("time",$localTime,0);
            $mqtt->publish("timestring",$localDateTime,0);
            $sun_info = date_sun_info($localTime, $lon, $lat);
            foreach ($sun_info as $key => $val) {
                if ($key==’civil_twilight_end’) $mqtt->publish("dusk",$val %86400),0);
                if ($key==’civil_twilight_begin’) $mqtt->publish("dawn",$val %86400,0);
            }
    $mqtt->close();
}
?>

No knocking my coding please – I do not profess to be a PHP expert – indeed – can you think of decent extra publications this page could produce for the little gadgets? If so lets have those ideas! 

This web page connects to your MOSQUITTO or other MQTT broker and every minute (every week if you want – my choice of every minute – gadget turns on – worse case it waits a minute to know the time) I am here publishing 4 items – the TIME in standard seconds-since-1970 format, British lighting up time (DUSK) in seconds since midnight, DAWN in seconds since midnight and TIMESTRING for those with no space in their project to format the time. ANY of my gadgets can choose to subscribe to these  publications – or not. And as you can imagine I’m planning more publications – timed an otherwise. The server load of the above is irrelevant – I don’t see anyone’s provider griping about this.

So – the people I’ve been with keep my web pages up and running 24-7 and and have done for as long as I can remember – why would I not trust this info.

And all of this brings us around to LUA+MQTT – possible only since this morning!!!

Check this out. Sorry it’s not formatted.

mqtt = net.createConnection(net.TCP, 0)
— init mqtt client with keepalive timer 30sec
mqtt:mqtt("myid", 30, "mymqttuser", "Tmqttpassmakeitagoodone")

— on publish message receive event
mqtt:on("receive", function(conn, topic, data) print(topic .. ":" .. data) end)
— on connection event (all event inherit from net module)
mqtt:on("connection", function(con) print("connected") end)
mqtt:connect(1884,"mymqttserver.address")
— subscribe topic with qos = 0
mqtt:subscribe("time",0, function(conn) print("subscribe success") end)
— publish a message
— mqtt:send("/topic","hello",0,0, function(conn) print("sent") end)

print (node.heap())

mqtt:subscribe("temperature",0, function(conn) print("subscribe success") end)

mqtt:send("dusk","hmm",0,0, function(conn) print("sent") end)

THAT which you see above and the code to log into your access point (which, once entered should stay there) is it – that code is all I had to put in – to have the little board be subscribing to time, dusk, dawn and temperature from another device (oh – a pair of minuses means comment – you need to see how to publish).

The REST is what you do with such info – Lua has timers (to keep that time going if you lose the internet connection) and GPIO control – to do things – already discussed previously – I’ve had a solid state relay running directly off the board and powering a mains lamp.

And that is it for now – the information isn’t here – it’s out there – check out the links here and in my previous blogs and bear in mind that before Christmas I knew NOTHING about compiling this stuff and I was determined determined not to get into Linux (and still successfully avoiding it)  – I’m now just about ready to openhabproperly embark on some real projects with real control in the comfort of my pretty-coloured Windows environment…  I hope that by putting all of this in one place gives you a head start.

For my next challenge – OpenHab but I think I’ll wait until the flu has gone – this one’s not going to be easy.

ESP8266 MQTT Publications

The PlanIn my last blog I referred to publishing temperature etc and there is one issue with the existing MQTT library that makes that an issue: The MQTT publications are not queued.

For the benefit of anyone still not into the MQTT thing – by publications I mean “sending a message” comprising a topic and a message. The topic might be “BathroomTemperatureReading” and the message might be “20.6”.

What happens right now – and Tuan has said he’ll fix this… if you send 2 messages out in a row IMMEDIATELY after each other – only the last one will go. Well, that’s fine unless you have to send one message off to a thermometer display and another message off to a humidity display.

So… this evening I wrote a queue. This is possible because of a “callback routine”. When you fire a message off it is handled in the background and when it’s done it calls an empty routine. I’ve trapped that.. the basic idea is this.  

Clear a “done” flag at the start and from there on..

If the done flag is clear, set it and send out a publication. If it is NOT clear, put the message in a queue.

In the callback routine, clear the flag – check a queue, if there is anything to go – send out the publication.

Simple really.

So here’s the code I’ve added to the start of the MQTT package..

#define QUEMAX 10  // 10 messages -  need more? – increase
#define QUESIZE 50 // my topics and messages are never more than 50 chars
struct Queues{
    MQTT_Client* cli;
    char top[QUESIZE];
    char mes[QUESIZE];
};

struct Queues myQueue[QUEMAX];

int queuein=0;
int queueout=0;
int isPublished=0; // my flag to see if I can publish or need to put in a queue

All simple enough – the queue can be maximum 10 deep (it’s rotary – when I get to the end I roll around to the start). I save the client pointer, the topic text and the message text in an array of simple structs.

At the beginning both the in pointer and out pointers for the queue are cleared and isPublished is clear.

Instead of the main publish routine – I have my own – which checks the flag before sending off a message.

void petesPub(MQTT_Client* cli, char *top, char *mes)
{
if (isPublished==0)    { isPublished=1; MQTT_Publish(cli, top,mes,strlen(mes), 0, 0); }
else
    {
    myQueue[queuein].cli=cli;
    strcpy(myQueue[queuein].top,top);
    strcpy(myQueue[queuein].mes,mes);
    if (++queuein>=QUEMAX) queuein=0;
    }
}

You can see the message going in the queue unless the flag is clear… and in the callback routine, called when a message has gone…

void mqttPublishedCb(uint32_t *args)
{
    MQTT_Client* client = (MQTT_Client*)args;
    INFO("MQTT: Published\r\n");
    isPublished=0;
    if (queuein!=queueout)
    {
        petesPub(myQueue[queueout].cli,myQueue[queueout].top,myQueue[queueout].mes);
        if (++queueout>=QUEMAX) queueout=0;
    }
}

The first two lines were there already – I don’t think I actually need to store the client info – I think it gets passed – however – what I have works for now.

So you can see I clear the published flag, look to see if there is anything in the queue and if there is – publish it. I’m sure there’s a bit of recursion going on there – but it all works.

And that’s me done for the day. Meetings again this week so I’ll not get much more done now until the weekend but DHT22, RTC and message queue – not a bad start for the week.

I’m still begging anyone with assembly skills to look at the WS2812B code to see if they can make an accurate job of it Smile

ESP8266 Memory Usage

debugGlen Cook has just made a neat little utility for checking the amount of RAM in Eclipse projects…  here’s the output – and there’s a link below – contact Glen, not me for more info.

Part of the problem of working with new processors – is understanding what is being used and where.  I look forward to seeing a visual version of this with used RAM, FLASH etc. and showing what’s used and what’s left. It would not be the first time I’ve run out of RAM.

http://www.betgear.biz/MemUsage.rar

Also linked to on my ESP8266 Facebook page.

Does anyone know the significance of the IRAM (initialised RAM) as in one project I’m down to 800 bytes – while having 38K of general RAM left.