Give me a Lua razor blade

After suddenly realising I could make a whole complicated APP based on a single Lua timer (even though today’s Lua update which I’m using can handle 7 timers), I realised that the only thing that stopped it being practical was the lack of non-volatile variables in the current LUA interpreter for ESP8266.

My first thoughts were….  create a global variable file (which would be small),  read  it during INIT  then when something needs saving – write to another file making updates and then when confirmed done, delete the original and rename the new file.. NO, SORRY – no rename facility in this Lua.

But as you’ll see that’s only part of it. Then I thought – ok I REALLY don’t want to use up RAM but I will – so we’ll read the file into an array (or two arrays in this case as I didn’t want to get TOO clever) – (hope the power does not go off and delete the original) – write the array to a new copy of the globals file and BOBS YOUR UNCLE.

But NO, after having SO much success with my little APP – TWO of us with completely different setups have FAILED to get this to work – I can’t even get past line TWO!!

The idea to call the file “globals.lua” is mine – and if we ever get past this point I’ll detail fully WHY this is needed and how my app can form the basis of your next App!!! But for now…

file.remove(“globals.lua”)
file.open(“globals.lua”,”w”)
file.writeline(“test1=1”)
file.writeline(“test2=2”)
file.writeline(“test3=3”)
file.close()

    newitem=”test2″
    newvalue=”AHAH!!!!!”
   
    counter=1
    items={}
    values={}

    file.open(“globals.lua”,”r”)
    x=file.readline()
                while x~=nil do
                items[counter],values[counter]=string.match(x, “(%w+)=(%w+)”)
                if items[counter]==newitem then values[counter]=newvalue .. “\n” end
                 x=file.readline()
    end
    file.close()
    file.remove(“globals.lua”)
    file.open(“globals.lua”,”w”)
    counter=1
    while items[counter]~=nil do
        file.writeline(items[counter] .. “=” .. values[counter])
    end
    file.close()
   
    items=nil
    values=nil
    counter=nil

dofile(“globals.lua”)

print(test1)
print(test2)
print(test3)

Clearly – the result should be 3 lines showing 1, AHAH!!!!! and 3. So the second line has been updated.

But I’ll bet you can’t get that far.

WHY??? – my friend’s system bombs half way through – while I’m running a simple temperature sensor and TCP/IP link in the background with no less than 13k of RAM available, I can’t get past line two!!!!!

16 thoughts on “Give me a Lua razor blade

  1. Hi Pete,
    I may have a simpler solution for you.
    I discovered pcall(loadstring(x)) in the nodeMCU manual. It executes the string as if we typed it in!
    You can adapt the code to save an array instead of discrete variables depending on how many variables need to be saved.
    The snippet below worked for me. I even tried turning off the power on the ESP just to be sure 🙂

    — Sample variables —
    a=7
    b=8
    c=”Hello”

    — Write data to a file —
    file.open(“data.lua”,”w”)
    file.writeline(“a=”..a)
    file.writeline(“b=”..b)
    file.writeline(“c=\””..c..”\””)
    file.close()

    —————————————

    node.restart()

    —————————————

    — All gone! —
    print(a)
    print(b)
    print(c)

    — Read data from a file —
    — pcall(loadstring(x)) executes the string as if we typed it! —
    file.open(“data.lua”,”r”)
    repeat
    line = file.readline()
    if line ~= nil then pcall(loadstring(line)) end
    until line == nil
    file.close()

    — We’re back! —
    print(a)
    print(b)
    print(c)

    • Hari

      I THINK you can achieve the same as the second half of your code with this..

      dofile(“data.lua”)

      Right now my big problem is that I can’t even remove or open a file without the board crashing 🙂 I don’t know what I’ve done – maybe my program is too big – but as we have no way to see how much FLASH memory is left…. I can’t tell.

      • When board crashes on file open or file remove then you have a corrupt file in there and need to reflash the esp. Happens to me a lot while developing. File can corrupt during writing process for some reason.

      • I hope Zeroday is reading this… I’ve just sent him this – an attempt to write to a file, read it back, changing one line – and firing back into the file – it dies and results in a reboot – which then won’t let you even delete a file. There could well be a bug in the code below – but even if there is – it should not kill the operating system… want to give it a go and see if it will mess up your board (temporarily of course)

        file.remove(“mtest.lua”)
        file.open(“mtest.lua”,”w”)
        file.writeline(“test1=1”)
        file.writeline(“test2=2”)
        file.writeline(“test3=3″)
        file.close()

        newitem=”test2″
        newvalue=”Rubbish”

        counter=1
        items={}
        values={}

        file.open(“mtest.lua”,”r”)
        x=file.readline()
        while x~=nil do
        items[counter],values[counter]=string.match(x, “(%w+)=(%w+)”)
        if items[counter]==newitem then values[counter]=newvalue .. “\n” end
        x=file.readline()
        end
        file.close()
        file.remove(“mtest.lua”)
        file.open(“mtest.lua”,”w”)
        counter=1
        while items[counter]~=nil do
        file.writeline(items[counter] .. “=” .. values[counter])
        end
        file.close()

        items=nil
        values=nil
        counter=nil

        dofile(“mtest.lua”)

        print(test1)
        print(test2)
        print(test3)

    • Zeroday

      Thanks – I found a solution to splitting the line up – it’s in the last blog item – and as for your suggestion of having two data files and a pointer – that is excellent and will indeed do the job…

      But I have a far more serious issue and I am out of ideas. I managed to get a nice, simple TCP/IP listener running – and it reads the temperature and makes it available to an Android App. It has been running for many hours, powers up reliably and runs reliably… and amazingly a test indicates nearly 14K of memory left.. BUT I cannot get past this point… watch… on the serial console..

      a=7
      b=8
      c=”Hello”

      — Write data to a file —
      file.open(“data.lua”,”w”)
      file.writeline(“a=”..a)
      file.writeline(“b=”..b)
      file.writeline(“c=\””..c..”\””)
      file.close()

      That’s my input….
      This is the result…

      print(“Hello”)
      Hello
      > print(node.heap())
      14072
      > a=7
      > b=8
      > c=”Hello”
      > file.open(“data.lua”,”w”)
      c_??RS?fJS?fJS?f?
      NodeMcu 0.9.2 build 20141209 powered by Lua 5.1.4
      Pete’s Appliance 1.0
      > Already have connection to192.168.0.7
      Starting TCPIP Listener
      Setup done

      Despite apparently having 14k of memory left, attempts to open a file result in the module crashing. My friend has different software running but he, too is getting similar results.

      I can run this repeatedly – and it is always the same – when I come to open the new file – the board simply reboots.

      Hence I cannot try out your idea…

      I cannot even remove INIT.LUa

      input:
      print(“testing”)
      file.remove(“init.lua”)

      result:
      print(“testing”)
      testing
      > file.remove(“init.lua”)
      c_??RS?fJS?fJS?f?
      NodeMcu 0.9.2 build 20141209 powered by Lua 5.1.4
      Pete’s Appliance 1.0
      > Already have connection to192.168.0.7
      Starting TCPIP Listener
      Setup done

      Do you have any ideas? I can send complete code if that helps but I don’t have your email. Mine is pete at scargill dot org

  2. Yes it did and now we know why. Frighteningly easy to crash….. but at least now I know it’s not my TCP/IP code overflowing – great – I’m onto the next stage. Thanks for replying and your help.

  3. Right I have a function – just got to document it – incidentally I was thinking of asking you for a 1 second clock as the microsecond clock wraps WAY too quickly – but I’ve just realised – you’ve just given us one in the 7 timers – one of them can presumably be used as a reasonably accurate clock given we can set the time from some website (I’m still having trouble with that… given a 1 second time, making a set of variables second, minute, hour, day should be trivial!

  4. I think I have another problem for Zeroday… in order to determine whether or not to write the init files I need to know if they exist already… and if you call a file that does not exist, rather than NIL you get an error message….

    So how can I programmatically tell if a file already exists?

Leave a reply to Scargill Cancel reply