I have a puzzle to solve….
In my ideal ESP8266 world,. I will have a board that spends it’s life as a TCP/IP listener – and that will talk to an APP or another board (read on). And that’s fine and that works.
However, when the board powers up it will have to go get the time – and it may log something with a server somewhere – and that’s fine TOO – all of that works.
The problem – it may not BE a problem but it’s got me… is how to do both at once. i.e. I want to continue to listen – while calling a routine to go check the time or log something – and you have to change MODE for that and when you’re done you can change BACK… but your TCP/IP listener is now defunct.
One could be silly and use TWO ESP-01 boards… but surely there’s a better way?
Any Ideas anyone?
My Tech Weblog 
Hi Peter,
I am really enjoying to be part of your journey and hope to catch up with your progress soon.
If I look at the ESP8266 block diagram on the Espressif data sheet, the system is half duplex, implying that simultaneous TX and RX is not possible, either TX or RX at a time as indicated by the antenna switch. It would thus appear as if the MODE switching you refer to is the only option as a type of a slow Time Division Multiplexing System. I have concerns that having two modules one as listener and the other as talker will lead to interference problems even if different WiFi channels are used.
It would be interesting to determine how fast the module could be switched between the two modes including the processing time in order to determine the “dead” time when the listener will be inactive and miss something. Maybe a message acknowledge system can be implemented to ensure a re-transmit if a message is not acknowledged due to being in talker mode.
Hi Peter,
How about the server actually send the data to the ESP instead of the ESP requesting the data from the server? something like a cronjob and a php page acting like tcp client and send the data every x min or hours. In that case you wont need to change the ESP mode and it can keep listening forever.
Well, now that’s what I like – a little lateral thinking… I’ve no idea how to use Chron being a died-in-the-wool Windows user but I can do PHP… erm, I might need the odd pointer there.. but more importantly, you may be onto something as the other end of this is logging data. If this server were to send out a series of requests to get the logging data – or one long one getting some kind of regular package back, like {t1=23;t2=24;t3=20} erm, well, it could then send the data off to whoever is graphing the data (I don’t want to do that myself as I’ll never be as good as services like Grovestream – who for free have been graphing my Spanish cave temperatures for a year.
You know I started this reply being a tad negative but that’s a damned good idea, Sameh… Hmmmm.
That does leave ONE un-answered item which I’d not put in the article – if I do away with internal data, lets say I have 4 or 5 of these around the house – it would be nice to have one of them as the point of contact and the rest talking to each other – which takes us back to being listener and client at the same time. So for example in the bedroom I might have one which is mainly showing temperature.. and another at the other end of the house which has a relay attached to the heating – I really want, when the temperature gets too low, for the bedroom unit to tell the other unit’s relay to turn on……(and not via a timed job using a server which might take some time) and for others to do the same, running chron might also be an issue – in my case I pay for web space with CPANEL so chances are I can use Chron or similar, others may not be so lucky.
Still that IS something to ponder and thanks for the idea…any comments on the above welcome (please don’t say Raspberry Pi – I’m trying to avoid that).
Similarly remotely I might be talking to a unit and ask it to turn a light on in a different part of the house – now, I could just bring the lot to the outside world…. htmm there’s another possibility…instead of a web page have one internal unit who’s sole job is to poll the lot for jobs to do – and also to collect time and other info to send around… more messy but do-able..
Hi Peter,
For the CronJob it is very similar to Task Scheduler in WIndows, you are just telling the system to open this php page every x min.
If you are looking for a fully automated system then how about:
– A base station ( Arduino Ethernet ) connected using web sockets to your online or local server.
– Several nodes around the house listening to TCP.
– The base station will be able to have 2 way communication with the server, send data, receive data or doing whatever.
– Once the server receive command from the Base Station it will open tcp connection with that node and turn thing on and off.
That’s a quick idea on how to do it using the ESP, but why use ESP on nodes while they don’t need internet connection? why you don’t just use nrf24l01?
The base station get the data and communicate directly and privately with the nodes by using RF24network.
Oh that’s an easy one to answer – I can say quite categorically because the NRF24L01s are horrible – I’ve struggled with them for a couple of years, learning all the tricks about decent decoupling, adequate power etc and they can still hardly make it through a stone wall. The ESP8266 idea is to get away from that – I have 3 properties all of which have adequate WIFI infrastructure – because I needed that anyway – so it makes sense to leverage that instead of planting otherwise useless NRF relay stations. Also the NRFs are unable to transmit and receive simultaneously – which means in heavy traffic they can lose stuff – the Radiohead library can guarantee node to node reliability only – but NOT end to end reliability, they can’t return signal strength so one of the advantages of a network, picking the best available route, is beyond the capability of an NRF-equipped Radiohead library. I was about to move lock stock and barrel to the little Si4332 modules you see here http://www.scargill.net/home-control-september-2014-update/ but when the ESP8266 came along I saw this as an opportunity to have one wireless unit on a board which would do the lot… there is of course currently a question mark over that hence these discussions 🙂
To be honest I love the ESP specially for its price, but I don’t see it mature enough to work standalone. still what I am sure of is that we are going to see 100s of firmwares soon for this little thing. you know what is my biggest wish? to have real 2 way socket communication in ESP.
You know what would this mean? all nodes will listen to the socket server and the server can control all of them without even a need to know their IP! which means you will be able to control any ESP online from anywhere which also means all ESPs will be able to talk to each other and the server will just pick the one to talk to.
Yes absolutely – I’m with you – not seeing it though…….
Hi Peter,
Sameh’s on the right track.
By occupation I’m primarily a ‘web’ programmer – backends, web applications, server-side processes – and I’ve also done a little with some custom hardware on networks. To me, the network dialogue is usually some sort of server/client interaction. Server sits waiting for something, client makes a request (with params or data within the request), server replies with data. Lather, rinse, repeat.
The key is that both the request and the response carry data.
Now, I’m about a week or m ore behind you in terms of hands-on fun with the ESP8266. (I’m busy at work with pushing a years worth of sales data into Quickbooks… yuck). I don’t know enough about your application, but assuming that the ESP8266 is part of (or the heart of) a standalone remote sensor that transmits data back to somewhere, I would do the following:
– the remote sensor is the client, the central control is the server. Whenever the remote sensor has power, it makes periodic calls (HTTP requests) to a url (the central server). The call contains the sensor data (GET or POST, GET is easier), The server responds with what is essentially an acknowledgement, but which can also contain instructions to the sensor, such as time/date, or a configuration change.
This takes the burden off of the central server to have to poll for the sensors, and means you don’t have to run a ‘server’ on every remote sensor.
Hope this helps, and like the others I’ve enjoyed sharing your journey with the ESP8266. (But, for heaven’s sake, treat yourself to a book on Linux and the command line, for Christmas. The IoT will not be running on Windows. 😉 )
Cheers
Thank you for that, worthy of considering – oh, IOT – well, mine will be running on 1284 chips in C# – and Arduino libraries – I’m well versed in most of them
I’ll come back to you on this – rather a large backlog of responses (for which I’m grateful) to catch up on before I go getting the soldering iron and keyboard out 🙂
The only issue I have with that is – when my APP on the phone is turned on – I want real time info coming on – time, various settings, status of lights etc, so “periodic” Right now I talk to a board which is connected by Ethernet – and if I ask it to give me the status of interna device X, it does that via short range radio.. I get responses back within say 20ms. Not sure how you’d manage that with above.
<>
It depends on the nature of your data. Info like the typical measurements and statuses of a home environment/alarm system can be automatically received and stored on a modest central server/controller and the remote sensors are faithfully sending updates to this server at a reasonable interval (eg a room temperature, and the update is every two minutes). Then when you hit the central unit’s server with your laptop or phone app, you’ll immediately get the last cached data from the server, together with its timestamp.
If the data are more volatile, or you don’t have a central server/controller in the picture, a different strategy would be necessary.
In 2012 I worked on a project that involved IP cameras and network-enabled card-readers. The card reader used exactly the network model I am discussing: It was an HTTP client that sent power-up and ‘heartbeat’ requests to a configured URL at a programmed interval, and when a card was swiped, it immediately transmitted card data in a request. In the server response you could include commands to light a LED, sound a buzzer or trip a magnetic catch, or even reconfigure the reader.
Pete,
Maybe I don’t understand the problem that you’re having…
lua CAN listen and talk at the same time?!
Here is the lua script that illustrates listening for requests on port 80 AND sending on port 3000:
https://github.com/hwiguna/g33k/blob/master/ArduinoProjects/RF/ESP8266/NodeMCU/Lua/WebServer02.lua
A bigger issue for me was how to affect a web page in a browser by something that happens on the ESP. I spent some time learning about NodeJS a few weeks ago, but I finally got it working.
NodeJS has socket.io that allows two way communication between the NodeJS server and the webbrowser.
As you can see in the video, the ESP is completely wireless. You can press a physical button on it, and the webpage reacts. And the clicking a buttons on the same webpage controls an LED on the ESP.
As Sameh said, there is still a lot of work to be done before we a home automation. but I was happy with this proof of concept. Furthermore, a central server (NodeJS) can mediate communication between ESP(s) and ESP(s). Using WiFi as a reverse mesh network 🙂
I’m not using Lua – I’m using the AT command set – the Lua setup was getting flaky – the more things I put together the less reliably they ran so I put it on the back boiler. I’ll take a look at your video and code.. ta.
Sorry to post another reply. WordPress does not handle youtube playlist index correctly and it showed the first instead of the fifth. Here is the correct video.
Hari,
That is very impressive! I have tons of questions for you 😀
– How did you manage to connect to the nodejs socket.io server? are you using custom firmware or using LUA?
– Did you manage to do the handshaking and all these stuff?
– You’ve mentioned in your video that the server and ESP should be on same network, why is that? the beauty of web sockets let you make the communication from anywhere on planet as long you can access the socket server cause mainly the server is the one who send and receive the data.
Man I am so excited to hear back from you, this seems like what I’ve exactly wanted!
Sameh,
ESP is running stock NodeMCU lua firmware. It is NOT involved with socket.io. Only the webbrowser and NodeJS knows about socket.io.
WebBrowser to ESP:
WebBrowser sends requests to NodeJS server which in turn sends a webrequest to the ESP webserver.
ESP to WebServer:
ESP sends request to NodeJS server which in turn uses socket.io to talk to WebBrowser.
I think that should quell your excitement and answer your questions. It is not exactly what you wanted, but it does what I wanted (hopefully what you wanted as well) 😉
aaaaaah yeh I got you now 😀
I thought you are talking directly to the web socket server from your ESP and the ESP is acting as a socket client.
Great work tho, I can’t wait to get my ESPs to start playing with them, I even started to learn how to establish a true socket connection and see the possibilities to integrate it with the mighty LUA
Thanks for the info Hari 🙂
Sameh,
When you make progress on your ESP, please email me or comment on my youtube channel.
My email is the typical first letter of first name followed by lastname at gmail dot com.
I’d be very interested in a native ESP8266 websocket! 🙂 or anything else ESP8266 related for that matter.
Sure, I will keep in touch with you 🙂
This is beginning to get worse… using the AT Command mode, I was considering the idea of making all the units listeners capable of up to 5 connections or thereabouts (CIPMODE=1)
But hree’s the rub – I’m picking up the ID – and sending it back out – all of that is fine – but when I tried my app, even with polling slowed down to 350ms (which is a bit obvious) – put one mobile App on – it connects to ID 0, turn that of, turn the tablet on same APP, it connects to ID 1. Wonderful – but under NO circumstances will it run both at once!!! Has anyone tried this – I need the unit to be talking to an APP – and perhaps also answer the odd query about temperature at the same time – surely that is the point of the multiple connections??
Nice article in using an ESP8266 with a publicly-available ‘IoT server’.
http://java.dzone.com/articles/tutorial-iot-datalogger
This is a fully-formed example of what I was trying to outline up-thread.
That’s a great article – I tried several and personally I found the best to be Grovestreams who’ve been tracking temperature and humidity at one of my properties since early summer without a hitch. I guess I should do a write up on the subject..