Uploading a Lua script to NodeMCU using Wifi

Question

Is it possible to upload a Lua script to a NodeMCU using the Wifi interface instead of serial?

The tutorials and examples that I have found all use the serial interface, i.e. a cable, to program the NodeMCU, but I would like to change the program without connecting anything (using a smartphone or a browser)

Answer 1

another option today is FTPServer
in nodemcu lua modules list.

FTPServer Module

https://nodemcu.readthedocs.io/en/release/lua-modules/ftpserver/

But this module is too large to run in SPIFFS.

limits:

• need to be run in LFS
• only operate files in SPIFFS
• only plain ftp, no ssl
• only PASV mode

But it's still useful.

use module in LFS

1. setup LFS partition via lua api //if not config at rom build https://nodemcu.readthedocs.io/en/release/lfs/#configuring-lfs
2. build lfs image //at pc use luac.cross
   https://nodemcu.readthedocs.io/en/dev/getting-started/#minimal-lfs-example
3. upload .img to SPIFFS
4. flash .img to LFS partition
5. use module in LFS

operation command example:

------ steps at nodemcu cli: in ESPlorer or serial
-- setupt lfs partition (change value as you need)
-- this step may format SPIFFS

=node.setpartitiontable{ lfs_addr = 0x070000, lfs_size = 0x020000,
 spiffs_addr = 0x090000, spiffs_size = 0x36d000 }

#------ steps at pc
# got lfs.img
luac.cross -s -f -o  lfs.img ftpserver.lua telnet.lua

# upload lfs.img to Nodemcu SPIFFS via serial (or use ESPlorer)
nodemcu-uploader.py upload lfs.img

------ steps at nodemcu cli
=node.flashreload('lfs.img')

-- start server
=node.LFS.ftpserver():createServer('user', 'pass', false)

-- stop server
=node.LFS.ftpserver():close()

---

NOTE: this solution no reboot control (upload file but without apply it).
another telnet server can do it (has same cli as serial interface)

https://nodemcu.readthedocs.io/en/release/lua-modules/telnet/

Answer 2

I have another solution, that is free from size limits. Also it does not require any other web servers, you can send the file straight from your workstation. The file below provides for both uploads and downloads to/from the chip.

Edit: I developed the source examples below into a file manager, capable of uploading, renaming, backing up, deleting and of course serving files, running on an ESP8266 and ESP32 (WiFi connection differences are abstracted.) The project can be found here. (Depends on NodeMCU.)

Unfortunately it doesn't use the standard upload scheme used by web browsers, a javascript file that uploads to it is provided at the end. It is possible to create a shortcut to the js in the SendTo folder, thus adding it to the list of Send To options in the context menu for every file, but it will only be capable of processing single-file selections. (A shell extension would be needed to process multiple files selected.)

It does support regular browser downloads.

Note that this scheme depends critically on a certain XMLHTTPRequest convention, namely that the body of a POST is sent in second/subsequent frames following the request. If that were not the case the code would need to find the first \r\n\r\n in the payload of the initial request and append the data that followed to the file.

headerBlock = "\r\nContent-type: text/html\r\nConnection: close\r\nAccess-Control-Allow-Origin: *\r\nCache-Control: no-cache\r\n\r\n"
local currentFileName = ""
local isPostData = false
print("filexfer")
local srv=net.createServer(net.TCP, 60) 
srv:listen(80,
    function(conn) 
        local function writefile(name, mode, data)
            if (file.open("temp_" .. name, mode) == nil) then
                return -1
            end
            file.write(data)
            file.close()
        end
        conn:on("disconnection", 
            function(conn) 
                isPostData = false
            end
        )
        conn:on("sent", 
            function(conn) 
                currentFileName = ""
                isPostData = false
                conn:close()
            end
        )
        conn:on("receive",
            function(conn, payload)
                tmr.wdclr();
                local s, e, m, buf, k, v
                local tbl = {}
                local i = 1
                local retval = ""
            
                if isPostData then
                    writefile(currentFileName, "a+", payload)
                else
                    s, e = string.find(payload, "HTTP", 1, true)
                    if e ~= nil then
                        buf = string.sub(payload, 1, s - 2)
                        for m in string.gmatch(buf, "/?([%w+%p+][^/+]*)") do
                            tbl[i] = m
                            i = i + 1
                        end
                        m = nil
                        if #tbl > 2 then
                            local cmd = tbl[2]
                            if (tbl[3] ~= nil) and (tbl[3] ~= "/") then
                                currentFileName = tbl[3]
                            --else return an error
                            end

                            if (cmd == "put") then
                                writefile(currentFileName, "w+", "")
                            end

                            if (cmd == "append") then
                                isPostData = true
                            end

                            if (cmd == "persist") then
                                file.rename("temp_" .. currentFileName, currentFileName)
                            end

                            buf = ""
                            if retval == nil then
                                retval = "[nil]"
                            end
                            buf = "HTTP/1.1 200 OK" .. headerBlock .. retval
                        else
                            local filename = "index.html"
                            if tbl[2] ~= nil and tbl[2] ~= "/" then
                                filename = tbl[2]
                            end
                            require("fileupload")(conn, filename)
                            buf = ""
                        end
                        conn:send(buf)
                    end
                end
            end
        ) 
    end
)

This is fileupload.lua, referenced by the call to require near line 75 (upload because the chip is sending the file to the requesting host.) It facilitates downloading files of any size, using a regular browser. If no file name is passed it defaults to "index.html".

local module =...
    return function(conn, fname)
        local buf
        tmr.wdclr()
        if file.list()[fname] ~= nil then
            file.open(fname, "r")
            buf = "HTTP/1.1 200 OK" .. headerBlock
        else
            file.open("error404.html", "r")
            buf = "HTTP/1.1 404 FILE NOT FOUND" .. headerBlock
        end

        conn:on ("sent",
            function(sck)
                function sendfile(sck)
                    buf = file.read(255)
                    if buf ~= nil then 
                        sck:send(buf)
                    else
                        sck:close()
                        if module ~= nil then
                            package.loaded[module] = nil
                        end
                        module = nil
                        return
                    end
                end
                sck:on("sent", sendfile)
                sck:on("disconnection",
                    function(sck)
                        print("[disconnection fileupload.sendfile]", node.heap())
                    end
                )
                sendfile(sck)
            end
        )
        conn:on ("receive",
            function(sck, pl)
                sck:close()
            end
        )
        if buf == nil then
            buf = ""    
        end
        conn:send(buf)
    end

And this is a client-side javascript file used to upload to the chip. Pass the full or relative path of the file to be uploaded as its first/only argument. (If no args are passed it will throw an error.)

var filepath = WScript.Arguments(0);
var fso = new ActiveXObject("Scripting.FileSystemObject");
var str = fso.OpenTextFile(filepath, 1);
var file = fso.GetFile(filepath);
var filename = file.Name;
var buf = "";

var xhr = new ActiveXObject("MSXML2.XMLHTTP.6.0");
xhr.open("GET", "http://192.168.4.1/put/" + filename, false);
xhr.send();

while (str.AtEndOfStream == false)
{
    buf = str.read(255);
    xhr.open("POST", "http://192.168.4.1/append/" + filename, false);
    xhr.send(buf);
} 

str.close();
xhr.open("GET", "http://192.168.4.1/persist/" + filename, false);
xhr.send();

Answer 3

There is a variant of solution described above (http://www.instructables.com/id/ESP8266-WiFi-File-Management/) but desktop .NET application is used instead of PHP Web Server https://github.com/Orudnev/.Net-WiFi-File-Manager-for-ESP8266. It might be more convenient if you don't like to install web server, it is enough to launch FileManager.exe application.

Answer 4

Another approach if you use Esplorer as the IDE is documented here

It uses a telnet server on the ESP8266, and redirects the Esplorer serial port to the telnet server's address; It is a windows example, but I have managed to make it work using "socat" under linux.

The only problem I found is if you want more than one tcp server, which Nodemcu doesn't allow, in which case maybe another ESP8266 acting as a tcp/serial console relay might be the answer.

Answer 5

I upload all the modules over wifi. I first upload a bootstrap.lua program in the usual way (over USB). This program can then be used to upload the real (larger) payload. Here is the bootstrap program:

ip, mask, host = wifi.sta.getip()
port, path, pgm = 80, "/upload", "u.lc"
file.remove(pgm) ; file.open(pgm, "w+") payloadFound = false
local conn = net.createConnection(net.TCP, 0)
conn:on("connection", function(conn)
        conn:send("GET "..path.."/"..pgm.." HTTP/1.0\r\n".."Host: "..host.."\r\nConnection: close\r\nAccept: */*\r\n\r\n") end)
conn:on("receive", function(conn, payload)
        if (payloadFound) then file.write(payload) file.flush()
        else payloadOffset = string.find(payload, "\r\n\r\n")
                if (payloadOffset) then
                        file.write(string.sub(payload, payloadOffset + 4)) file.flush() payloadFound = true
                end end end)
conn:on("disconnection", function(conn) file.close() dofile(pgm) end) conn:connect(port,host)

The first line uses the gateway server as the web server from which programs are uploaded. The second line sets the port (80), path (/upload) and name (u.lc) of the program to upload. It then GETs the file and finally runs it (last line).

You must have your wireless connection active before running this, and your web server should be active of course, with your payload in /upload/u.lc.

Naturally you can change the hardwired values, or even make them dynamic.

Heading ##This Should be a simple starting point for what you want.

BTW, the condensed format is there to make the initial upload fast, I upload with luatool.py using the --dofile option.

Updating your program (u.lc) later is a simple repeat of dofile("bootstrap.lua").

My u.lc is a stage 2 bootstrap that uploads a long list of files (mostly .lc). Probably too involved for this short answer.

Finally, I should mention that this is loosely based on https://github.com/Manawyrm/ESP8266-HTTP/

HTH

Answer 6

Yes it is possible. It is kind of a home-brewed option but it works to a certain extent. The only limitation is in size, of course but other than that it works pretty well. Take a look at:

http://www.instructables.com/id/ESP8266-WiFi-File-Management/

You need to have a way to write the PHP program (I wrote it in C#) if you can't write the code in another language you can download and reuse what this user wrote and use your own PHP server and you should be good to go.

If you have questions, please ask.