CODEWITHSUNDEEP
httplualuasocket
user1973386
user1973386

Reputation: 1135

Download large file with LuaSocket's HTTP module while keeping UI responsive

I would like to use LuaSocket's HTTP module to download a large file while displaying progress in the console and later on in a GUI. The UI must never block, not even when the server is unresponsive during the transfer. Additionally, creating a worker thread to handle the download is not an option.

Here's what I got so far:

local io = io

local ltn12 = require("ltn12")
local http = require("socket.http")

local fileurl = "http://www.example.com/big_file.zip"
local fileout_path = "big_file.zip"

local file_size = 0
local file_down = 0

-- counter filter used in ltn12
function counter(chunk)
    if chunk == nil then
        return nil
    elseif chunk == "" then
        return ""
    else
        file_down = file_down + #chunk
        ui_update(file_size, file_down) -- update ui, run main ui loop etc.
        return chunk -- return unmodified chunk
    end
end

-- first request
-- determine file size
local r, c, h = http.request {
    method = "HEAD",
    url = fileurl
}
file_size = h["content-length"]

-- second request
-- download file
r, c, h = http.request {
    method = "GET",
    url = fileurl,
    -- set our chain, count first then write to file
    sink = ltn12.sink.chain(
        counter,
        ltn12.sink.file(io.open(fileout_path, "w"))
    )
}

There are a few problems with the above, ignoring error checking and hard-coding:

  1. It requires 2 HTTP requests when it is possible with only 1 (a normal GET request also sends content-length)
  2. If the server is unresponsive, then the UI will also be unresponsive, as the filter only gets called when there is data to process.

How could I do this making sure the UI never blocks?

Upvotes: 3

Views: 1081

Answers (1)

Paul Kulchenko
Paul Kulchenko

Reputation: 26794

There is an example on non-preemptive multithreading in Programming in Lua that uses non-blocking luasocket calls and coroutines to do a multiple parallel downloads. It should be possible to apply the same logic to your process to avoid blocking. I can only add that you should consider calling this logic from IDLE event in your GUI (if there is such a thing) to avoid getting "attempt to yield across metamethod/c-call boundary" errors.

Upvotes: 1

Related Questions