CODEWITHSUNDEEP
nginxlua
Sachin Shetty
Sachin Shetty

Reputation: 49

nginx lua make an external http call without blocking the request

We need to log an event to our event server from nginx when some condition happens. I understand that I could use access_by_lua to make the http call. Since the event logging can fail or take longer, We don't want to request processing to wait for the http operation to finish.

If I use access by lua, the amount of time takes for logging would be added to the request time.

Is there a way to kick off the event logging process in async process so that request can move on while event logging can take it's time.

Upvotes: 4

Views: 3749

Answers (2)

Alexander Altshuler
Alexander Altshuler

Reputation: 3064

You can do your usual processing within content_by_lua_* and explicitly specify the end of the response output stream by ngx.eof()

So HTTP request will be processed without any delay.

After that you may do anything you want, use cosocket API or ngx.location.capture() for example

https://github.com/openresty/lua-nginx-module#ngxeof

Here is example from the documentation:

 location = /async {
     keepalive_timeout 0;
     content_by_lua_block {
         ngx.say("got the task!")
         ngx.eof()  -- well written HTTP clients will close the connection at this point
         -- access MySQL, PostgreSQL, Redis, Memcached, and etc here...
     }
 }

Upvotes: 2

moteus
moteus

Reputation: 2235

I do not try it on nginx. But in my app I use separate thread to do all IO and communicate with it using ZeroMQ library. According doc you can create/run work thread with communication socket in init_by_lua_block and then use it in other parts. But to do this you should remember if request/sec is constantly greater than write/sec then in the end you can get out of memory. Also with ZeromMQ is quite easy make several work threads. Of course in this scenario work thread have no acces to nginx module. But you can use e.g. LuaSocket or Lua-cURL (last one can be used to perform multiple requests in async mode). Basic sceleton with single thread.

local thread = function(actor)
  local msg, err while true do
    msg, err = actor:recvx()
    if not msg then break end
    print(msg)
  end
  print('Close thread: ', tostring(err))
end

local zthreads = require "lzmq.threads"
local ztimer   = require "lzmq.timer"
local actor    = zthreads.xactor(thread):start()

for i = 1, 10 do
  actor:send(string.format('message #%d', i))
  ztimer.sleep(1000)
end

Upvotes: 0

Related Questions