Feature request: dynamic listen/close socket in lua

[rainingmaster]

Hi there,

I think many user need the new feature that we can dynamic listen or close a socket in the lua context, such as the discuss here, or some user' PR #1393 .

About the usage of this function, I have some ideas:

1. The usage should like this:

function accept_hook(socket)
     local b, err = sock:receive(8) -- like ngx.tcp.socket/ngx.udp.socket, it is a stream-typed cosocket
     if err then
         return
    end

    sock:send("welcome")
    ngx.exit() -- close the socket for tcp
end

local err = ngx.socket.listen("tcp://127.0.0.1:5050",  accept_hook) -- can not bind same address more than one time
if err then
    ngx.log(ngx.ERR, "listen failed")
    return
end

And support close the listen by the address, like this:

local err = ngx.socket.close("tcp://127.0.0.1:5050")
if err then
    ngx.log(ngx.ERR, "listen failed")
    return
end

2. ngx.socket.listen for TCP, will bind, listen and accept a socket, and put the socekt into event list, like ngx_event_accept, and for UDP, will bind and recvmsg for the socket, like [ngx_event_recvmsg], and put the socekt into event list(https://github.com/nginx/nginx/blob/master/src/event/ngx_event_udp.c#L32).

3. This should be like ngx.timer, can use in each phase. If run it in init_by_lua, all worker will bind same address by reuseport.

4. Different worker can bind same address in different time by reuseport.

5. When the worker exit, all listen will close automatic.

6. A new independent phase should be build(like timer, maybe name as lua_socket?), some function, such as ngx.req.* can not be use.

[spacewander]

Some additional notes here.

4. Different worker can bind same address in different time by reuseport.

The unix socket doesn't support reuseport, so we can't listen on it like the vanilla TCP address.

6. A new independent phase should be build(like timer, maybe name as lua_socket?), some function, such as ngx.req.* can not be use.

If we treat the socket as a long-live timer, be aware of the Nginx's per-request memory pool model. The memory is only freed when the request/long-live timer finished. Another problem is that multiple coroutines created in a reques will be scheduled in O(n) complexity (even dead ones): #1215.

[rainingmaster]

@spacewander Thanks for your suggest.

The unix socket doesn't support reuseport, so we can't listen on it like the vanilla TCP address.

Thanks for your recommend, we could note this in the README.

If we treat the socket as a long-live timer, be aware of the Nginx's per-request memory pool model. The memory is only freed when the request/long-live timer finished.

I think this should be acceptable? Listen a socket must be occupy the memory until close the socket or Nginx exits.

Another problem is that multiple coroutines created in a reques will be scheduled in O(n) complexity (even dead ones): #1215.

I didn't realize this problem before， do we have some method to solve it so far?

[spacewander]

I didn't realize this problem before， do we have some method to solve it so far?

So far the solution is to avoid creating too much coroutines in a single request. It is, the user of dynamic listen socket need to avoid creating coroutine per accept, or reuse the coroutines like #1215 suggested.

[agentzh]

I like such new APIs for listening and accepting but I don't like the callback function argument design. It should be a synchronous but still nonblocking method call on the listening socket (also it should return ok, err instead of err for Lua's API convention), as in

local sock_tcp = require "ngx.socket.tcp"
local listen_sock, err = sock_tcp.listen("127.0.0.1", 5050)
local timeout = 3000  -- 3 seconds
local client_sock, err = listen_sock:accept(timeout)
-- then we can use client_sock like existing cosockets, e.g., client_sock:receive("l")

Similarly, for unix domain sockets:

local sock_unix = require "ngx.socket.unix"
local listen_sock, err = sock_unix.listen("/path/to/unix.sock")
local timeout = 3000  -- 3 seconds
local client_sock, err = listen_sock:accept(timeout)
-- then we can use client_sock like existing cosockets, e.g., client_sock:receive("l")

The client socket objects should follow the same restriction and cleanup semantics of existing cosockets (that is, not exceeding the lifetime of the current request handler or timer handler) so that we can avoid socket leaks by design. The listening sockets can outlive the handler contexts creating them though, otherwise it won't be very useful.

To simplify the implementation here, we can simply support listening in workers, which is already very useful and also simpler than handling reuseport, binary upgrade, HUP reload, and etc.

[agentzh]

Yeah, we should avoid creating a new coroutine per accept(). Otherwise it's too expensive for such a frequent method call. The accept() method should only be used contexts which allow yielding, like most of the existing cosocket methods.