Ability to Specify Port for Binding

[alercelik]

Hello. Using bare zmq It is possible to choose a port for binding (using socket.bind()), could you also add this feature? I believe this is a must for docker containers.

[jeffbass]

imageZMQ already has the ability to choose a port for binding. When you instantiate either the ImageSender or ImageHub class, you must specify a TCP address and port. When using an ImageHub in PUB/SUB mode, the ImageHub can connect to multiple senders at the same time using the ImageHub.connect() method to Subscribe to additional Publishers. The ImageHub.connect() method requires specifying an address and port. The API for the ImageHub and ImagerSender classes is here. There a a number of programs that set addresses and ports in the Examples Folder.

[alercelik]

Sorry for the late response.

While creating ImageSender, what provided as argument is ip:port of the Hub. Besides, I could not see any other option to specify the port for binding (not for connecting). Therefore, a random port is allocated (I believe) on ImageSender side, which is quite a problem while working on Docker contaiers. I found out that zmq library lets you specify a port for binding. I downloaded the source code of imagezmq and added below line to init_reqrep function and i was able to specify a local port for binding.

self.zmq_socket.bind(bind_address) # bind address provided as another argument

https://github.com/jeffbass/imagezmq/blob/master/imagezmq/imagezmq.py#L55

[jeffbass]

My understanding of ZMQ is that the "bind before connect" procedure that is sometimes seen in non-ZMQ socket protocols is not necessary when using ZMQ. The address specified in self.zmq_socket.connect(address) includes a port number. The same port number needs to be specified for both bind() and connect(). That port number is both "bound" and "connected" when the self.zmq_socket.connect(address) line is executed. ZMQ makes sure the OS socket hooks are bound appropriately using the same port number at both ends. I don't use Docker containers, but I think the extra bind statement you are using is unnecessary. Even if the extra self.zmq_socket.bind(address) line is used, it would have to specify the same port number. If a different port number is specified using another argument, I believe that you are binding a port that will never be used. See my commented code line below.

    def init_reqrep(self, address):
        """ Creates and inits a socket in REQ/REP mode
        """

        socketType = zmq.REQ
        self.zmq_context = SerializingContext()
        self.zmq_socket = self.zmq_context.socket(socketType)
        self.zmq_socket.bind(address)   # this is not needed, it is done by the line below,
                                        # using the same socket number
        self.zmq_socket.connect(address)

Did you get a failure in your Docker container? Is that why you added the bind statement? I am definitely not a Docker expert and would appreciate any learnings you could share. I am leaving this issue open so others can read this and add their experiences. Thanks for starting this discussion.

[alercelik]

If I am not wrong, binding means "allocating a local port to connect a port on another host" whereas in connect() you provide destination's port. So, what you provide in bind() and connect() have different meanings and are irrelevant. In my case, it is possible to think using Docker as you have only a few ports for binding, all other ports are not visible to outside therefore connection can not be established if one of those non-visible ports is chosen. So I need to specify a local port address for binding.

I believe workflow should be like this:

def init_reqrep(self, address):
    """ Creates and inits a socket in REQ/REP mode
    """

    socketType = zmq.REQ
    self.zmq_context = SerializingContext()
    self.zmq_socket = self.zmq_context.socket(socketType)

    if self.bind_address:
        self.zmq_socket.bind(self.bind_address)   # bind_address should be an another argument to __init__

    self.zmq_socket.connect(address)

You are welcome. Thanks for this effective & simple to use library.

[alercelik]

There is also a random option. It would be really helpful it both options are available while creating ImageSender.

self.zmq_socket.bind_to_random_port('tcp://*', min_port=6001, max_port=6004, max_tries=100)

[jeffbass]

I believe you are misunderstanding what the ZMQ connect method does. It does much more than just "connect". The ZMQ connect method parses out the port number provided in the connect address. ZMQ then uses that port number to do all the local OS port binding that is required.

In ZMQ, one side of each message stream connects and the other side of the same message stream binds. The Python code on both sides must specify the same port number. With the REQ / REP message pair, the REQ side uses ZMQ connect and the REP side uses ZMQ bind. You can see this in the interactive ZMQ Guide if you select Python for the examples. In the ZMQ Guide, the REP code is shown first and is called the server. The REQ code is shown next and is called the client.

ZMQ does everything required and the ZMQ Guide refers to this "do everything needed in a few lines of code" as "superpowers". Quoting from the ZMQ Guide:

Now this looks too simple to be realistic, but ZeroMQ sockets have, as we already learned, superpowers.

Thanks for your suggestions for these potential new options to ImageSender. My personal bias is to keep imageZMQ as simple as possible. I am reluctant to add new options to imageZMQ that may increase its API complexity, but I'd love to hear from others. I am going to leave this issue open and see if anyone else thinks these 2 options should be added.

If you fork imageZMQ to add these options and would like for other imageZMQ users to be aware of them, please send me a link to your GitHub repository. I'll add your fork to the "Helpful Forks" section of the README.