Object picking

[rossant]

One area that may require careful consideration in the protocol is object picking. Since it may involve specific and complex GPU implementation, I think it needs to be part of the protocol.

There are several things to work out:

• Multiple types of picking:

  • single pixel
  • axis-parallel rectangle
  • axis-parallel horizontal or vertical region (ie between two axhlines or axvlines)
  • polygon

• Renderer -> client reverse communication: the client sends asynchronous requests to the renderer, but the renderer should also have a channel to send back asynchronous responses to the client. These responses include error messages as well as picking results.

Request examples

- action: pick_point
  parameters:
    viewport_id: str
    canvas_id: str
    x: int  # in pixels
    y: int  # in pixels

- action: pick_rectangle
  parameters:
    viewport_id: str
    canvas_id: str
    x0: int  # in pixels
    y0: int  # in pixels
    x1: int  # in pixels
    y1: int  # in pixels

- action: pick_vertical|horizontal_area
  parameters:
    viewport_id: str
    canvas_id: str
    x0|y0: int  # in pixels
    x1|y1: int  # in pixels

- action: pick_polygon
  parameters:
    viewport_id: str
    canvas_id: str
    xs: array of int  # in pixels
    ys: array of int  # in pixels

Response examples

- response: pick_point
  parameters:
    request_id: str
    visual_id: str
    item: int specifying the index of the selected visual item

- response: pick_rectangle|area|polygon
  parameters:
    request_id: str
    visuals: array of ids specifiying the visual id of each selected item
    items: array of int specifying the index of each item within its visual