Skip to content

ZooKeeper cluster-state reads issue a redundant exists() before each read, doubling check operations #8712

@GGraziadei

Description

@GGraziadei

This issue is related to org.apache.storm.zookeeper.ClientZookeeper

Sniffing inbound ZooKeeper traffic during normal cluster-state polling shows a paired sequence for each read path.

Actually, when Storm reads cluster state from ZooKeeper, it first asks "does this node exist?" and only then asks "give me its data", two requests to the zk for a single logical read.
The initial check is unnecessary: the read call already returns the data when the node exists and reports a "no node" result when it doesn't. As a result, every value Storm reads (assignments, supervisor info, heartbeats, errors) sends zk roughly twice the requests it needs.

This is not a tuple data-plan performance issue; these are coordination reads. It is an avoidable request volume against the zk.

I propose to fix these methods with the pattern attached:

  • getData
  • getDataWithVersion:
  • getVersion:

current

 GIVEN path, wantWatch:
      if EXISTS path (requesting a watch if wantWatch):  // request #1
          result <- READ path (requesting a watch if wantWatch)   // request #2
          return result.value
      else:
          return EMPTY   // exists-watch (if requested) is already set

proposed

GIVEN path, wantWatch:
      result <- READ path (requesting a watch if wantWatch)  // request #1

      if result is present:
          return result.value   // watch already set if requested
      else:  // node is absent (e.g. `NoNodeException`)
          if wantWatch:
              WATCH path  // request #2, only here
          return EMPTY
      on any other error:
          FAIL 

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions