[RFCs] Add Flow Graph try_put_and_wait RFC

[kboyarinov]

Description

Add a comprehensive description of proposed changes

Fixes # - issue number(s) if exists

Type of change

Choose one or multiple, leave empty if none of the other choices apply

Add a respective label(s) to PR if you have permissions

• bug fix - change that fixes an issue
• new feature - change that adds functionality
• tests - change in tests
• infrastructure - change in infrastructure and CI
• documentation - documentation update

Tests

• added - required for new features and some bug fixes
• not needed

Documentation

• updated in # - add PR number
• needs to be updated
• not needed

Breaks backward compatibility

• Yes
• No
• Unknown

Notify the following users

List users with @ to send notifications

Other information

[vossmjp]

You should say what "lost" means in this context. How does it affect waiting? Does a lost message/metainfo properly decrement the count?

[vossmjp]

Suggested change

	Since the metainformation is kept in the buffer together with the message itself, even if the computation is completed, the ``try_put_and_wait`` would stuck because of the reference
	held by the buffer.
	Since the metainformation is kept in the buffer together with the message itself, even if the message is consumed by a successor,
	``try_put_and_wait`` will not complete because of the reference held by the buffer until the node is explicitly cleared.

[vossmjp]

Suggested change

	* The ``overwrite_node`` should be explicitly reset by calling ``node.reset()`` or the element with the stored metainfo should be overwritten with another element.
	* The ``write_once_node`` should be explicitly reset by calling ``node.reset()`` since the item cannot be overwritten.
	* The ``overwrite_node`` should be explicitly reset by calling ``node.clear()`` or the element with the stored metainfo should be overwritten with another element.
	* The ``write_once_node`` should be explicitly reset by calling ``node.clear()`` since the item cannot be overwritten.

[vossmjp]

Suggested change

	Each input port of the join_node should support the queue for both values and the associated metainformations. Once all of the input ports would contain the value, the values
	Each input port of the join_node should support the queue for both values and the associated metainformations. Once all of the input ports contain the value, the values

[vossmjp]

Suggested change

	If the item with the metainformation is stored in the internal queue of one of the input ports, but items never received by other ports, the item and the metainformation would be kept in the
	queue and block the corresponding ``try_put_and_wait`` call.
	If an item with the metainformation is stored in the internal queue of one of the input ports, but items are never received by other ports, the item and the metainformation will be kept in the
	queue and block the corresponding ``try_put_and_wait`` call.

[vossmjp]

We may need to add a clear function to join_node to allow explicit clearing of the internal queues.

[vossmjp]

Should we add a clear function to all buffering nodes?