Low-level interface: replaced nb::inst_wrap, added nb::inst_replace_*

The ``nb::inst_wrap`` API had a serious flaw: there was no way to
control whether the instance, when eventually garbage collected, should
free its storage via a call to the untyped ``operator delete``
(following the in-place destructor). This commit removes the function
completely. While technically an API break, there is no alternative
because the function was broken in practice.

The commit adds two more fine-grained replacements:

- ``nb::inst_take_ownership()``: analogous to a cast with
  ``nb::rv_policy::take_ownership``: deleting the instance will call the
  C++ in-place destructor and untyped ``operator delete``.

- ``nb::inst_reference()``: analogous to a cast with
  ``nb::rv_policy::reference`` or ``nb::rv_policy::reference_internal``
  depending on usage: deleting the instance will neither call the
  in-place destructor nor the untyped ``operator delete``.

  If an optional ``parent`` parameter is provided, nanobind will ensure
  that the parent remains alive while the instance exists.

Besides these changes, the commit also adds two compound operations:
``nb::inst_replace_copy()`` and ``nb::inst_replace_move()`` that replace
the contents of an already initialized instance by copying or moving
from another instance.

Author: wjakob

docs/api_core.rst

@@ -2113,8 +2113,7 @@ Low-level type and instance access
 nanobind exposes a low-level interface to provide fine-grained control over
 the sequence of steps that instantiates a Python object wrapping a C++
 instance. An thorough explanation of these features is provided in a
-:ref:`separate section <lowlevel>`. The function listing below merely
-summarizes their signatures.
+:ref:`separate section <lowlevel>`. 
 
 Type objects
 ^^^^^^^^^^^^
@@ -2144,9 +2143,9 @@ Type objects
    Return a reference to supplemental data stashed in a type object.
    The type ``T`` must exactly match the type specified in the
    :cpp:class:`nb::supplement\<T\> <supplement>` annotation used when
-   creating the type; no type check is performed, and accessing an
-   incorrect supplement type may crash the interpreter.
-   See :cpp:class:`supplement`.
+   creating the type; no type check is performed, and invalid supplement
+   accesses may crash the interpreter. Also refer to
+   :cpp:class:`nb::supplement\<T\> <supplement>`.
 
 .. cpp:function:: str type_name(handle h)
 
@@ -2163,6 +2162,14 @@ Type objects
 Instances
 ^^^^^^^^^
 
+The documentation below refers to two per-instance flags with the following meaning:
+
+- *ready*: is the instance fully constructed? nanobind will not permit passing
+  the instance to a bound C++ function when this flag is unset.
+
+- *destruct*: should nanobind call the C++ destructor when the instance is
+  garbage-collected?
+
 .. cpp:function:: bool inst_check(handle h)
 
    Returns ``true`` if `h` represents an instance of a type that was
@@ -2171,63 +2178,129 @@ Instances
 .. cpp:function:: template <typename T> T * inst_ptr(handle h)
 
    Assuming that `h` represents an instance of a type that was previously bound
-   via :cpp:class:`class_`, return a pointer to the C++ instance.
+   via :cpp:class:`class_`, return a pointer to the underlying C++ instance.
 
-   The function *does not check* that `T` is consistent with the type of `h`.
+   The function *does not check* that `h` actually contains an instance with
+   C++ type `T`.
 
 .. cpp:function:: object inst_alloc(handle h)
 
-   Assuming that `h` represents a bound type (see :cpp:func:`type_check`),
-   allocate an uninitialized Python object of type `h` and return it.
+   Assuming that `h` represents a type object that was previously created via
+   :cpp:class:`class_` (see :cpp:func:`type_check`), allocate an unitialized
+   object of type `h` and return it. The *ready* and *destruct* flags of the
+   returned instance are both set to ``false``.
 
 .. cpp:function:: object inst_alloc_zero(handle h)
 
-   Assuming that `h` represents a bound type (see :cpp:func:`type_check`),
-   allocate a zero-initialized Python object of type `h` and return it. This
-   operation is equilvalent to calling :cpp:func:`inst_alloc` followed by
+   Assuming that `h` represents a type object that was previously created via
+   :cpp:class:`class_` (see :cpp:func:`type_check`), allocate a zero-initialized
+   object of type `h` and return it. The *ready* and *destruct* flags of the
+   returned instance are both set to ``true``.
+
+   This operation is equivalent to calling :cpp:func:`inst_alloc` followed by
    :cpp:func:`inst_zero`.
 
-.. cpp:function:: object inst_wrap(handle h, void * p)
+.. cpp:function:: object inst_reference(handle h, void * p, handle parent = handle())
 
-   Assuming that `h` represents an instance of a type that was previously bound
-   via :cpp:class:`class_`, create an object of type `h` that wraps an existing
-   C++ instace `p`.
+   Assuming that `h` represents a type object that was previously created via
+   :cpp:class:`class_` (see :cpp:func:`type_check`) create an object of type
+   `h` that wraps an existing C++ instance `p`.
+
+   The *ready* and *destruct* flags of the returned instance are respectively
+   set to ``true`` and ``false``.
+
+   This is analogous to casting a C++ object with return value policy
+   :cpp:enumerator:`rv_policy::reference`.
+
+   If a `parent` object is specified, the instance keeps this parent alive
+   while the newly created object exists. This is analogous to casting a C++
+   object with return value policy
+   :cpp:enumerator:`rv_policy::reference_internal`.
+
+.. cpp:function:: object inst_take_ownership(handle h, void * p)
+
+   Assuming that `h` represents a type object that was previously created via
+   :cpp:class:`class_` (see :cpp:func:`type_check`) create an object of type
+   `h` that wraps an existing C++ instance `p`.
+
+   The *ready* and *destruct* flags of the returned instance are both set to
+   ``true``.
+
+   This is analogous to casting a C++ object with return value policy
+   :cpp:enumerator:`rv_policy::take_ownership`.
 
 .. cpp:function:: void inst_zero(handle h)
 
-   Zero-initialize the contents of `h`.
+   Zero-initialize the contents of `h`. Sets the *ready* and *destruct* flags
+   to ``true``.
 
 .. cpp:function:: bool inst_ready(handle h)
 
-   Query the *ready* field of the object `h`.
+   Query the *ready* flag of the instance `h`.
 
-.. cpp:function:: void inst_mark_ready(handle h)
+.. cpp:function:: std::pair<bool, bool> inst_state(handle h)
 
-   Mark the object `h` as *ready*.
+   Separately query the *ready* and *destruct* flags of the instance `h`.
 
-.. cpp:function:: std::pair<bool, bool> inst_state(handle h)
+.. cpp:function:: void inst_mark_ready(handle h)
 
-   Query the *ready* and *destruct* fields of the object `h`.
+   Simultaneously set the *ready* and *destruct* flags of the instance `h` to ``true``.
 
 .. cpp:function:: void inst_set_state(handle h, bool ready, bool destruct)
 
-   Set the *ready* and *destruct* fields of the object `h`.
+   Separately set the *ready* and *destruct* flags of the instance `h`.
 
 .. cpp:function:: void inst_destruct(handle h)
 
-   Destruct the object `h`.
+   Destruct the instance `h`. This entails calling the C++ destructor if the
+   *destruct* flag is set and then setting the *ready* and *destruct* fields to
+   ``false``.
 
 .. cpp:function:: void inst_copy(handle dst, handle src)
 
-   Move-construct the contents of `src` into `dst` and mark `dst` as *ready*.
+   Copy-construct the contents of `src` into `dst` and set the *ready* and
+   *destruct* flags of `dst` to ``true``.
+
+   `dst` should be an uninitialized instance of the same type. Note that
+   setting the *destruct* flag may be problematic if `dst` is an offset into an
+   existing object created using :cpp:func:`inst_reference` (the destructor
+   will be called multiple times in this case). If so, you must use
+   :cpp:func:`inst_set_state` to disable the flag following the call to
+   :cpp:func:`inst_copy`.
 
 .. cpp:function:: void inst_move(handle dst, handle src)
 
-   Copy-construct the contents of `src` into `dst` and mark `dst` as *ready*.
+   Move-construct the contents of `src` into `dst` and set the *ready* and
+   *destruct* flags of `dst` to ``true``.
+
+   `dst` should be an uninitialized instance of the same type. Note that
+   setting the *destruct* flag may be problematic if `dst` is an offset into an
+   existing object created using :cpp:func:`inst_reference` (the destructor
+   will be called multiple times in this case). If so, you must use
+   :cpp:func:`inst_set_state` to disable the flag following the call to
+   :cpp:func:`inst_move`.
+
+.. cpp:function:: void inst_replace_copy(handle dst, handle src)
+
+   Destruct the contents of `dst` (even if the *destruct* flag is ``false``).
+   Next, copy-construct the contents of `src` into `dst` and set the *ready*
+   flag of ``dst``. The value of the *destruct* flag is subsequently set to its
+   value prior to the call.
+
+   This operation is useful to replace the contents of one instance with that
+   of another regardless of whether `dst` has been created using
+   :cpp:func:`inst_alloc`, :cpp:func:`inst_reference`, or
+   :cpp:func:`inst_take_ownership`.
+
+.. cpp:function:: void inst_replace_move(handle dst, handle src)
+
+   Analogous to :cpp:func:`inst_replace_copy`, except that a move constructor
+   is used here.
 
 .. cpp:function:: str inst_name(handle h)
 
-   Return the full (module-qualified) name of the instance's type object as a Python string.
+   Return the full (module-qualified) name of the instance's type object as a
+   Python string.
 
 Global flags
 ------------

docs/lowlevel.rst

@@ -11,8 +11,8 @@ instance. This is useful when writing generic binding code that manipulates
 nanobind-based objects of various types.
 
 Given a previous :cpp:class:`nb::class_\<...\> <class_>` binding declaration,
-this interface can be used to look up the Python type object associated with
-a C++ class named ``MyClass``.
+the :cpp:func:`nb::type\<T\>() <type>` template function can be used to look up
+the Python type object associated with a C++ class named ``MyClass``.
 
 .. code-block:: cpp
 
@@ -28,9 +28,10 @@ redundant in this case):
 
    assert(py_type.is_valid() && nb::type_check(py_type));
 
-nanobind knows the size, alignment, and C++ RTTI ``std::type_info``
-record of all bound types. They can be queried on the fly in situations
-where this is useful.
+nanobind knows the size, alignment, and C++ RTTI ``std::type_info`` record of
+all bound types. They can be queried on the fly via :cpp:func:`nb::type_size()
+<type_size>`, :cpp:func:`nb::type_align() <type_size>`, and
+:cpp:func:`nb::type_info() <type_size>` in situations where this is useful.
 
 .. code-block:: cpp
 
@@ -47,9 +48,11 @@ to prevent undefined behavior. It must first be initialized.
 
    nb::object py_inst = nb::inst_alloc(py_type);
 
-We can confirm that this newly created instance is managed by nanobind,
-that it has the correct type in Python, and that it is not ``ready``
-(i.e. uninitialized):
+We can confirm via :cpp:func:`nb::inst_check() <inst_check>` that this newly
+created instance is managed by nanobind, that it has the correct type in
+Python. Calling :cpp:func:`nb::inst_ready() <inst_ready>` reveals that the
+*ready* flag of the instance is set to ``false`` (i.e., it is still
+uninitialized).
 
 .. code-block:: cpp
 
@@ -58,16 +61,17 @@ that it has the correct type in Python, and that it is not ``ready``
           !nb::inst_ready(py_inst));
 
 For simple *plain old data* (POD) types, the :cpp:func:`nb::inst_zero()
-<inst_zero>` function can be used to zero-initialize the object and mark it
+<inst_zero>` function can be used to *zero-initialize* the object and mark it
 as ready.
 
 .. code-block:: cpp
 
    nb::inst_zero(py_inst);
    assert(nb::inst_ready(py_inst));
 
-We can destruct this default instance and convert it back to non-ready
-status. This memory region can then be reinitialized once more.
+We can destruct this default instance via :cpp:func:`nb::inst_destruct()
+<inst_destruct>` and convert it back to non-ready status. This memory region
+can then be reinitialized once more.
 
 .. code-block:: cpp
 
@@ -88,9 +92,9 @@ by nanobind.
    new (ptr) MyClass(/* constructor arguments go here */);
 
 Following this constructor call, we must inform nanobind that the instance
-object is now fully constructed. When its reference count reaches zero,
-nanobind will automatically call the in-place destructor
-(``MyClass::~MyClass``).
+object is now fully constructed via :cpp:func:`nb::inst_mark_ready()
+<inst_mark_ready>`. When its reference count reaches zero, nanobind will then
+automatically call the in-place destructor (``MyClass::~MyClass``).
 
 .. code-block:: cpp
 
@@ -105,12 +109,14 @@ the C++ destructor and mark the Python object as non-ready).
    nb::inst_destruct(py_inst);
 
 Another useful feature is that nanobind can copy- or move-construct ``py_inst``
-from another instance of the same type. This calls the C++ copy or move
-constructor and transitions ``py_inst`` back to ``ready`` status. Note that
-this is equivalent to calling an in-place version of these constructors above
-but compiles to more compact code (the :cpp:class:`nb::class_\<MyClass\>
-<class_>` declaration had already created bindings for both constructors, and
-this simply calls those bindings).
+from another instance of the same type via :cpp:func:`nb::inst_copy()
+<inst_copy>` and :cpp:func:`nb::inst_move() <inst_move>`. These functions call
+the C++ copy or move constructor and transition ``py_inst`` back to ``ready``
+status. This is equivalent to calling an in-place version of these constructors
+followed by a call to :cpp:func:`nb::inst_mark_ready() <inst_mark_ready>` but
+compiles to more compact code (the :cpp:class:`nb::class_\<MyClass\> <class_>`
+declaration had already created bindings for both constructors, and this simply
+calls those bindings).
 
 .. code-block:: cpp
 
@@ -119,6 +125,19 @@ this simply calls those bindings).
    else
        nb::inst_move(/* dst = */ py_inst, /* src = */ some_other_instance);
 
+Both functions assume that the destination object is uninitialized. Two
+alternative versions :cpp:func:`nb::inst_replace_copy() <inst_replace_copy>`
+and :cpp:func:`nb::inst_replace_move() <inst_replace_move>` destruct an
+initialized instance and replace it with the contents of another by either
+copying or moving.
+
+.. code-block:: cpp
+
+   if (copy_instance)
+       nb::inst_replace_copy(/* dst = */ py_inst, /* src = */ some_other_instance);
+   else
+       nb::inst_replace_move(/* dst = */ py_inst, /* src = */ some_other_instance);
+
 Note that these functions are all *unsafe* in the sense that they do not
 verify that their input arguments are valid. This is done for
 performance reasons, and such checks (if needed) are therefore the
@@ -127,10 +146,14 @@ only be called with nanobind type objects, and functions labeled
 ``nb::inst_*`` should only be called with nanobind instance objects.
 
 The functions :cpp:func:`nb::type_check() <type_check>` and
-:cpp:func:`nb::inst_check() <inst_check>` are exceptions to this rule: they
-accept any Python object and test whether something is a nanobind type or
+:cpp:func:`nb::inst_check() <inst_check>` are exceptions to this rule:
+they accept any Python object and test whether something is a nanobind type or
 instance object.
 
+Two further functions :cpp:func:`nb::type_name() <type_name>` and
+:cpp:func:`nb::inst_name() <inst_name>` determine the type name associated with
+a type or instance thereof. These also accept non-nanobind types and instances.
+
 Even lower-level interface
 --------------------------
 
@@ -149,18 +172,14 @@ The functions :cpp:func:`nb::inst_zero() <inst_zero>`,
 flags to ``true``, and :cpp:func:`nb::inst_destruct() <inst_destruct>` sets
 both of them to ``false``.
 
-In rare situations, the destructor should *not* be invoked when the
-instance is garbage collected, for example when working with a nanobind
-instance representing a field of a parent instance created using the
+In rare situations, the destructor should *not* be invoked when the instance is
+garbage collected, for example when working with a nanobind instance
+representing a field of a parent instance created using the
 :cpp:enumerator:`nb::rv_policy::reference_internal
 <rv_policy::reference_internal>` return value policy. The library therefore
-exposes two more functions that can be used to read/write these two flags
-individually.
-
-.. code-block:: cpp
-
-   void inst_set_state(handle h, bool ready, bool destruct);
-   std::pair<bool, bool> inst_state(handle h);
+exposes two more functions :cpp:func:`nb::inst_state() <inst_state>` and
+:cpp:func:`nb::inst_set_state() <inst_set_state>` that can be used to access
+them individually.
 
 Referencing existing instances
 ------------------------------
@@ -178,19 +197,25 @@ with the binding ``py_type``.
    ... omitted, see the previous examples ...
 
 What if the C++ instance already exists? nanobind also supports this case via
-the :cpp:func:`nb::inst_wrap() <inst_wrap>` function—in this case, the Python
-object references the existing memory region, which is potentially (slightly)
-less efficient due to the need for an extra indirection.
+the :cpp:func:`nb::inst_reference() <inst_reference>` and
+:cpp:func:`nb::inst_take_ownership() <inst_take_ownership>` functions—in this
+case, the Python object references the existing memory region, which is
+potentially (slightly) less efficient due to the need for an extra indirection.
 
 .. code-block:: cpp
 
    MyClass *inst = new MyClass();
-   nb::object py_inst = nb::inst_wrap(py_type, inst);
 
-   // Mark as ready, garbage-collecting 'py_inst' will cause 'inst' to be
-   // deleted as well. Call nb::inst_set_state (documented above) for more
-   // fine-grained control.
-   nb::inst_mark_ready(py_inst);
+   // Transfer ownership of 'inst' to Python (which will use a delete
+   // expression to free it when the Python instance is garbage collected)
+   nb::object py_inst = nb::inst_take_ownership(py_type, inst);
+
+   // We can also wrap C++ instances that should not be destructed since
+   // they represent offsets into another data structure. In this case,
+   // the optional 'parent' parameter ensures that 'py_inst' remains alive
+   // while 'py_subinst' exists to prevent undefined behavior.
+   nb::object py_subinst = nb::inst_reference(
+       py_field_type, &inst->field, /* parent = */ py_inst);
 
 .. _supplement: