diff options
Diffstat (limited to '3rdparty/pybind11/docs/advanced/functions.rst')
| -rw-r--r-- | 3rdparty/pybind11/docs/advanced/functions.rst | 507 |
1 files changed, 507 insertions, 0 deletions
diff --git a/3rdparty/pybind11/docs/advanced/functions.rst b/3rdparty/pybind11/docs/advanced/functions.rst new file mode 100644 index 00000000..3e1a3ff0 --- /dev/null +++ b/3rdparty/pybind11/docs/advanced/functions.rst @@ -0,0 +1,507 @@ +Functions +######### + +Before proceeding with this section, make sure that you are already familiar +with the basics of binding functions and classes, as explained in :doc:`/basics` +and :doc:`/classes`. The following guide is applicable to both free and member +functions, i.e. *methods* in Python. + +.. _return_value_policies: + +Return value policies +===================== + +Python and C++ use fundamentally different ways of managing the memory and +lifetime of objects managed by them. This can lead to issues when creating +bindings for functions that return a non-trivial type. Just by looking at the +type information, it is not clear whether Python should take charge of the +returned value and eventually free its resources, or if this is handled on the +C++ side. For this reason, pybind11 provides a several *return value policy* +annotations that can be passed to the :func:`module::def` and +:func:`class_::def` functions. The default policy is +:enum:`return_value_policy::automatic`. + +Return value policies are tricky, and it's very important to get them right. +Just to illustrate what can go wrong, consider the following simple example: + +.. code-block:: cpp + + /* Function declaration */ + Data *get_data() { return _data; /* (pointer to a static data structure) */ } + ... + + /* Binding code */ + m.def("get_data", &get_data); // <-- KABOOM, will cause crash when called from Python + +What's going on here? When ``get_data()`` is called from Python, the return +value (a native C++ type) must be wrapped to turn it into a usable Python type. +In this case, the default return value policy (:enum:`return_value_policy::automatic`) +causes pybind11 to assume ownership of the static ``_data`` instance. + +When Python's garbage collector eventually deletes the Python +wrapper, pybind11 will also attempt to delete the C++ instance (via ``operator +delete()``) due to the implied ownership. At this point, the entire application +will come crashing down, though errors could also be more subtle and involve +silent data corruption. + +In the above example, the policy :enum:`return_value_policy::reference` should have +been specified so that the global data instance is only *referenced* without any +implied transfer of ownership, i.e.: + +.. code-block:: cpp + + m.def("get_data", &get_data, return_value_policy::reference); + +On the other hand, this is not the right policy for many other situations, +where ignoring ownership could lead to resource leaks. +As a developer using pybind11, it's important to be familiar with the different +return value policies, including which situation calls for which one of them. +The following table provides an overview of available policies: + +.. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}| + ++--------------------------------------------------+----------------------------------------------------------------------------+ +| Return value policy | Description | ++==================================================+============================================================================+ +| :enum:`return_value_policy::take_ownership` | Reference an existing object (i.e. do not create a new copy) and take | +| | ownership. Python will call the destructor and delete operator when the | +| | object's reference count reaches zero. Undefined behavior ensues when the | +| | C++ side does the same, or when the data was not dynamically allocated. | ++--------------------------------------------------+----------------------------------------------------------------------------+ +| :enum:`return_value_policy::copy` | Create a new copy of the returned object, which will be owned by Python. | +| | This policy is comparably safe because the lifetimes of the two instances | +| | are decoupled. | ++--------------------------------------------------+----------------------------------------------------------------------------+ +| :enum:`return_value_policy::move` | Use ``std::move`` to move the return value contents into a new instance | +| | that will be owned by Python. This policy is comparably safe because the | +| | lifetimes of the two instances (move source and destination) are decoupled.| ++--------------------------------------------------+----------------------------------------------------------------------------+ +| :enum:`return_value_policy::reference` | Reference an existing object, but do not take ownership. The C++ side is | +| | responsible for managing the object's lifetime and deallocating it when | +| | it is no longer used. Warning: undefined behavior will ensue when the C++ | +| | side deletes an object that is still referenced and used by Python. | ++--------------------------------------------------+----------------------------------------------------------------------------+ +| :enum:`return_value_policy::reference_internal` | Indicates that the lifetime of the return value is tied to the lifetime | +| | of a parent object, namely the implicit ``this``, or ``self`` argument of | +| | the called method or property. Internally, this policy works just like | +| | :enum:`return_value_policy::reference` but additionally applies a | +| | ``keep_alive<0, 1>`` *call policy* (described in the next section) that | +| | prevents the parent object from being garbage collected as long as the | +| | return value is referenced by Python. This is the default policy for | +| | property getters created via ``def_property``, ``def_readwrite``, etc. | ++--------------------------------------------------+----------------------------------------------------------------------------+ +| :enum:`return_value_policy::automatic` | **Default policy.** This policy falls back to the policy | +| | :enum:`return_value_policy::take_ownership` when the return value is a | +| | pointer. Otherwise, it uses :enum:`return_value_policy::move` or | +| | :enum:`return_value_policy::copy` for rvalue and lvalue references, | +| | respectively. See above for a description of what all of these different | +| | policies do. | ++--------------------------------------------------+----------------------------------------------------------------------------+ +| :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the | +| | return value is a pointer. This is the default conversion policy for | +| | function arguments when calling Python functions manually from C++ code | +| | (i.e. via handle::operator()). You probably won't need to use this. | ++--------------------------------------------------+----------------------------------------------------------------------------+ + +Return value policies can also be applied to properties: + +.. code-block:: cpp + + class_<MyClass>(m, "MyClass") + .def_property("data", &MyClass::getData, &MyClass::setData, + py::return_value_policy::copy); + +Technically, the code above applies the policy to both the getter and the +setter function, however, the setter doesn't really care about *return* +value policies which makes this a convenient terse syntax. Alternatively, +targeted arguments can be passed through the :class:`cpp_function` constructor: + +.. code-block:: cpp + + class_<MyClass>(m, "MyClass") + .def_property("data" + py::cpp_function(&MyClass::getData, py::return_value_policy::copy), + py::cpp_function(&MyClass::setData) + ); + +.. warning:: + + Code with invalid return value policies might access uninitialized memory or + free data structures multiple times, which can lead to hard-to-debug + non-determinism and segmentation faults, hence it is worth spending the + time to understand all the different options in the table above. + +.. note:: + + One important aspect of the above policies is that they only apply to + instances which pybind11 has *not* seen before, in which case the policy + clarifies essential questions about the return value's lifetime and + ownership. When pybind11 knows the instance already (as identified by its + type and address in memory), it will return the existing Python object + wrapper rather than creating a new copy. + +.. note:: + + The next section on :ref:`call_policies` discusses *call policies* that can be + specified *in addition* to a return value policy from the list above. Call + policies indicate reference relationships that can involve both return values + and parameters of functions. + +.. note:: + + As an alternative to elaborate call policies and lifetime management logic, + consider using smart pointers (see the section on :ref:`smart_pointers` for + details). Smart pointers can tell whether an object is still referenced from + C++ or Python, which generally eliminates the kinds of inconsistencies that + can lead to crashes or undefined behavior. For functions returning smart + pointers, it is not necessary to specify a return value policy. + +.. _call_policies: + +Additional call policies +======================== + +In addition to the above return value policies, further *call policies* can be +specified to indicate dependencies between parameters or ensure a certain state +for the function call. + +Keep alive +---------- + +In general, this policy is required when the C++ object is any kind of container +and another object is being added to the container. ``keep_alive<Nurse, Patient>`` +indicates that the argument with index ``Patient`` should be kept alive at least +until the argument with index ``Nurse`` is freed by the garbage collector. Argument +indices start at one, while zero refers to the return value. For methods, index +``1`` refers to the implicit ``this`` pointer, while regular arguments begin at +index ``2``. Arbitrarily many call policies can be specified. When a ``Nurse`` +with value ``None`` is detected at runtime, the call policy does nothing. + +When the nurse is not a pybind11-registered type, the implementation internally +relies on the ability to create a *weak reference* to the nurse object. When +the nurse object is not a pybind11-registered type and does not support weak +references, an exception will be thrown. + +Consider the following example: here, the binding code for a list append +operation ties the lifetime of the newly added element to the underlying +container: + +.. code-block:: cpp + + py::class_<List>(m, "List") + .def("append", &List::append, py::keep_alive<1, 2>()); + +For consistency, the argument indexing is identical for constructors. Index +``1`` still refers to the implicit ``this`` pointer, i.e. the object which is +being constructed. Index ``0`` refers to the return type which is presumed to +be ``void`` when a constructor is viewed like a function. The following example +ties the lifetime of the constructor element to the constructed object: + +.. code-block:: cpp + + py::class_<Nurse>(m, "Nurse") + .def(py::init<Patient &>(), py::keep_alive<1, 2>()); + +.. note:: + + ``keep_alive`` is analogous to the ``with_custodian_and_ward`` (if Nurse, + Patient != 0) and ``with_custodian_and_ward_postcall`` (if Nurse/Patient == + 0) policies from Boost.Python. + +Call guard +---------- + +The ``call_guard<T>`` policy allows any scope guard type ``T`` to be placed +around the function call. For example, this definition: + +.. code-block:: cpp + + m.def("foo", foo, py::call_guard<T>()); + +is equivalent to the following pseudocode: + +.. code-block:: cpp + + m.def("foo", [](args...) { + T scope_guard; + return foo(args...); // forwarded arguments + }); + +The only requirement is that ``T`` is default-constructible, but otherwise any +scope guard will work. This is very useful in combination with `gil_scoped_release`. +See :ref:`gil`. + +Multiple guards can also be specified as ``py::call_guard<T1, T2, T3...>``. The +constructor order is left to right and destruction happens in reverse. + +.. seealso:: + + The file :file:`tests/test_call_policies.cpp` contains a complete example + that demonstrates using `keep_alive` and `call_guard` in more detail. + +.. _python_objects_as_args: + +Python objects as arguments +=========================== + +pybind11 exposes all major Python types using thin C++ wrapper classes. These +wrapper classes can also be used as parameters of functions in bindings, which +makes it possible to directly work with native Python types on the C++ side. +For instance, the following statement iterates over a Python ``dict``: + +.. code-block:: cpp + + void print_dict(py::dict dict) { + /* Easily interact with Python types */ + for (auto item : dict) + std::cout << "key=" << std::string(py::str(item.first)) << ", " + << "value=" << std::string(py::str(item.second)) << std::endl; + } + +It can be exported: + +.. code-block:: cpp + + m.def("print_dict", &print_dict); + +And used in Python as usual: + +.. code-block:: pycon + + >>> print_dict({'foo': 123, 'bar': 'hello'}) + key=foo, value=123 + key=bar, value=hello + +For more information on using Python objects in C++, see :doc:`/advanced/pycpp/index`. + +Accepting \*args and \*\*kwargs +=============================== + +Python provides a useful mechanism to define functions that accept arbitrary +numbers of arguments and keyword arguments: + +.. code-block:: python + + def generic(*args, **kwargs): + ... # do something with args and kwargs + +Such functions can also be created using pybind11: + +.. code-block:: cpp + + void generic(py::args args, py::kwargs kwargs) { + /// .. do something with args + if (kwargs) + /// .. do something with kwargs + } + + /// Binding code + m.def("generic", &generic); + +The class ``py::args`` derives from ``py::tuple`` and ``py::kwargs`` derives +from ``py::dict``. + +You may also use just one or the other, and may combine these with other +arguments as long as the ``py::args`` and ``py::kwargs`` arguments are the last +arguments accepted by the function. + +Please refer to the other examples for details on how to iterate over these, +and on how to cast their entries into C++ objects. A demonstration is also +available in ``tests/test_kwargs_and_defaults.cpp``. + +.. note:: + + When combining \*args or \*\*kwargs with :ref:`keyword_args` you should + *not* include ``py::arg`` tags for the ``py::args`` and ``py::kwargs`` + arguments. + +Default arguments revisited +=========================== + +The section on :ref:`default_args` previously discussed basic usage of default +arguments using pybind11. One noteworthy aspect of their implementation is that +default arguments are converted to Python objects right at declaration time. +Consider the following example: + +.. code-block:: cpp + + py::class_<MyClass>("MyClass") + .def("myFunction", py::arg("arg") = SomeType(123)); + +In this case, pybind11 must already be set up to deal with values of the type +``SomeType`` (via a prior instantiation of ``py::class_<SomeType>``), or an +exception will be thrown. + +Another aspect worth highlighting is that the "preview" of the default argument +in the function signature is generated using the object's ``__repr__`` method. +If not available, the signature may not be very helpful, e.g.: + +.. code-block:: pycon + + FUNCTIONS + ... + | myFunction(...) + | Signature : (MyClass, arg : SomeType = <SomeType object at 0x101b7b080>) -> NoneType + ... + +The first way of addressing this is by defining ``SomeType.__repr__``. +Alternatively, it is possible to specify the human-readable preview of the +default argument manually using the ``arg_v`` notation: + +.. code-block:: cpp + + py::class_<MyClass>("MyClass") + .def("myFunction", py::arg_v("arg", SomeType(123), "SomeType(123)")); + +Sometimes it may be necessary to pass a null pointer value as a default +argument. In this case, remember to cast it to the underlying type in question, +like so: + +.. code-block:: cpp + + py::class_<MyClass>("MyClass") + .def("myFunction", py::arg("arg") = (SomeType *) nullptr); + +.. _nonconverting_arguments: + +Non-converting arguments +======================== + +Certain argument types may support conversion from one type to another. Some +examples of conversions are: + +* :ref:`implicit_conversions` declared using ``py::implicitly_convertible<A,B>()`` +* Calling a method accepting a double with an integer argument +* Calling a ``std::complex<float>`` argument with a non-complex python type + (for example, with a float). (Requires the optional ``pybind11/complex.h`` + header). +* Calling a function taking an Eigen matrix reference with a numpy array of the + wrong type or of an incompatible data layout. (Requires the optional + ``pybind11/eigen.h`` header). + +This behaviour is sometimes undesirable: the binding code may prefer to raise +an error rather than convert the argument. This behaviour can be obtained +through ``py::arg`` by calling the ``.noconvert()`` method of the ``py::arg`` +object, such as: + +.. code-block:: cpp + + m.def("floats_only", [](double f) { return 0.5 * f; }, py::arg("f").noconvert()); + m.def("floats_preferred", [](double f) { return 0.5 * f; }, py::arg("f")); + +Attempting the call the second function (the one without ``.noconvert()``) with +an integer will succeed, but attempting to call the ``.noconvert()`` version +will fail with a ``TypeError``: + +.. code-block:: pycon + + >>> floats_preferred(4) + 2.0 + >>> floats_only(4) + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + TypeError: floats_only(): incompatible function arguments. The following argument types are supported: + 1. (f: float) -> float + + Invoked with: 4 + +You may, of course, combine this with the :var:`_a` shorthand notation (see +:ref:`keyword_args`) and/or :ref:`default_args`. It is also permitted to omit +the argument name by using the ``py::arg()`` constructor without an argument +name, i.e. by specifying ``py::arg().noconvert()``. + +.. note:: + + When specifying ``py::arg`` options it is necessary to provide the same + number of options as the bound function has arguments. Thus if you want to + enable no-convert behaviour for just one of several arguments, you will + need to specify a ``py::arg()`` annotation for each argument with the + no-convert argument modified to ``py::arg().noconvert()``. + +.. _none_arguments: + +Allow/Prohibiting None arguments +================================ + +When a C++ type registered with :class:`py::class_` is passed as an argument to +a function taking the instance as pointer or shared holder (e.g. ``shared_ptr`` +or a custom, copyable holder as described in :ref:`smart_pointers`), pybind +allows ``None`` to be passed from Python which results in calling the C++ +function with ``nullptr`` (or an empty holder) for the argument. + +To explicitly enable or disable this behaviour, using the +``.none`` method of the :class:`py::arg` object: + +.. code-block:: cpp + + py::class_<Dog>(m, "Dog").def(py::init<>()); + py::class_<Cat>(m, "Cat").def(py::init<>()); + m.def("bark", [](Dog *dog) -> std::string { + if (dog) return "woof!"; /* Called with a Dog instance */ + else return "(no dog)"; /* Called with None, dog == nullptr */ + }, py::arg("dog").none(true)); + m.def("meow", [](Cat *cat) -> std::string { + // Can't be called with None argument + return "meow"; + }, py::arg("cat").none(false)); + +With the above, the Python call ``bark(None)`` will return the string ``"(no +dog)"``, while attempting to call ``meow(None)`` will raise a ``TypeError``: + +.. code-block:: pycon + + >>> from animals import Dog, Cat, bark, meow + >>> bark(Dog()) + 'woof!' + >>> meow(Cat()) + 'meow' + >>> bark(None) + '(no dog)' + >>> meow(None) + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + TypeError: meow(): incompatible function arguments. The following argument types are supported: + 1. (cat: animals.Cat) -> str + + Invoked with: None + +The default behaviour when the tag is unspecified is to allow ``None``. + +.. note:: + + Even when ``.none(true)`` is specified for an argument, ``None`` will be converted to a + ``nullptr`` *only* for custom and :ref:`opaque <opaque>` types. Pointers to built-in types + (``double *``, ``int *``, ...) and STL types (``std::vector<T> *``, ...; if ``pybind11/stl.h`` + is included) are copied when converted to C++ (see :doc:`/advanced/cast/overview`) and will + not allow ``None`` as argument. To pass optional argument of these copied types consider + using ``std::optional<T>`` + +Overload resolution order +========================= + +When a function or method with multiple overloads is called from Python, +pybind11 determines which overload to call in two passes. The first pass +attempts to call each overload without allowing argument conversion (as if +every argument had been specified as ``py::arg().noconvert()`` as described +above). + +If no overload succeeds in the no-conversion first pass, a second pass is +attempted in which argument conversion is allowed (except where prohibited via +an explicit ``py::arg().noconvert()`` attribute in the function definition). + +If the second pass also fails a ``TypeError`` is raised. + +Within each pass, overloads are tried in the order they were registered with +pybind11. + +What this means in practice is that pybind11 will prefer any overload that does +not require conversion of arguments to an overload that does, but otherwise prefers +earlier-defined overloads to later-defined ones. + +.. note:: + + pybind11 does *not* further prioritize based on the number/pattern of + overloaded arguments. That is, pybind11 does not prioritize a function + requiring one conversion over one requiring three, but only prioritizes + overloads requiring no conversion at all to overloads that require + conversion of at least one argument. |