#################### Python API Reference #################### .. contents:: .. default-domain:: py .. module:: links_and_nodes :synopsis: Python wrapper for LN communication services. .. index:: pair: thread safety; Python API Thread-Safety ============= The functions and objects of the Python API are (in difference to the corresponding objects of the C and C++ API) **not** thread-safe. That means it is not possible to create and use objects from it from different threads, unless you make sure that only one thread accesses each object at the same time. The latter can be done by using a lock for each object, and lock it each time the object is accessed. .. Tip:: If your program uses several threads, you should consider to split it into several processes which communicate. .. index:: pair: Python API; os.fork() calls os.fork() calls =============== Calling the ``os.fork()`` system function is not supported by the LN library. This means that ``os.fork()`` should not be called when any LN client instance is active. Otherwise, strange things can happen. .. _reference/python/module: Using the :mod:`links_and_nodes` Module: Public API =================================================== .. _reference/python/client_class: client Class ------------ .. _reference/python/client_class/constructor: client Constructor ^^^^^^^^^^^^^^^^^^ connection to ln-manager is handled by :class:`client` class. .. py:class:: client(client_name, ln_manager_or_args=None) Tries to TCP-connect to the ln-manager. *client_name* should be a string to help identify this client within the ln-manager. This suggestion will be ignored when the :envvar:`LN_PROGRAM_NAME` environment variable var is set, which will take precedence and will be used used instead in this case. The suggested name is only used if no other client instance with that name is already running. The exact behavior is described in :ref:`guide/concepts/clients/client-names` if *ln_manager_or_args* is as string in the form ``HOST:PORT`` of the ln-managers TCP-address. if *ln_manager_or_args* is a list, it is interpreted as command line arguments to be parsed by the LN-client library. this is typically used to specify the ln-manager address like this ``my_client.py -ln_manager HOST:PORT``. when *ln_manager_or_args* is ``None`` the client will read the ln-manager address from the :envvar:`LN_MANAGER` environment variable. Examples:: >> client("client name") # get manager address from LN_MANAGER env-var <_ln.client object at 0x7f0aab44b228> >> client("client name", sys.args[1:]) # pass command line arguments <_ln.client object at 0x7f0aab44b228> >> client("client name", [ "-ln_manager", "HOST:PORT"]) <_ln.client object at 0x7f0aab44b228> .. seealso:: * :c:func:`ln_init()` in the C API * :c:func:`ln_init_to_manager()` in the C API * :c:type:`ln_client` in the C API * :cpp:class:`client` in the C++ API .. _reference/python/service/client_objects: Client Objects -------------- Instances of the :class:`client` class have the following methods: .. py:method:: client.subscribe(topic_name, message_definition_name=None, \ rate=-1, buffers=1, need_reliable_transport=false) Subscribe to named topic and returns an instance of :py:class:`inport`. *topic_name* is the name of the :ref:`topic `. *message_definition_name* is the name of the :term:`message definition` to use If *message_definition_name* is ``None`` and the topic is already published, the message definition name will be retrieved from the ln-manager. .. tip:: See :ref:`the tutorial ` for more information on message definitions. A short summary in :ref:`tutorial/python/summary_api/message-definitions` in the :doc:`tutorial_python` chapter. For more detailed information, see ref:`guide/message_definitions` in the user guide. For detailed information on setting the message definition search path, see section :envvar:`LN_MESSAGE_DEFINITION_DIRS` in the reference part. .. note:: Leaving *message_definition_name* at ``None`` is not really suggested. If the topic is not yet published, the client can not infer the message definition name from the ln_manager and subscription setup will fail. Passing the expected message definition explicitly also makes the code easier to understand and catches mismatches earlier. If the *message_definition_name* is specified but is unknown to the ln-manager, an :exc:`Exception` exception is raised. *rate* is a limit which specifies the maximum sending-rate in messages per second. With the default of ``-1`` the topic will be subscribed without rate-limiter, the publisher decides the update-rate. *buffers* is good to set to at least ``3``. setting this to higher values helps to avoid data-loss. *need_reliable_transport* is of interest when the topic needs to be transported across a network. If set to ``true`` TCP will be used to get a reliable communication channel. Otherwise UDP is used. .. seealso:: * :c:func:`ln_subscribe` (C API) .. py:method:: client.publish(topic_name, message_definition_name, buffers=1) Create an :class:`outport` for a specific topic and message definition. *topic_name* Is the name of the :ref:`topic ` for which the data is published. *message_definition_name* The name if the :term:`message definition` to use (see :ref:`the tutorial introduction on message-passing ` for more information). *buffers* is the number of buffers to use. The default value is 1. If data is lost, set it to a higher value. .. seealso:: * :cpp:func:`client::publish` in the C++ API * :c:func:`ln_publish` (C API) * :c:func:`ln_unpublish` (C API) .. py:method:: client.get_service(client, service_name, interface_name, mainloop) Create a :class:`service` instance (*see also* :term:`service object `). *client* is a :class:`client` instance. Typically, it would be the ``self`` parameter of the object instance. *service_name* is the name of a service. *interface_name* is the name of a service message definition, without any leading slashes. .. tip:: See section :ref:`tutorial/python/summary_api/service-message-definitions` in the :doc:`tutorial_python` chapter for an overview how to define and use them. *mainloop* can be ``None`` for synchronous service calls, or an event-loop object implementing :class:`links_and_nodes_base.mainloop.MainloopInterface` for integrated asynchronous handling. Such objects provide operations like ``iterate()``, ``timeout_add()``, ``idle_add()``, ``fd_add()`` and ``source_remove()``. .. seealso:: * :cpp:func:`ln::client::get_service()` (C++ API) * :c:type:`ln_service` (C API) * :c:func:`ln_service_init` (C API) .. py:method:: client.get_service_provider(service_name, interface_name) Create a :class:`service_provider` instance (*see also* :term:`service object `). *client* is a :class:`client` instance. Typically, it would be the ``self`` parameter of the object instance. *service_name* is the name of a service. *interface_name* is the name of a service message definition, without any leading slashes. .. tip:: See section :ref:`tutorial/python/summary_api/service-message-definitions` in the :doc:`tutorial_python` chapter for an overview how to define and use them. The returned provider object can either be used directly with an explicit handler via :meth:`service_provider.set_handler`, or be integrated into an event loop later via methods such as ``do_register_with_mainloop()``. The expected *mainloop* object should implement :class:`links_and_nodes_base.mainloop.MainloopInterface`. .. seealso:: * :cpp:func:`ln::client::get_service()` (C++ API) * :c:type:`ln_service` (C API) * :c:func:`ln_service_init` (C API) .. py:method:: client.wait_and_handle_service_group_requests(group_name=None, timeout=-1) Process service requests sequentially in the main thread. This function waits for a maximum number of timeout seconds, or blocks if timeout is equal or smaller than zero, until a requests has arrived. Because no code is executed concurrently, no locking is necessary (unless other threads are started by the program). This function can also be called from a GUI main loop, like the Gtk3 / GObject mainloop.Gtk3 / GObject mainloop .. note:: Concurrent calls of ``wait_and_handle_service_group_requests()`` for the same service group are serialized internally. This is a safety guard, not a recommended scheduling model. Prefer one dispatcher thread per service group, or use :py:meth:`client.handle_service_group_in_thread_pool()` when service handlers should run concurrently. *group_name* name of the thread group. The group name can be None. *timeout* timeout for requests. The maximum time, in seconds, that the method will wait for requests before returning. This can be used to handle requests to different service groups in quick succession, by handling them sequentially, while not incurring risk that bugs occur because of incorrect locking. .. seealso:: * :cpp:func:`ln::client::wait_and_handle_service_group_requests()` * :c:func:`ln_wait_and_handle_service_group_requests()` .. py:method:: client.handle_service_group_in_thread_pool(group_name, pool_name) Defines the name of a service group and a thread pool for service threads, and starts to handle service requests. This method is an alternative to :py:meth:`client.wait_and_handle_service_group_requests()`. In difference to the latter, service requests are handled concurrently, which can yield lower latencies or be faster. Important: Any data that is modified during a service request (by the handling thread or by another thread) needs to be protected with mutexes. If in doubt, use always :py:meth:`client.wait_and_handle_service_group_requests()`, because correct programming in multiple threads is more difficult and prone to non-deterministic bugs. *group_name* name of the service group. This value can be set to ``None``. *pool_name* name of the thread pool. This name can be chosen freely, for example it can be set to "main pool". If a pool with that name does not exist, it will be automatically created with a pool size of 1. The number of threads can be set by using the :py:meth:`client.set_max_threads()` method. .. seealso:: * :cpp:func:`ln::client::handle_service_group_in_thread_pool()` in the C++ API * :c:func:`ln_handle_service_group_in_thread_pool()` in the C API .. py:method:: client.set_max_threads(pool_name, n_threads) Sets the maximum number of threads for a thread pool with the given name. Here, the number ``n_threads`` means: 1. Only one service request can be handled at one time, the requests for the corresponding service groups are handled sequentially. 2. Still, only one service request is handled at the same time, but the setup of the connection is done in an own thread. 3. Up to two service requests can be handled at the same time (and so on). .. warning:: Remember that if different service handlers (or any other parts of the same program) access and modify the same data at the same time, it needs to be protected by :term:`mutexes ` or locks, in order to avoid data corruption. Otherwise, non-deterministic bugs can occur. .. seealso:: * :cpp:func:`ln::client::set_max_threads()` in the C++ API * :c:func:`ln_set_max_threads()` in the C API .. _reference/python/port_objects: Port Objects ------------ .. py:class:: port() Is a connection of a subscriber or publisher to send and receive messages of a given type. Ports have two sub-types, one for sending and one for receiving data, .. seealso:: * :c:type:`ln_inport` (C API) * :c:type:`ln_outport` (C API) .. _reference/python/port/members: port.packet ^^^^^^^^^^^^ .. py:attribute:: port.packet Is an instance member which has elements whose name and types are corresponding to each elements of the message definition which the port was configured with. To and from the elements (or members) of the request attribute, values can be assigned and read from. Specifically, they can be assigned to with data to transmit, and elements can be read after data has been received. .. seealso:: * :c:type:`ln_packet` Subscriber Ports ---------------- .. py:class:: inport() An ``inport`` is a port for receiving data via the :meth:`port.read()` function. .. seealso:: * :c:type:`ln_inport` (C API) .. _reference/python/port/method/read: inport methods ^^^^^^^^^^^^^^ .. index:: pair: time stamps of logged messages; port.read() pair: synchronization; of messages exchanged by read() and write() .. py:method:: port.read(blocking_or_timeout) The port ``read()`` method received data which has been transmitted to a port. *blocking_or_timeout* is a parameter which controls :term:`blocking` and :term:`time-out` behavior: * If it is Boolean ``True`` and no data is present, the ``read()`` call will block until data is available. * If it is a floating point number and no data is available, it will wait at most this number of seconds before it returns ``False`` and without transferring data. * If it is a number of zero, a negative number, or a Boolean with the value of ``False``, the port will be :term:`non-blocking`: It will never wait if no data is present, but immediately return ``False``. (For further explanation, see the description :ref:`in the tutorial `.) * in any case, the data is either copied completely or not at all, and if no new data was send, the input buffer will not be changed, keeping its old content. *return value* is the port.packet member (which, when used in an ``if`` statement, has a Boolean value of ``True``) if data could be read, and otherwise ``None`` (which evaluates to Boolean ``False``), meaning that either no data was present, or a time-out was set and the time-out expired without receiving data. The method can throw exceptions of type ``Exception`` if an error occurs. .. seealso:: * :c:func:`ln_read` (C API) * :c:func:`ln_read_timeout` (C API) Publisher Ports --------------- .. py:class:: outport() :noindex: .. seealso:: * :c:type:`ln_outport` (C API) .. _reference/python/port/methods: .. _reference/python/port/method/write: outport Methods ---------------- .. index:: pair: time stamps of logged messages; port.write() pair: synchronization; of messages exchanged by read() and write() .. py:method:: port.write(timestamp=0.0) Write and transmit request data. The data is always transmitted without blocking and completely. *timestamp*: is a floating-point number which can be used as a time-stamp which can flag whether data is fresh, and is used for rate-limiting if a subscriber has set a rate-limit. The required unit is seconds. The function can throw an exception in the case of errors. The method can throw exceptions of type ``Exception`` if an error occurs. .. seealso:: * :c:func:`ln_write` (C API) The Python ``inport`` and ``outport`` objects documented below are wrapper objects implemented in ``python/links_and_nodes/ln_wrappers.py``. They wrap the lower-level LN port objects and expose Python-friendly attributes like ``packet``, ``topic_name``, ``msg_def_name``, ``msg_def_hash`` and ``timestamp``. .. _reference/python/service_class: Classes for instantiating Services ---------------------------------- Service provider (non-wrapped or "direct") version ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. py:method:: service_provider.set_handler(handler_function) Set a handler method for a service provider handle. This user-porivided method will be called when a service request arrives, to handle the request. .. seealso:: * :c:func:`ln_service_provider_set_handler()` * :cpp:func:`ln::service::set_handler()` .. py:method:: service_provider.do_register(service_group_name) Register the service provider with the LN manager, so that service requests will be routed to the handler. *service_group_name* is the name of the :term:`service group` for which requests will be handled in the same thread (by default the main thread). .. py:method:: HANDLE_SERVICE_CALL([self,] conn, req, resp): general prototype of the service request handler. The handler gets called with the request paremeters and the API user's implementation is expected to return the result of the request. Here, *conn* is a connection object which will be called to respond the call. *req* is the request data struct, a class instance whose members are the request fields *resp* is the response data struct, which holds the response fields .. py:method:: connection.respond() Responds the service call by transferring the data in the response struct to the service client. The call returns after the client has received the data. Any error occuring during sending the data will raise an exception here, so when the method call returns, the service provider can be sure that the data has arrived at the client. This can be used for synchronization between clients and provider. .. seealso:: * :cpp:func:`ln::service_request::respond()` (C++ API) * :c:func:`ln_service_request_respond()` (C API) .. py:method:: service_provider.handle_service_group_requests(group_name=None, timeout=-1) Performs the same function as :meth:`client.wait_and_handle_service_group_requests()`, called repeatedly in an endless loop, see below. Process service requests sequentially in the main thread. This function waits repeatedly for a maximum number of timeout seconds, or blocks if timeout is equal or smaller than zero, until a requests has arrived. Because no code is executed concurrently, no locking is necessary (unless other threads are started by the program). This function can also be called from a GUI main loop, like the Gtk3 / GObject mainloop.Gtk3 / GObject mainloop *group_name* name of the thread group. The group name can be None. *timeout* timeout for requests. The maximum time, in seconds, that the method will wait for requests before returning. This can be used to handle requests to different service groups in quick succession, by handling them sequentially, while not incurring risk that bugs occur because of incorrect locking. .. _reference/python/service/provider/wrapper: Service Provider Wrapper Class ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. py:class:: service_provider() Base class with the purpose to derive a service provider class written by the user from it. One must call :meth:`service_provider.__init__()` when using the base class, in order to initialize it. .. py:method:: service_provider.__init__(self, clnt, service_name) Method needed to initialize sub-classes of :class:`service_provider`. *self* is the instance handle of the subclass which is to be initialized. *clnt* is a :class:`client` instance. *service_name* is the name of the service. .. py:method:: service_provider.wrap_service_provider(method_name, interface, \ group_name=None) Method used to register an instance method as the method to be called when a specific service request message is received by the class. *method_name* name of the method, as a string. *interface* Name of the service message definition which is used to transmit both request parameters and return parameters. *group_name* is the name of a service group, which defaults to ``None``. Named service groups allow to execute service request handlers in specific thread pools. (*For details and simple examples on message definitions for services, see the tutorial, starting with section* :ref:`tutorial/lnm/defining-service-message-definitions` ). .. seealso:: * :c:type:`ln_service_handler` (C API) * :c:type:`ln_service_init` (C API) * :c:type:`ln_service_deinit` (C API) * :c:func:`ln_service_provider_set_handler` (C API) * :c:func:`ln_service_provider_register` (C API) * :c:func:`ln_service_request_respond` (C API) .. _reference/python/service/client/wrapper: Service Client Wrapper Class ----------------------------- .. py:class:: services_wrapper() Base class which can be sub-classed to derive a service client. .. py:method:: services_wrapper.__init__(self, clnt, service_name) method which is required to register an instance of a class derived from :class:`services_wrapper` as a service client with the LN manager. This is required to use the class as a client. *self* is the instance handle of the derived class *clnt* is an :class:`client` instance which was registered with the LNM. *service_name* is the name of the service message definition. .. py:class:: ServiceErrorResponse Is an exception which, for LN versions 2.1.0 and newer, will be returned if a call to the service wrapper class fails. (prior to this, a standard Python exception of type ``Exception`` was raised, if (and only if) the ``error_message`` field of a response was non-empty). .. py:method:: services_wrapper.wrap_service(method_name, interface, \ throw_on_these_error_indicators=["error_message", ], \ postprocessor=None, preprocessors=None, \ postprocessors=None) Is a method of :class:`services_wrapper` which can be used to register a specific service call with a method name. *method_name* is the name of the method which will be attached to the instance of the derived class. *interface* is the name of a service message definition which will be used when calling the method, for requesting the service from a service provider. After the registration for a client, say with the name ``c``, the call ``c.(args ...)`` is forwarded to the service provider and the result is returned in a single object (if the response parameters only have a single element) or in a Python dictionary (if the result has several elements), with each key of the dictionary corresponding to an element of the response message definition. *throw_on_these_error_indicators* Is a list of message response fields which will cause an exception of type :class:`ServiceErrorResponse` to be raised (new in LN 2.1.0). *postprocessor* Allows to pass a function which will get passed the whole response as a dictionary, and can be used e.g. for error handling or filtering of the return data. (new in LN 2.1.0) *postprocessors* Is a dictionary where the keys are the names of message response fields, and the values are a function for each which can be used for error handling and response validation. (new in LN 2.1.0) *preprocessors* Is a dictionary where the keys are the names of service request fields, and each value is a function which can filter and validate the request data. (new in LN 2.1.0) For an example to post-processing and use of specific error return flags, see the example in :ref:`python/service/wrapped-api/error-handling` for the description of the wrapped API in page :doc:`quickstart_python_services_wrapped-api`. Service Objects --------------- .. _reference/pyton/service/client/objects: .. py:class:: service() Is a class that allows to call an function in another process, passing input parameters to it and returning result values as output parameters. Class instances can be created by calling :meth:`client.get_service()`, after the client has been registered. ``service`` instances behave similarly to :class:`port` objects, with the difference that they are designed to be used for a kind of :term:`remote procedure call`, while ports can only be used to write or read messages unidirectionally. .. seealso:: * :c:type:`ln_service` (C API) * :c:type:`ln_service_init` (C API) * :c:type:`ln_service_deinit` (C API) Members ^^^^^^^ .. py:attribute:: service.req Is a data member object which receives the service request or input parameters, similarly as with the :attr:`port.packet` data member. .. seealso:: * :c:type:`ln_service_request` (C API) .. py:attribute:: service.resp is a data member which returns the service return parameters, similarly to the members of :attr:`port.packet` of an :class:`inport` instance. Methods ^^^^^^^ .. py:method:: service.call() Calls an LN service by sending the request parameters to the service provider, waiting for a response, and making the result available in the :attr:`service.response` data attribute. .. seealso:: * :cpp:func:`ln::service::call` (CPP API) * :c:func:`ln_service_call` (C API) Private (Internal API) ====================== .. py:class:: port_base() private! Attributes commonly exposed by Python port wrapper objects include: * ``topic_name``: name of the topic * ``msg_def_name``: message definition name used by the port * ``msg_def_hash``: hash of the flattened message definition * ``packet``: the wrapped message object used for reading or writing payload data * ``client``: owning :class:`client` instance .. py:class:: inport() : port_base private! ``inport`` instances additionally expose ``timestamp`` as a floating-point value. After a successful :meth:`inport.read`, it contains the publisher timestamp received with the current packet. .. py:method:: inport.read(blocking_or_timeout=True) .. py:method:: inport.has_publisher() .. py:method:: inport.unblock() .. py:class:: outport() private! .. py:method:: outport.write(timestamp=None) *timestamp* is set to ``clock_gettime(CLOCK_REALTIME)`` if not provided. .. py:method:: outport.has_subscriber() Relevant Source Code Files for further Reference ================================================ :file:`python/links_and_nodes/_ln.cpp` :file:`python/links_and_nodes/ln_wrappers.py` :file:`libln/include/ln/cpp_wrapper.h` :file:`libln/include/ln/cpp_wrapper_impl.h` :file:`libln/include/ln/ln.h`