5.2. Summary of basic Concepts and Features of Links and Nodes

In section What makes it difficult to set up distributed robotic systems? of the Introduction chapter, we explained a number of specific difficulties of distributed real-time systems in robotics, and claimed that links and nodes is addressing these.

The main aspects of distributed real-time system which LN addresses are:

• allow to compose and manage systems from modules with loose coupling, which simplifies to develop and debug components independently, and re-use these components.

• start and manage processes and their start-up dependencies in the right order, and with the right priorities.

• enable very fast inter-process communication between these modules in several ways, mainly via message passing, which is suitable for real-time applications.

Here, we want to explain somewhat more concretely how LN achieves that, so that you can follow the conceptual trail from challenges over goals to why and how specific features are implemented.

Don’t be intimidated when the following list looks a bit dense; there are a lot of concepts and features to cover. We will just list them here, and after that, we will look at them step-by-step:

5.2.1. Process Management and Start-Up Dependencies

• Links and Nodes supports the composition of systems from modules, which run as independent processes, that are connected via inter-process communication. This makes debugging far easier than using facilities such as pthreads (which require expertise to avoid race conditions and similar bugs).

• Similar to the init systems of general-purpose Unix systems like System V init, or systemd, Links and Nodes allows to define dependencies in the order of system start-up so that processes can always count on that other processes they depend on are running.

  In difference to the above systems, however, the means to do this is much more flexible and tailored to evolving experimental systems.

• To make management of processes and their dependencies easier, is is configured via a single configuration file, the LN manager configuration file, which is a standard YAML file. This file is in a human-readable text format and can be managed with version-control software like git, like any other source code file. This makes the configuration reproducible.

• Processes can be controlled via a graphical user interface, or GUI, the LNM GUI. In particular, processes can be stopped and re-started manually, which is very convenient if a part of a system was re-compiled or saved during development, and needs to be re-started, preferably without re-starting other, unrelated parts of a system.

• Processes can be synchronized after initialization

• As an alternative to the GUI, processes can also be controlled via a command-line interface, or CLI.

• It is also possible to launch processes on different computers connected by network links, or hosts, in just the same way, after indicating in the same configuration file on which computer they should run.

• Also, scheduling priorities of each process can be defined via this configuration file.

5.2.2. Isolation from the Environment

• To make distributed systems reliable and reproducible, links and nodes isolates processes from the standard user environment. The reason for this is that, by experience, differences in the user environment set-up are a quite frequent source of errors and surprising behaviors. Instead, Links and Nodes allows to define the process environment in a tightly controlled way, by directives in the configuration file.

5.2.3. Templating for reducing Boiler-Plate

• Links and Nodes also provides ways to reduce tedious and error-prone boilerplate and bookkeeping code in the configuration by providing templates and parameters, which allow to write repeated parts of definitions only once.

5.2.4. Inter-process Communication via Message-passing

• Links and Nodes supports communication between theses processes or components via message passing. Messaging is implemented efficiently depending on whether processes run on the same host, or are connected via a network. If processes run on the same host, transferring data is as fast as copying it into shared RAM and switch locks to allow for its access - this allows latencies down to the nanosecond range.

• The communication is available for several languages, including C, C++, Python2 and Python3.

• This communication is made possible by supporting language-independent message definitions which are automatically mapped to compound data types or objects such as structs in C, or members of object instances in python.

• The main communication paradigm within Links and Nodes is via publish/subscribe communication, while other paradigms are also available. This kind of communication keeps very little internal state which makes it very easy to use in distributed systems.

• Another functionality which makes it easy to call functions and object methods in other processes, including processes on remote hosts, are so-called LN services, which implement remote procedure calls.

• If processes want to participate in this communication, they can do that by using a language-specific API which provides specific functions and methods to send and receive messages. This API is fully network-transparent: It does not differentiate between sending messages to other processes on the same host, or processes on other hosts.

The above list of features is not exhaustive, but is already quite a mouthful from the bottle, and might feel a bit difficult to navigate.

In the next sections, we will look at them step by step and explain how they fit together, starting with the simplest of them. We will do this more or less in the order of the list above, so that related concepts and steps will be grouped with each other.