Architecture

Overview

PyCyphal is a full-featured implementation of the Cyphal protocol stack intended for non-embedded, user-facing applications such as GUI software, diagnostic tools, automation scripts, prototypes, and various R&D cases. It is designed to support GNU/Linux, MS Windows, and macOS as first-class target platforms.

The reader should understand the basics of Cyphal and be familiar with asynchronous programming in Python to read this documentation.

The library consists of several loosely coupled submodules, each implementing a well-segregated part of the protocol:

• pycyphal.dsdl — DSDL language support: transcompilation (code generation) and object serialization. This module is a thin wrapper over Nunavut.

• pycyphal.transport — the abstract Cyphal transport layer model and several concrete transport implementations (Cyphal/CAN, Cyphal/UDP, Cyphal/serial, etc.). This submodule exposes a relatively low-level API where data is represented as serialized blocks of bytes. Users may build custom concrete transports based on this module as well. Typical applications are not expected to use this API directly.

• pycyphal.presentation — this layer binds the transport layer together with DSDL serialization logic, providing a higher-level object-oriented API. At this layer, data is represented as instances of auto-generated Python classes (code generation is managed by pycyphal.dsdl). Typical applications are not expected to use this API directly.

• pycyphal.application — the top-level API for the application. The factory pycyphal.application.make_node() is the main entry point of the library.

• pycyphal.util — a loosely organized collection of various utility functions and classes that are used across the library. User applications may benefit from them also.

Note

In order to use this library the user should at least skim through the API docs for pycyphal.application and check out the Demo.

The overall structure of the library and its mapping onto the Cyphal protocol is shown on the following diagram:

The dependency relations of the submodules are as follows:

Submodule interdependency

Every submodule is imported automatically except the application layer and concrete transport implementation submodules — those must be imported explicitly by the user:

>>> import pycyphal
>>> pycyphal.dsdl.serialize         # OK, the DSDL submodule is auto-imported.
<function serialize at ...>
>>> pycyphal.transport.can          # Not the transport-specific modules though.
Traceback (most recent call last):
...
AttributeError: module 'pycyphal.transport' has no attribute 'can'
>>> import pycyphal.transport.can   # Import the necessary transports explicitly before use.
>>> import pycyphal.transport.serial
>>> import pycyphal.application     # Likewise the application layer -- it depends on DSDL generated classes.

Transport layer

The Cyphal protocol itself is designed to support different transports such as CAN bus (Cyphal/CAN), UDP/IP (Cyphal/UDP), raw serial links (Cyphal/serial), and so on. Generally, a real-time safety-critical implementation of Cyphal would support a limited subset of transports defined by the protocol (often just one) in order to reduce the validation & verification efforts. PyCyphal is different — it is created for user-facing software rather than reliable deeply embedded systems; that is, PyCyphal can’t be put onboard a vehicle, but it can be put onto the computer of an engineer or a researcher building said vehicle to help them implement, understand, validate, verify, and diagnose its onboard network. Hence, PyCyphal trades off simplicity and constrainedness (desirable for embedded systems) for extensibility and repurposeability (desirable for user-facing software).

The library consists of a transport-agnostic core which implements the higher levels of the Cyphal protocol, DSDL code generation, and object serialization. The core defines an abstract transport model which decouples it from transport-specific logic. The main component of the abstract transport model is the interface class pycyphal.transport.Transport, accompanied by several auxiliary definitions available in the same module pycyphal.transport.

The concrete transports implemented in the library are contained in nested submodules; here is the full list of them:

pycyphal.transport.serial.SerialTransport	The Cyphal/Serial transport is designed for OSI L1 byte-level serial links and tunnels, such as UART, RS-422/485/232 (duplex), USB CDC ACM, TCP/IP, etc.
pycyphal.transport.redundant.RedundantTransport	This is a composite over a set of pycyphal.transport.Transport.
pycyphal.transport.udp.UDPTransport	The Cyphal/UDP (IP v4/v6) transport is designed for low-latency, high-throughput, high-reliability vehicular networks based on Ethernet.
pycyphal.transport.can.CANTransport	The standard Cyphal/CAN transport implementation as defined in the Cyphal specification.
pycyphal.transport.loopback.LoopbackTransport	The loopback transport is intended for basic testing and API usage demonstrations.

Important

Typical applications are not expected to initialize their transport manually, or to access this module at all. Initialization of low-level components is fully managed by pycyphal.application.make_node().

Users can implement their own custom transports by subclassing pycyphal.transport.Transport.

Whenever the API documentation refers to monotonic time, the time system of asyncio.AbstractEventLoop.time() is implied. Per asyncio, it defaults to time.monotonic(); it is not recommended to change this. This principle is valid for all other components of the library.

Media sub-layers

Typically, a given concrete transport implementation would need to support multiple different lower-level communication mediums for the sake of application flexibility. Such lower-level implementation details fall outside of the scope of the Cyphal transport model entirely, but they are relevant for this library as we want to encourage consistent design across the codebase. Such lower-level modules are called media sub-layers.

Media sub-layer implementations should be located under the submodule called media, which in turn should be located under its parent transport’s submodule, i.e., pycyphal.transport.*.media.*. The media interface class should be pycyphal.transport.*.media.Media; derived concrete implementations should be suffixed with *Media, e.g., SocketCANMedia. Users may implement their custom media drivers for use with the transport by subclassing Media as well.

Take the CAN media sub-layer for example; it contains the following classes (among others):

• pycyphal.transport.can.media.socketcan.SocketCANMedia

• pycyphal.transport.can.media.pythoncan.PythonCANMedia

Media sub-layer modules should not be auto-imported. Instead, the user should import the required media sub-modules manually as necessary. This is important because sub-layers may have specific dependency requirements which are not guaranteed to be satisfied in all deployments; also, unnecessary submodules slow down package initialization and increase the memory footprint of the application, not to mention possible software reliability issues.

Some transport implementations may be entirely monolithic, without a dedicated media sub-layer. For example, see pycyphal.transport.serial.SerialTransport.

Redundant pseudo-transport

The pseudo-transport pycyphal.transport.redundant.RedundantTransport is used to operate with Cyphal networks built with redundant transports. In order to initialize it, the application should first initialize each of the physical transports and then supply them to the redundant pseudo-transport instance. Afterwards, the configured instance is used with the upper layers of the protocol stack, as shown on the diagram.

The Cyphal Specification adds the following remark on redundant transports:

Reassembly of transfers from redundant interfaces may be implemented either on the per-transport-frame level or on the per-transfer level. The former amounts to receiving individual transport frames from redundant interfaces which are then used for reassembly; it can be seen that this method requires that all transports in the redundant group use identical application-level MTU (i.e., same number of transfer pay-load bytes per frame). The latter can be implemented by treating each transport in the redundant group separately, so that each runs an independent transfer reassembly process, whose outputs are then deduplicated on the per-transfer level; this method may be more computationally complex but it provides greater flexibility.

Per this classification, PyCyphal implements per-transfer redundancy.

Advanced network diagnostics: sniffing/snooping, tracing, spoofing

Packet capture (aka sniffing or snooping) and their further analysis (either real-time or postmortem) are vital for advanced network diagnostics or debugging. While existing general-purpose solutions like Wireshark, libpcap, npcap, SocketCAN, etc. are adequate for low-level access, they are unsuitable for non-trivial use cases where comprehensive analysis is desired.

Certain scenarios require emission of spoofed traffic where some of its parameters are intentionally distorted (like fake source address). This may be useful for implementing complex end-to-end tests for Cyphal-enabled equipment, running HITL/SITL simulation, or validating devices for compliance against the Cyphal Specification.

These capabilities are covered by the advanced network diagnostics API exposed by the transport layer:

• pycyphal.transport.Transport.begin_capture() — capturing on a transport refers to monitoring low-level network events and packets exchanged over the network even if they neither originate nor terminate at the local node.

• pycyphal.transport.Transport.make_tracer() — tracing refers to reconstructing high-level processes that transpire on the network from a sequence of captured low-level events. Tracing may take place in real-time (with PyCyphal connected to a live network) or offline (with events read from a black box recorder or from a log file).

• pycyphal.transport.Transport.spoof() — spoofing refers to faking network transactions as if they were coming from a different node (possibly a non-existent one) or whose parameters are significantly altered (e.g., out-of-sequence transfer-ID).

These advanced capabilities exist alongside the main communication logic using a separate set of API entities because their semantics are incompatible with regular applications.

Virtualization

Some transports support virtual interfaces that can be used for testing and experimentation instead of physical connections. For example, the Cyphal/CAN transport supports virtual CAN buses via SocketCAN, and the serial transport supports TCP/IP tunneling and local loopback mode.

DSDL support

The DSDL support module pycyphal.dsdl is used for automatic generation of Python classes from DSDL type definitions. The auto-generated classes have a high-level application-facing API and built-in auto-generated serialization and deserialization routines.

By default, pycyphal installs an import hook, which automatically compiles DSDLs on import (if not yet compiled). Import hook is triggered when all other import handlers fail (local folder or PYTHONPATH). The import hook then checks for a root namespace matching imported module name inside one of the paths in the CYPHAL_PATH environment variable. If found, DSDL root namespace is compiled into output directory given by the PYCYPHAL_PATH environment variable, or if not provided, into ~/.pycyphal (or OS equivalent). The default import hook can be disabled by setting the PYCYPHAL_NO_IMPORT_HOOK environment variable to 1.

The main API entries are:

• pycyphal.dsdl.compile() — transcompiles a DSDL namespace into a Python package. Normally, one should rely on the import hook instead of invoking this directly.

• pycyphal.dsdl.serialize() and pycyphal.dsdl.deserialize() — serialize and deserialize an instance of an autogenerated class. These functions are wrappers of the Nunavut generated support functions in nunavut_support.py.

• pycyphal.dsdl.to_builtin() and pycyphal.dsdl.update_from_builtin() — used to convert a DSDL object instance to/from a simplified representation using only built-in types such as dict, list, int, float, str, and so on. These can be used as an intermediate representation for conversion to/from JSON, YAML, and other commonly used serialization formats. These functions are wrappers of the Nunavut generated support functions in nunavut_support.py.

Presentation layer

The role of the presentation layer submodule pycyphal.presentation is to provide a high-level object-oriented interface and to route data between port instances (publishers, subscribers, RPC-clients, and RPC-servers) and their transport sessions.

A typical application is not expected to access the presentation-layer API directly; instead, it should rely on the higher-level API entities provided by pycyphal.application.

Application layer

Submodule pycyphal.application provides the top-level API for the application and implements certain standard application-layer functions defined by the Cyphal Specification (chapter 5 Application layer). The main entry point of the library is pycyphal.application.make_node().

This submodule requires the standard DSDL namespace uavcan to be compiled, so it is not auto-imported. A typical usage scenario is to either distribute compiled DSDL namespaces together with the application, or to generate them lazily relying on the import hook.

Chapter Demo contains a complete usage example.

High-level functions

There are several submodules under this one that implement various application-layer functions of the protocol. Here is the full list them:

pycyphal.application.diagnostic	This module implements forwarding between the standard subject uavcan.diagnostic.Record and Python's standard logging facilities (logging).
pycyphal.application.file
pycyphal.application.heartbeat_publisher	Publishes uavcan.node.Heartbeat periodically and provides a couple of basic auxiliary services; see pycyphal.application.heartbeat_publisher.HeartbeatPublisher.
pycyphal.application.node_tracker	Keeps track of online nodes by subscribing to uavcan.node.Heartbeat and requesting uavcan.node.GetInfo when necessary; see pycyphal.application.node_tracker.NodeTracker.
pycyphal.application.plug_and_play	Plug-and-play node-ID allocation logic.
pycyphal.application.register	Implementation of the Cyphal register interface as defined in the Cyphal Specification (section 5.3 Application-layer functions).

Excepting some basic functions that are always initialized by default (like heartbeat or the register interface), these modules are not auto-imported.

Utilities

Submodule pycyphal.util contains a loosely organized collection of minor utilities and helpers that are used by the library and are also available for reuse by the application.