pycyphal.transport.commons.high_overhead_transport package

Module contents

This module contains common classes and algorithms used in a certain category of transports which we call High Overhead Transports. They are designed for highly capable mediums where packets are large and data transfer speeds are high.

For example, UDP, Serial, and IEEE 802.15.4 are high-overhead transports. CAN, on the other hand, is not a high-overhead transport; none of the entities defined in this module can be used with CAN.

class pycyphal.transport.commons.high_overhead_transport.Frame(priority: Priority, transfer_id: int, index: int, end_of_transfer: bool, payload: memoryview)

Bases: object

The base class of a high-overhead-transport frame. It is used with the common transport algorithms defined in this module. Concrete transport implementations should make their transport-specific frame dataclasses inherit from this class. Derived types are recommended to not override __repr__().

priority: Priority

Transfer priority should be the same for all frames within the transfer.

transfer_id: int

Transfer-ID is incremented whenever a transfer under a specific session-specifier is emitted. Always non-negative.

index: int

Index of the frame within its transfer, starting from zero. Always non-negative.

end_of_transfer: bool

True for the last frame within the transfer.

__delattr__(name)

__eq__(other)

__hash__()

__init__(priority: Priority, transfer_id: int, index: int, end_of_transfer: bool, payload: memoryview) → None

__match_args__ = ('priority', 'transfer_id', 'index', 'end_of_transfer', 'payload')

__setattr__(name, value)

payload: memoryview

The data carried by the frame. Multi-frame transfer payload is suffixed with its CRC32C. May be empty.

property single_frame_transfer: bool

__repr__() → str

If the payload is unreasonably long for a sensible string representation, it is truncated and suffixed with an ellipsis.

pycyphal.transport.commons.high_overhead_transport.serialize_transfer(fragmented_payload: Sequence[memoryview], max_frame_payload_bytes: int, frame_factory: Callable[[int, bool, memoryview], pycyphal.transport.commons.high_overhead_transport.FrameType]) → Iterable[pycyphal.transport.commons.high_overhead_transport.FrameType]

Constructs an ordered sequence of frames ready for transmission from the provided data fragments. Compatible with any high-overhead transport.

Parameters:

• fragmented_payload – The transfer payload we’re going to be sending.

• max_frame_payload_bytes – Max payload per transport-layer frame.

• frame_factory – A callable that accepts (frame index, end of transfer, payload) and returns a frame. Normally this would be a closure.

Returns:

An iterable that yields frames.

>>> import dataclasses
>>> from pycyphal.transport.commons.high_overhead_transport import Frame
>>> @dataclasses.dataclass(frozen=True)
... class MyFrameType(Frame):
...     pass    # Transport-specific definition goes here.
>>> priority = pycyphal.transport.Priority.NOMINAL
>>> transfer_id = 12345
>>> def construct_frame(index: int, end_of_transfer: bool, payload: memoryview) -> MyFrameType:
...     return MyFrameType(priority=priority,
...                        transfer_id=transfer_id,
...                        index=index,
...                        end_of_transfer=end_of_transfer,
...                        payload=payload)
>>> frames = list(serialize_transfer(
...     fragmented_payload=[
...         memoryview(b'He thought about the Horse: '),         # The CRC of this quote is 0xDDD1FF3A
...         memoryview(b'how was she doing there, in the fog?'),
...     ],
...     max_frame_payload_bytes=53,
...     frame_factory=construct_frame,
... ))
>>> frames
[MyFrameType(..., index=0, end_of_transfer=False, ...), MyFrameType(..., index=1, end_of_transfer=True, ...)]
>>> bytes(frames[0].payload)    # 53 bytes long, as configured.
b'He thought about the Horse: how was she doing there, '
>>> bytes(frames[1].payload)    # The stuff at the end is the four bytes of multi-frame transfer CRC.
b'in the fog?:\xff\xd1\xdd'

>>> single_frame = list(serialize_transfer(
...     fragmented_payload=[
...         memoryview(b'FOUR'),
...         ],
...         max_frame_payload_bytes=8,
...         frame_factory=construct_frame,
... ))
>>> single_frame
[MyFrameType(..., index=0, end_of_transfer=True, ...)]
>>> bytes(single_frame[0].payload)    # 8 bytes long, as configured.
b'FOUR-\xb8\xa4\x81'

class pycyphal.transport.commons.high_overhead_transport.TransferReassembler(source_node_id: int, extent_bytes: int, on_error_callback: Callable[[Error], None])

Bases: object

Multi-frame transfer reassembly logic is arguably the most complex part of any Cyphal transport implementation. This class implements a highly transport-agnostic transfer reassembly state machine designed for use with high-overhead transports, such as UDP, Serial, IEEE 802.15.4, etc. Any transport whose frame dataclass implementation derives from Frame can use this class.

Out-of-order frame reception is supported, and therefore the reassembler can be used with redundant interfaces directly, without preliminary frame deduplication procedures or explicit interface index assignment, provided that all involved redundant interfaces share the same MTU setting. OOO support includes edge cases where the first frame of a transfer is not received first and/or the last frame is not received last.

OOO is required for frame-level modular transport redundancy (more than one transport operating concurrently) and temporal transfer redundancy (every transfer repeated several times to mitigate frame loss). The necessity of OOO is due to the fact that frames sourced concurrently from multiple transport interfaces and/or frames of a temporally redundant transfer where some of the frames are lost result in an out-of-order arrival of the frames. Additionally, various non-vehicular and/or non-mission-critical networks (such as conventional IP networks) may deliver frames out-of-order even without redundancy.

Distantly relevant discussion: https://github.com/OpenCyphal/specification/issues/8.

A multi-frame transfer shall not contain frames with empty payload.

class Error(value)

Bases: Enum

Error states that the transfer reassembly state machine may encounter. Whenever an error is encountered, the corresponding error counter is incremented by one, and a verbose report is dumped into the log at the DEBUG level.

INTEGRITY_ERROR = 1

A transfer payload did not pass integrity checks. Transfer discarded.

UNEXPECTED_TRANSFER_ID = 2

The transfer-ID of a frame does not match the anticipated value.

MULTIFRAME_MISSING_FRAMES = 3

New transfer started before the old one could be completed. Old transfer discarded.

MULTIFRAME_EMPTY_FRAME = 4

A frame without payload received as part of a multiframe transfer (not permitted by Specification). Only single-frame transfers can have empty payload.

MULTIFRAME_EOT_MISPLACED = 5

The end-of-transfer flag is set in a frame with index N, but the transfer contains at least one frame with index > N. Transfer discarded.

MULTIFRAME_EOT_INCONSISTENT = 6

The end-of-transfer flag is set in frames with indexes N and M, where N != M. Transfer discarded.

__init__(source_node_id: int, extent_bytes: int, on_error_callback: Callable[[Error], None])

Parameters:

• source_node_id – The remote node-ID whose transfers this instance will be listening for. Anonymous transfers cannot be multi-frame transfers, so they are to be accepted as-is without any reassembly activities.

• extent_bytes – The maximum number of payload bytes per transfer. Payload that exceeds this size limit may be implicitly truncated (in the Specification this behavior is described as “implicit truncation rule”). This value can be derived from the corresponding DSDL definition. Note that the reassembled payload may still be larger than this value.

• on_error_callback – The callback is invoked whenever an error is detected. This is intended for diagnostic purposes only; the error information is not actionable. The error is logged by the caller at the DEBUG verbosity level together with reassembly context info.

process_frame(timestamp: Timestamp, frame: Frame, transfer_id_timeout: float) → TransferFrom | None

Updates the transfer reassembly state machine with the new frame.

Parameters:

• timestamp – The reception timestamp from the transport layer.

• frame – The new frame.

• transfer_id_timeout – The current value of the transfer-ID timeout.

Returns:

A new transfer if the new frame completed one. None if the new frame did not complete a transfer.

Raises:

Nothing.

property source_node_id: int

__repr__() → str

static construct_anonymous_transfer(timestamp: Timestamp, frame: Frame) → TransferFrom | None

A minor helper that validates whether the frame is a valid anonymous transfer (it is if the index is zero, the end-of-transfer flag is set and crc checks out) and constructs a transfer instance if it is. Otherwise, returns None. Observe that this is a static method because anonymous transfers are fundamentally stateless.

pycyphal.transport.commons.high_overhead_transport.TransferCRC

alias of CRC32C

class pycyphal.transport.commons.high_overhead_transport.AlienTransferReassembler(source_node_id: int)

Bases: object

This is a wrapper over TransferReassembler optimized for tracing rather than real-time communication. It implements heuristics optimized for diagnostics and inspection rather than real-time operation.

The caller is expected to keep a registry (dict) of session tracers indexed by their session specifiers, which are extracted from captured transport frames.

__init__(source_node_id: int) → None

process_frame(timestamp: Timestamp, frame: Frame) → TransferFrom | Error | None

property transfer_id_timeout: float

The current value of the auto-deduced transfer-ID timeout. It is automatically adjusted whenever a new transfer is received.