host.transfer¶

File transfer utilities for UnixHost.

Defines the canonical TransferProgressHandler callback type used across all transfer protocols (SCP, SFTP, FTP, netcat). Rich (or any other progress reporting library) lives only in make_rich_progress_handler — changing Rich’s API requires touching nothing else in the transfer stack.

Callback signature mirrors asyncssh’s progress_handler so that SCP and SFTP can forward it directly without any adaptation layer.

otto.host.transfer.make_rich_progress_handler(progress, host_name)¶

Return a TransferProgressHandler that drives the given Rich Progress bar.

One task is created per source file, detected by a change in src_path. The caller is responsible for the Progress context (entering and exiting it).

Example:

with make_transfer_progress() as progress:
    handler = make_rich_progress_handler(progress, host_name=host.hostname)
    status, err = await host.get(files, dest, progress_handler=handler)

Return type:¶: Callable[[str, str, int, int], None]

otto.host.transfer.make_rich_progress_factory(progress, host_name)¶

Return a factory that creates a fresh TransferProgressHandler per file.

Each call to the returned factory produces an independent handler with its own closure state, so concurrent transfers don’t share progress tracking.

Example:

with make_transfer_progress() as progress:
    factory = make_rich_progress_factory(progress, host_name=host.name)
    status, err = await host.put(files, dest)

Return type:¶: Callable[[], Callable[[str, str, int, int], None]]

otto.host.transfer.make_transfer_progress()¶

Return a pre-configured Rich Progress suited for file transfers.

Return type:¶: Progress

class otto.host.transfer.TransferContext(transfer, host_name, max_filename_len=255, exec_cmd=None, connections=None, nc_options=None, scp_options=None, get_local_ip=None, filesystem=None)¶

Bases: object

Construction inputs a host provides to build its transfer backend via BaseFileTransfer.create(). The frozen public seam for custom transfer backends. Carries the union of what any family’s built-ins receive at their call sites; a unix backend reads the unix fields, an embedded backend the embedded ones. Selector validation (host-family applicability) runs before construction, so a backend never sees a ctx missing the fields it needs.

transfer : --is-rst--:py:class:`str`¶

host_name : --is-rst--:py:class:`str`¶

max_filename_len : --is-rst--:py:class:`int` = 255¶

exec_cmd : --is-rst--:py:class:`~collections.abc.Callable`\ \[:py:data:`...<Ellipsis>`, :py:class:`~collections.abc.Coroutine`\ \[:py:data:`~typing.Any`, :py:data:`~typing.Any`, :py:class:`~otto.utils.CommandStatus`]] | :py:obj:`None` = None¶

connections : --is-rst--:py:class:`~otto.host.connections.ConnectionManager` | :py:obj:`None` = None¶

nc_options : --is-rst--:py:class:`~otto.host.options.NcOptions` | :py:obj:`None` = None¶

scp_options : --is-rst--:py:class:`~otto.host.options.ScpOptions` | :py:obj:`None` = None¶

get_local_ip : --is-rst--:py:class:`~collections.abc.Callable`\ \[\[], :py:class:`str`] | :py:obj:`None` = None¶

filesystem : --is-rst--:py:class:`~otto.host.embedded_filesystem.EmbeddedFileSystem` | :py:obj:`None` = None¶

otto.host.transfer.validate_filename_lengths(files, limit, host_name)¶

Reject files whose basename exceeds the host’s filesystem cap.

Shared by FileTransfer (Unix) and EmbeddedFileTransfer (embedded) so every backend surfaces the same self-explaining error. Without this guard the failure modes are: :rtype: tuple[Status, str]

Unix SCP/SFTP/FTP: server returns File name too long (errno 36), mid-transfer, after the local file is already read.
Embedded FAT (8.3, no LFN) or LittleFS over NAME_MAX: device fails fs_open with -ENOENT, giving no hint that the name was the problem.

Returns (Status.Success, '') when every basename fits.

class otto.host.transfer.BaseFileTransfer(name, max_filename_len=255)¶

Bases: ABC

Shared API + progress plumbing for any file-transfer backend.

The public put_files / get_files surface (filename-length validation, shared Rich progress acquisition) is owned by this base. Concrete backends (Unix’s FileTransfer, embedded’s EmbeddedFileTransfer, and any future ones such as TFTP) implement two abstract methods — _run_put and _run_get — both of which receive a TransferProgressFactory and are responsible for invoking it at least once per source file, terminating with bytes_done == bytes_total to mark completion.

The progress-bar capability is enforced at the type system level: abc.abstractmethod refuses to instantiate a subclass that omits either method, so a new backend cannot be defined without supplying a way to report progress. The runtime contract test (TestTransferProgressContract) verifies the factory is actually invoked, not just that the methods exist.

host_families : --is-rst--:py:class:`frozenset`\ \[:py:class:`str`] = frozenset({})¶

a subset of {'unix', 'embedded'}. Subclasses declare it; the spec field_validator rejects a backend on a host of the wrong family. A backend with an empty set can never validate and is rejected at registration.

Type:¶: Host families this backend serves

classmethod create(ctx)¶

Build a transfer backend from a TransferContext.

The uniform construction seam (WS#4). Concrete backends override this to run their exact construction against the ctx fields they need. Not an abstractmethod deliberately: only registered built-ins are ever constructed through create, and test doubles that subclass BaseFileTransfer only to exercise the progress contract must not be forced to implement it.

Return type:¶: BaseFileTransfer

async put_files(src_files, dest_dir, show_progress=True)¶

Return type:¶: tuple[Status, str]

async get_files(src_files, dest_dir, show_progress=True)¶

Return type:¶: tuple[Status, str]

otto.host.transfer.NcPortStrategy¶

Strategy for finding free ports on the remote host for netcat transfers.

Available strategies:

'auto' (default) — try each built-in strategy in order (ss → netstat → python → proc) and cache the first one that succeeds.
'ss' — parse ss -tln output to find unused ports.
'netstat' — parse netstat -tln output (fallback for hosts without ss).
'python' — bind a socket to port 0 via a python/python3 one-liner and let the OS assign a free port.
'proc' — read /proc/net/tcp directly (Linux-only, always available as a last resort).
'custom' — run the shell command specified in nc_port_cmd; the command must print a free port number to stdout.

alias of Literal[‘auto’, ‘ss’, ‘netstat’, ‘python’, ‘proc’, ‘custom’]

otto.host.transfer.NcListenerCheck¶

Strategy for checking if a remote nc listener is ready.

Available strategies:

'auto' (default) — probe for ss, then netstat, falling back to proc. The first tool found is cached and reused for subsequent checks.
'ss' — check for a LISTEN socket via ss -tln sport = :<port>.
'netstat' — grep netstat -tln output for the port.
'proc' — scan /proc/net/tcp for LISTEN state (0A) on the port (Linux-only, always available as a last resort).
'custom' — run the shell command specified in nc_listener_cmd with a {port} placeholder. Must exit 0 when the port is listening.

alias of Literal[‘auto’, ‘ss’, ‘netstat’, ‘proc’, ‘custom’]

class otto.host.transfer.FileTransfer(connections, name, transfer, nc_options, scp_options, get_local_ip, exec_cmd, max_filename_len=255)¶

Bases: BaseFileTransfer

Handles all file-transfer protocols (SCP, SFTP, FTP, netcat) for a UnixHost.

Receives injectable callables for open_session and oneshot so it can be tested without real connections.

Inherits put_files / get_files from BaseFileTransfer; implements the abstract _run_put / _run_get as protocol dispatch on self.transfer.

host_families : --is-rst--:py:class:`frozenset`\ \[:py:class:`str`] = frozenset({'unix'})¶

Type:¶: Host families this backend serves

classmethod create(ctx)¶

Build a transfer backend from a TransferContext.

Return type:¶: FileTransfer

async prepare()¶

Resolve port + listener strategies in a single round-trip.

Runs the shared _STRATEGY_PROBE script through _control_run so the port and listener strategies are resolved up front rather than lazily at first-transfer time. Idempotent — a second call with both strategies already cached is a no-op.

Callers use _warmup_for_transfer to run this concurrently with exec-pool warming; direct callers can invoke prepare() alone.

If the probe itself fails (non-zero exit, malformed output), the caches stay unset and the lazy cascades in _find_free_port_auto / _resolve_listener_strategy still kick in as fallbacks.

Return type:¶: None

otto.host.transfer.register_transfer_backend(name, cls)¶

Make a custom transfer backend available to lab data under name.

Call from an init module listed in .otto/settings.toml. The backend must declare a non-empty BaseFileTransfer.host_families; otherwise it could never validate against any host and is rejected here.

Return type:¶: None

otto.host.transfer.build_transfer_backend(name)¶

Return the transfer-backend class registered under name.

Return type:¶: type[BaseFileTransfer]

Raises¶

ValueError: If name is not registered; the message lists the registered names.