cli.test¶

Run a registered OttoSuite test suite.

Each test suite decorated with @register_suite() appears as a subcommand of otto test. Suite-specific options (declared in the suite’s inner Options dataclass) are automatically registered as Typer parameters with full type enforcement and --help documentation.

Markers

integration: Requires live Vagrant VMs. Skip with --markers "not integration".
timeout(seconds): Fail the test if it runs longer than seconds.
retry(n): Retry a failing test up to n times before reporting failure.

Listing tests

--list-suites List test suites with run syntax and exit.

Options on ``otto test`` (before the suite name)

--markers / -m EXPRESSION: pytest -m marker expression applied after collection.
--iterations / -i N: Repeat each test N times within a single setup/teardown cycle (0 = disabled).
--duration / -d SECONDS: Repeat tests for N seconds within a single setup/teardown cycle (0 = disabled).
--threshold FLOAT: Minimum per-test pass rate percentage required in stability mode (0-100, default: 100).
--results PATH: Write test results (JUnit XML) to PATH (default: auto-written to the log directory).

When both --iterations and --duration are specified, testing stops when either limit is reached first.

--cov: Fetch .gcda files from remote hosts after the suite finishes and place them in a cov/ directory in the suite’s output directory.
--cov-dir PATH: Write coverage data to PATH instead of the default <output_dir>/cov. Implies --cov. The directory is created if missing; if it already exists and is non-empty, the command aborts unless --overwrite-cov-dir is also given.
--overwrite-cov-dir: Clear the contents of the --cov-dir destination before the run so stale data from a previous invocation cannot be mixed with the new results.
--cov-clean / --no-cov-clean: Delete .gcda files on remote hosts before the test run. Enabled by default; use --no-cov-clean to keep stale data.
--cov-report / -r: After coverage collection, render an HTML report. Implies --cov. Default location: <output_dir>/cov_report.
--cov-report-dir PATH: Write the HTML report to PATH instead of the default. Implies --cov-report (and therefore --cov). Empty/overwrite rules match --cov-dir: created if missing, aborts if non-empty unless --overwrite-cov-report-dir is also given.
--overwrite-cov-report-dir: Clear the contents of --cov-report-dir before the report is rendered.
--project-name STR: Title shown in the HTML report header (only used with --cov-report).
--monitor: Enable host performance monitoring for the duration of the run. Samples every host (or those matched by --monitor-hosts) on a fixed interval and emits per-test start/end events automatically. At the end of the run a JSON snapshot of all metrics and events is written to <output_dir>/monitor.json.
--monitor-interval SECONDS: Sampling interval for --monitor (default: 5).
--monitor-output PATH: Override the destination for the captured monitor data. Format inferred from the suffix: .json (default) writes a self-contained snapshot, .db writes a SQLite database loadable via otto monitor --file.
--monitor-hosts REGEX: Restrict --monitor to host IDs matching this regex (re.search).

Examples:

otto test --list-suites
otto test TestMyDevice --help
otto test TestMyDevice --device-type switch --firmware 2.1
otto test --iterations 50 --threshold 95 TestMyDevice
otto test --duration 300 --threshold 90 TestMyDevice
otto test --iterations 100 --duration 60 TestMyDevice
otto test --cov TestMyDevice
otto test --cov-dir /tmp/myrun TestMyDevice
otto test --cov-dir /tmp/myrun --overwrite-cov-dir TestMyDevice
otto test --cov --no-cov-clean TestMyDevice
otto test --cov --cov-report TestMyDevice
otto test -r --cov-report-dir /tmp/myreport TestMyDevice

otto.cli.test.resolve_suite(suite, repos)¶

Expand a sutDir-relative suite path to an absolute path for pytest.

Return type:¶: str

otto.cli.test.run_suite(suite_class, suite_file, opts_instance)¶

Execute a registered suite via pytest.main().

Runner options (--markers, --iterations, --duration, --threshold, --results) and coverage options (--cov / --cov-clean) are read from the parent Typer context set by the otto test callback.

Return type:¶: None

otto.cli.test.list_suites_callback(value)¶

Return type:¶: None

otto.cli.test.main(ctx, list_suites=False, markers='', iterations=0, duration=0, threshold=100.0, results='', cov=False, cov_dir=None, overwrite_cov_dir=False, cov_clean=True, cov_report=False, cov_report_dir=None, overwrite_cov_report_dir=False, project_name='Coverage Report', monitor=False, monitor_interval=5.0, monitor_output=None, monitor_hosts=None)¶

Return type:¶: None