otto cov — the coverage pipeline

The problem: embedded and cross-compiled products execute where no coverage tooling runs. gcov counters (.gcda) accumulate on the target — in memory or on an on-device filesystem — while the compile-time graph (.gcno) and sources live in the build tree on the runner. Neither side alone can make a report; this pipeline marries them.

Roadmap items pending

The coverage-tier collection model — declarative tiers, otto cov get / otto cov clean, pinned per-board captures, the committed manual store — ships in this release. Remaining roadmap items (todo/coverage_roadmap.md and the plan’s later phases) — fail-under thresholds, console summaries, per-ticket rollups, embedded counter reset for cov clean, wiring custom exclusion markers into lcov’s percentages — are explicitly not in it.

coverage test otto test --cov / otto cov get instrumented run or retrieval fetch fetch .gcda from covered hosts (transfer on Unix, console extraction embedded) test->fetch correlate correlate match .gcda ↔ .gcno graph, remap sysroot paths, merge hosts + runs (lcov) fetch->correlate capture capture.json per board: parsed hits, git-pinned coordinates correlate->capture err CoverageDataMismatchError stale build → instructions, not a wrong report correlate->err stamp mismatch render otto cov report captures + unit harvest + manual store → HTML capture->render

The stages (packages otto.coverage.fetcher → correlator → capture → renderer → reporter):

  1. Fetch — pull .gcda data from each covered host after the run. Fetchers are per-family: file transfer for Unix hosts, console extraction for embedded targets. Which hosts are covered is repo-declared — the [coverage].hosts regex in settings.toml — never inferred, so hop hosts and uninstrumented beds can’t sneak into a report.

  2. Correlate — match counters to the build tree’s .gcno graph and remap embedded/sysroot paths back to source paths, merging counters across hosts and runs (lcov semantics).

  3. Capture — freeze the merged result into a pinned per-board capture.json: parsed hits in committed-code coordinates, anchored to the repo’s HEAD and per-file blob SHAs.

  4. Render / report — otto cov report assembles every tier — e2e captures, a fresh unit-tier harvest, the committed manual store — into an HTML report plus summary tiers.

The correlator’s core invariant is build/counter identity: .gcda files are only meaningful against the exact .gcno graph the binary was compiled with. That pairing happens once, at collection — the capture holds parsed hits, so the report step never touches the build tree again and a later rebuild cannot invalidate it (a capture’s own guard is its git pin, which must match HEAD at report time). When the raw pairing disagrees — a stale or partially rebuilt product tree at collection time, or a pre-capture run directory re-merged via the legacy fallback — the pipeline stops with a diagnostic error that names the mismatch and the rebuild that fixes it, rather than a gcov stack trace or a silently wrong report. That fail-with-instructions posture is a house rule (Design principles).

Tiers and what is committed

Coverage is organized into tiers — system (e2e), unit, manual, or any other name — declared in .otto/settings.toml under [coverage.tiers] with a kind (e2e / unit / manual) that selects how otto collects that tier’s data. Only the manual tier’s data is pinned and committed into the repo: selecting a manual-kind tier on otto cov get copies the capture into the repo’s committed store at .otto/coverage/manual/ — proof of a manual test session that travels with the code and is PR-reviewable. E2e data lives in each test run’s output directory, and unit data is harvested fresh from the build tree’s harvest_dirs at report time.

otto cov report assembles a store from all three sources per tier kind: e2e captures from the given output directories (behind the pin guard above), the unit harvest, and every committed manual capture — loaded automatically, no path needed. A report-time validity pass (otto.coverage.validity) anchors each manual capture’s lines against the current tree by git blob SHA: unchanged lines stay valid, changed/deleted lines go stale (coverage revoked — the evidence no longer describes this code), and valid-but-old lines past the tier’s max_age are flagged aging without losing coverage credit.

What is unique about cov

otto cov report runs after the fact, over directories otto test --cov or otto cov get already wrote: it still loads the lab — per-host toolchain resolution (gcov, lcov) comes from host configuration, with the .gcno header’s gcov version stamp as the fallback (a clang stamp routes counters through llvm-cov gcov) — but it creates no output directory of its own and runs no gate: reporting on yesterday’s run must never be blocked by today’s reservations (The command lifecycle). Its siblings do touch the lab: otto cov get fetches counters — into the standard per-invocation output directory, or --output — and otto cov clean zeroes them on the remotes.

otto cov --help

otto cov --help Usage: otto cov [OPTIONS] COMMAND [ARGS]... Generate coverage reports from otto test --cov output. ╭─ Options ────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit. │ ╰──────────────────────────────────────────────────────────────────────────────╯ ╭─ Commands ───────────────────────────────────────────────────────────────────╮ │ report Generate a coverage report from otto test --cov output directories. │ │ get Fetch .gcda coverage from the lab and produce pinned per-board │ │ captures. │ │ clean Zero .gcda counters on the lab's Unix coverage hosts. │ ╰──────────────────────────────────────────────────────────────────────────────╯