Host Database¶

Otto builds its lab — the set of hosts a command can touch — from a host source. By default that source is the hosts.json files under your labs directories, but the source is a pluggable backend: point otto at a CMDB, an inventory API, or any system of record by implementing one small interface.

Otto is strictly a consumer of host data. It reads hosts; it never writes back to your source of record.

The interface¶

A host source implements the LabRepository protocol — two read-only methods:

load_lab(name, preferences=None) -> Lab

Build and return the named lab. Raises LabNotFoundError if the name is unknown.

list_labs() -> list[str]

The lab names this source can provide.

Configuration is supplied at construction time, so a backend is built once and then queried.

Quick start: the built-in JSON source¶

The default backend is "json": it reads hosts.json from each directory in your labs setting. No [lab] block is required — a repo with just labs = [...] already uses it:

name = "my_project"
version = "1.0.0"

labs = ["${sut_dir}/lab_data"]

Writing it out explicitly is equivalent:

[lab]
backend = "json"

The per-host hosts.json schema — every field, and how labs merge — lives in Lab Configuration.

Annotating entries with _-prefixed keys¶

hosts.json is plain JSON, which has no comment syntax. Any key beginning with _ (e.g. _comment) on a host entry is stripped before validation, so it is otto’s sanctioned way to leave a note inline without tripping the schema’s extra="forbid" check:

{
    "_comment": "Replace before connecting to a real host.",
    "ip": "192.0.2.1",
    "element": "example-device",
    "os_type": "unix",
    "valid_terms": ["ssh"],
    "creds": [{ "login": "admin", "password": "CHANGE_ME" }],
    "labs": ["example_lab"]
}

This idiom is scoped to host entries only — it is not a general convention elsewhere in otto’s JSON/TOML configuration.

Credentials and login proxies¶

A host’s creds field is an ordered list of cred entries, each with a required login and four optional fields:

The first entry is the default login — the user otto authenticates as unless user names a different entry:

"creds": [
    {"login": "admin", "password": "hunter2"},
    {"login": "mysql", "proxy": "mysql-su", "via": "admin",
     "params": {"service": "mysqld"}}
]

Here otto logs in as admin by default. Setting "user": "mysql" on the host entry (or calling switch_user("mysql") at runtime) authenticates as admin first, then runs the mysql-su proxy to become mysql.

Validated at load, alongside the usual schema checks: every login is unique; via/params are only allowed alongside proxy; via must name another entry in the same list, never itself; a chain of via links must terminate at a proxy-less entry (a cycle is rejected at load, not discovered mid-connection); and proxy names are checked against the live login-proxy registry the same way term/transfer selectors are checked against theirs — an unregistered name fails loud, listing what’s registered, instead of failing later mid-connection.

Ownership when a login is proxied¶

Every command surface (run, oneshot, named sessions) executes as the proxied user once a session has switched to it — but file transfer is not uniform, because not every transfer protocol rides a shell:

• nc transfers ride pooled, already-proxied shell sessions, so a file it puts lands owned by the target (proxied) user.

• scp / sftp / ftp authenticate at the transport layer directly as the resolved direct (via) cred — they cannot replay proxy steps, since they are not interactive shells — so a file they put lands owned by the via user, not the proxied target.

Pick nc ("transfer": "nc", or include it in valid_transfers) when a proxied host’s file ownership needs to match the target account rather than the account otto authenticated as.

Breaking change: creds was a dict, now a list¶

creds used to be a flat {"login": "password"} mapping; it is now the ordered list described above (feat(host)!). A hosts.json still written in the old dict shape is rejected loudly at load:

ValueError: creds is now a list of cred objects: [{"login": "user", "password": "pw"}, ...]
(was: {user: password}). See the host-database guide.

Update every entry to [{"login": ..., "password": ...}, ...]. The first entry keeps the old “first dict entry is the default login” behavior — now explicit and ordered, rather than relying on dict insertion order.

Selecting a different source¶

[lab] backend selects any registered backend by name. Register your backend from an init module (one of the modules listed in init = [...]), then name it in settings:

# my_lab_source.py  (listed in init = [...])
from otto.storage import register_lab_repository
from my_company.cmdb import CmdbLabRepository

register_lab_repository("cmdb", CmdbLabRepository)

[lab]
backend = "cmdb"

[lab.cmdb]
url = "https://cmdb.example.com"

Otto constructs the backend as CmdbLabRepository(repo_dir=<repo root>, url="https://cmdb.example.com") — the [lab.<name>] sub-table becomes keyword arguments, plus repo_dir for resolving any relative paths. Selecting an unregistered name raises LabRepositoryError, listing the registered names.

Writing a custom backend¶

A backend is any class satisfying the two-method protocol. Otto ships a small, dependency-free reference implementation — otto.examples.lab_repository.ExampleLabRepository — that you can copy from src/otto/examples/lab_repository.py as a starting point. It holds a mapping of lab name to host dicts and builds real hosts with create_host_from_dict so each becomes a RemoteHost keyed by its id — which is what the rest of otto expects.

The shipped sample works out of the box and demonstrates the contract:

>>> from otto.examples.lab_repository import ExampleLabRepository
>>> repo = ExampleLabRepository()
>>> repo.list_labs()
['east', 'west']
>>> lab = repo.load_lab("east")
>>> lab.name
'east'
>>> sorted(lab.hosts)
['router1']

Loading an unknown lab raises the contract’s error — never a bare KeyError or None:

>>> from otto.storage import LabNotFoundError
>>> try:
...     repo.load_lab("does-not-exist")
... except LabNotFoundError:
...     print("not found")
not found

Error contract¶

A backend signals trouble through two exceptions (from otto.storage):

LabNotFoundError

load_lab was asked for a name the backend does not know. Raise this — never return None or raise a bare KeyError.

LabRepositoryError

Any other failure (I/O, network, parse, credentials) that prevents a definitive answer. LabNotFoundError is a subclass, so callers can catch the base.

Verify your backend¶

Otto ships a conformance helper that checks a backend against the full contract and reports every violation at once (it raises a single AssertionError listing each failed rule). The shipped sample conforms:

>>> from otto.testing import assert_lab_repository_conforms
>>> from otto.examples.lab_repository import ExampleLabRepository
>>> assert_lab_repository_conforms(
...     ExampleLabRepository(), expected_labs=["east", "west"]
... )

Call it from your own test suite, passing expected_labs=[...] to also assert specific labs are present and loadable against your known fixtures:

from otto.testing import assert_lab_repository_conforms
from my_lab_source import CmdbLabRepository

def test_cmdb_conforms():
    assert_lab_repository_conforms(CmdbLabRepository(repo_dir="."))

Troubleshooting¶

"Unknown lab repository backend '...'"

[lab] backend names a backend that was never registered. Check the name, and confirm the init module that calls register_lab_repository(...) is listed in init = [...].

LabNotFoundError: Lab '...' not found

The backend has no lab by that name. Check --lab / OTTO_LAB against list_labs().