release: 1.7.0

CI/lint hardening since v1.6.14: clippy on stable with the MSRV single-sourced
from Cargo.toml, cargo-deny dependency-audit gate with first-party git sources
allowed (#15), and audience-based comment-guard (#17). No library code changes.

docs/keydb-format.md

@@ -0,0 +1,157 @@
+# keydb_format.rs — design notes
+
+## Full parser API retained on purpose
+
+The parser is copied verbatim, so it carries the full KeyDb/DiscEntry API
+even though this crate's consumer (`keydb.rs`) only exercises a subset
+(`load`, `find_disc`, `iter_disc_entries`, and the public fields read by
+`KeydbSource::unit_keys_from` / `KeydbSource::host_certs`). NOTE: an earlier
+revision of this comment cited a `candidates_from` function — no such
+function has ever existed in this crate; the real consumer is
+`KeydbSource::unit_keys_from` (keydb.rs), which reads `unit_keys`, `vuk`,
+`media_key` AND `vid` (`vid` is load-bearing for the MK+VID derivation the
+kat_c / kat_d KATs pin — it is NOT unused). The genuinely unused items —
+`empty`, `find_vuk`, `DiscEntry::title` — are part of the faithful copy and
+are retained rather than pruned; allow dead_code so the byte-for-byte copy
+compiles clean without diverging from the libfreemkv original.
+
+## ParseStats — what `KeyDb::parse` threw away
+
+The parser is fed a THIRD-PARTY file: a mirror can serve a truncated or
+byte-shifted download that is still valid UTF-8, still contains enough
+recognisable rows to pass `KeydbSource::save`'s "at least one entry" check,
+and is then atomically committed as the new keydb — after which every disc
+past the corruption point silently resolves nothing. Before this counter,
+each malformed `| DK` / `| PK` / `| HC` / `| HC2` / `0x…` row was dropped by
+an `if let Some(..) = .. { }` with no `else`, no counter and no log, so that
+file was indistinguishable from a healthy one at every layer above.
+
+The counts are logged ONCE per parse (a per-line log would be 181k lines on
+a real keydb, which is why the rate limit is "one summary", never "hide the
+count").
+
+## `KeyDb::load_counted` — same-descriptor stamp + parse
+
+Two callers need this seam, both in `crate::KeydbSource`:
+
+* The identity stamp and the bytes must come from the SAME descriptor.
+  Stat-the-path-then-load-the-path is two independent resolutions of one
+  name, so an atomic rename landing between them stored a stamp
+  describing one file next to the parsed contents of another — and the
+  cache then served that pairing until the stamp changed again. `fstat`
+  on the open handle cannot be pointed at a different inode.
+* The rejection counts have to be RETAINED, not just logged, so a cache
+  hit can re-emit them.
+
+## `to_keydb_cfg` — trailing-comment placement
+
+The trailing `; <comment>` (MKB version / volume size / UHD) is emitted
+ONLY after a `U` (unit-keys) field — that is the one place the parser
+splits the value on `;`. Gluing a comment onto an `M`/`I`/`V` value would
+make `parse_hex16` reject the whole field, so a comment-bearing entry that
+has no unit keys drops its comment (keys always survive; the metadata is a
+derivable hint). Real per-disc rows that carry metadata also carry keys.
+
+## `parse_disc_entry` title handling
+
+Title = everything between `= ` and the first ` | ` field (or the trailing
+`;` comment), kept VERBATIM (trimmed). This is a FAITHFUL copy of the keydb
+title, so it must round-trip exactly: a previous version extracted a `(...)`
+substring as a "display title", but that TRUNCATED real titles that
+legitimately contain parentheses ("Lawrence of Arabia (Restored Version) –
+Disc 2 …" → "Restored Version") and broke serialize→parse idempotence.
+Display prettification, if wanted, belongs in the title-display layer, NOT
+this codec.
+
+## Real-data test notes
+
+`to_keydb_cfg_is_idempotent_on_real_keydb`: parse the full keydb -> serialize
+(S1) -> parse S1 -> serialize again (S2). S1 MUST equal S2 byte-for-byte.
+This is the right invariant: a raw keydb.cfg has formatting variance
+(whitespace, optional fields, comment style) that our CANONICAL serializer
+normalizes, so `text == to_keydb_cfg` is NOT expected — but once normalized,
+a re-load+re-serialize must be stable. Idempotence here proves `parse` is
+lossless on its own output and `to_keydb_cfg` is deterministic. Also asserts
+no rows are dropped. It used to carry no `#[ignore]` and `return` early when
+`KEYDB_PATH` was unset — so CI ran it, asserted NOTHING about 181k-entry
+behaviour, and reported green. A skip has to be visible in the test output
+to mean anything, and an ignored test that is RUN without its fixture must
+fail loudly rather than silently pass.
+
+`real_keydb_path`: the old pattern (`match keydb_path() { Some(p) => p, None
+=> return }`) on a non-ignored test is the worst of both worlds: CI ran the
+test, asserted nothing, and reported success. These tests are `#[ignore]`d,
+so running one at all is an explicit request — and an explicit request with
+no fixture must fail, not quietly pass.
+
+## `load_counted_reads_the_handle_not_the_path` test
+
+`load_counted` must read the HANDLE it was given, never re-resolve the
+path. `KeydbSource::cached_db` stamps the open file with `fstat` and then
+parses from that same descriptor precisely so the recorded identity and
+the parsed bytes cannot describe two different files; the moment this
+function re-opens the path, that guarantee is gone and an atomic rename
+landing in between stores one file's stamp beside another file's keys —
+a mismatch the cache then serves as if it were valid.
+
+The race itself cannot be scheduled deterministically in-process, so the
+SEAM is what is pinned: with the path already replaced, the handle's
+contents must still win. Catches the mutation that restores an internal
+`File::open(path)`.
+
+## `host_cert_debug_is_redacted_including_the_sibling_wrapper` test
+
+The SIBLING type. `KeyDb`'s hand-written `Debug` prints `host_certs_len`, so
+it protects the whole-db rendering and nothing else — `KeyDb::host_certs` is
+a public field, and `{:?}` on ONE element of it walks a completely different
+code path: `KeydbHostCert`'s derive, then `HostCert`'s impl. That path
+carries the AACS host PRIVATE key, the only key material in the crate that
+authenticates US rather than opening a disc.
+
+It is safe today only because `libfreemkv::HostCert` hand-writes a redacting
+`Debug` (`aacs::types`, `private_key: "<redacted>"` + `certificate_len`) —
+an EXTERNAL guarantee this crate silently inherits through the derive, with
+nothing on this side pinning it. This test pins it: it catches a libfreemkv
+upgrade that swaps that impl for `#[derive(Debug)]`, which would leak the
+host private key out of this crate's public API with no diff in this
+repository at all.
+
+Rendered three ways because callers reach the cert three ways: the whole
+`Vec` (`db.host_certs`), one element, and the inner `HostCert` a caller
+gets back from `KeyDb::host_certs(mkb)`.
+
+## KEYDB-parser integration tests relocated from libfreemkv
+
+These exercise the parser (`KeyDb::load`) end-to-end against a real
+keydb.cfg and feed its material into libfreemkv's AACS crypto
+(`derive_vuk`, then `decrypt_unit` + `is_clean_ts`). They live here now
+that the parser lives here. All are `KEYDB_PATH`-env-gated and no-op in CI
+when the env is unset; they must still COMPILE.
+
+## `test_decrypt_real_unit` — no-decrypt outcome
+
+Reaching the fallback path is the EXPECTED outcome for the AACS 2.0 (BEE)
+sample this test was written against: the unit is still bus-encrypted, so
+no unit key alone can open it. That makes "no key worked" an unusable
+assertion on its own — which is exactly why this test used to end in an
+`eprintln!` and fall off the end, i.e. pass unconditionally. A no-op
+`decrypt_unit`, an `is_clean` stuck at `false`, or a parser that yielded
+zero keys would all have produced this same clean exit.
+
+What CAN be asserted regardless of the sample: the pipeline actually ran,
+and the decrypt primitive actually transformed the bytes.
+
+## Hand-written redacting `Debug`
+
+Key material must never reach a log, a panic message, or a bug report. Both
+`KeyDb` and `DiscEntry` carry raw AACS key bytes, and both are public API, so
+a `#[derive(Debug)]` on either is a leak of the crown jewels: `{:?}` on a
+loaded `KeyDb` printed every processing key plus every disc's Media Key,
+VID, VUK and Unit Keys. libfreemkv's equivalent types (`aacs::types`:
+`DeviceKey`, `HostCert`, `Vid`, `MediaKey`, `Vuk`, `ProcessingKey`,
+`UnitKey`, its own `DiscEntry`) all hand-write a redacting `Debug` for
+exactly this reason; this crate's copy of the parser dropped that
+convention when it was relocated. These restore it, field-for-field in the
+same style (`<redacted>` for key bytes, a `_len` for key-bearing
+collections). Pinned by `keydb_debug_is_redacted` /
+`disc_entry_debug_is_redacted`.

docs/keydb.md

@@ -0,0 +1,432 @@
+# `keydb.rs` design notes
+
+Long-form rationale moved out of `src/keydb.rs` doc comments to satisfy the
+repo's comment-guard line caps. Each section is pointed to from the file by a
+one-line `// See docs/keydb.md#<anchor>` comment.
+
+## candidate-order
+
+Full derivation path this source walks, cheapest-first:
+
+1. per-disc **Unit Keys** (hash hit) → returned terminal, no derivation.
+2. per-disc **VUK** (hash hit) → `uks_from_vuk` over the disc's encrypted
+   title keys.
+3. a **Media Key**, then `derive_vuk` → `uks_from_vuk`. The MK comes from, in
+   order: the disc's stored MK (hash hit); the keydb's **Processing Key**
+   pool walked against THIS disc's MKB via `derive_media_key_from_pk`; or the
+   device-key pool via `derive_media_key_from_dk`. The PK and DK pools
+   resolve the Media Key WITHOUT a VID; the final `derive_vuk` still needs
+   one. The VID is the unlocker's physical VID (`ResolveCtx::vid`) when
+   present, else the keydb entry's OWN stored VID (the `I` field, `vid`) for
+   the non-physical / ISO path. With no VID from either source the MK path
+   cannot complete — return nothing.
+
+The cross-disc MK-pool brute (trying OTHER discs' stored media keys against
+this disc) stays RETIRED: every MK path here is anchored to the matched
+disc's own MKB or stored material.
+
+The library still OWNS the crypto; this source owns only which primitive to
+call with which material. Returning an empty `Vec` is a genuine "no key for
+this disc here".
+
+## file-identity
+
+`(len, modified)` ALONE IS NOT AN IDENTITY. mtime is stored at whole-second
+resolution on HFS+, on many container overlays and on most network
+filesystems, so a writer that replaces `keydb.cfg` within the same second
+with a file of the same byte length leaves `(len, modified)` bit-identical —
+and the cache then serves the superseded parse for the rest of the process's
+life. For a key database that is not a stale page: it is serving retired or
+since-corrected keys while reporting success. The writers that hit this are
+real and named in this crate's own docs: the daily-refresh job, a second
+freemkv process, an operator's editor, a sync tool.
+
+Two independent discriminators close it:
+
+1. `dev` + `ino`. Every writer that publishes ATOMICALLY (temp file, fsync,
+   rename — what `KeydbSource::save` does, and what any correct publisher
+   does) creates a NEW inode, so the stamp differs immediately and
+   unconditionally, whatever the clock or the mtime resolution did. Zero on
+   platforms without them (see `FileStamp::of`), which costs nothing: rule 2
+   stands alone.
+2. A freshness rule on the cache entry — `CacheEntry::is_settled` — which
+   covers the residual in-place rewrite that reuses the inode.
+
+What this still does NOT catch: a writer that FORGES the timestamp, i.e.
+restores an mtime equal to the cached one (`touch -t`, `rsync --times`, a
+tar/backup restore) into the SAME inode with the SAME length. Nothing short
+of hashing the content can see that, and hashing 62 MiB per lookup is
+precisely the cost the cache exists to remove (~141 ms × ≥3 loads per rip).
+`KeydbSource::save` therefore keeps invalidating the cache directly, so the
+one write path this crate owns never depends on any of the above.
+
+## settle-proof
+
+Whether a cache entry's stamp can be TRUSTED to change if the file changes.
+
+The proof, for a filesystem storing mtime at granularity `G`: suppose this
+entry was stamped at `stamped_at` with `stamped_at - modified >= G`, and some
+writer later rewrites the file in place at wall time `T > stamped_at`. The
+recorded mtime `m'` of that write is within one granule of `T`, so
+`m' >= T - G > stamped_at - G >= modified`. Hence `m' != modified` and the
+stamp DOES change. The ambiguous window is therefore exactly the one where
+the cached entry was stamped less than `G` after the file's own mtime — i.e.
+we looked at the file while it was still "hot" — and in that window the
+entry is refused and the file is re-read. Once the file has been quiet for
+`G`, its entry settles and the cache is fully effective again.
+
+Cost: after every keydb write, lookups re-parse for up to `G` (2 s). The
+published keydb is rewritten daily, so that is ~2 s of re-parsing per day
+against a cache that saves ~420 ms per rip.
+
+`modified: None` (a filesystem that reports no mtime) never settles: the only
+two discriminators left would be size and inode, so the entry is re-read
+every time rather than being trusted on a weaker basis.
+
+CLOCK CAVEAT: `stamped_at` is the local clock and `modified` is the
+filesystem's. On a network share whose server clock runs AHEAD of the
+client's by more than `G`, entries never settle (safe: more re-parsing). A
+server clock BEHIND the client's makes an entry settle early, which re-opens
+the same-second window it closes — the inode rule (1) is what covers the
+realistic writers there.
+
+## keydbsource-cache
+
+`KeydbSource` caches the parsed database behind the file's identity stamp.
+The published keydb is ~62 MiB / ~181k entries and a single AACS-cert rip
+parses it repeatedly — `host_certs()` for the drive handshake (freemkv's
+`pipe.rs`/`engine.rs`, autorip's `keysource.rs` each build their own source),
+then the trait `host_certs(mkb)` during resolution, then `get_unit_keys` —
+and every one of those calls used to re-read all 62 MiB from disk and
+re-parse it from scratch, because the struct held nothing but a `PathBuf`.
+
+A keydb replaced underneath this source is picked up on the next call, with
+ONE stated exception: a writer that rewrites the file in place, into the
+same inode, at the same byte length, having restored the previous mtime
+exactly (`touch -t` / `rsync --times` / a backup restore). Everything else —
+any atomic rename, any length change, any real clock advance — changes the
+stamp. `FileStamp` and `CacheEntry::is_settled` carry the full argument and
+the clock caveats; do not weaken either without re-reading them. (The
+earlier wording here — "an externally replaced keydb is still picked up" —
+was simply false for a same-second, same-length replacement, which is the
+common shape of a daily-refresh job on a whole-second-mtime filesystem.)
+
+MEMORY RESIDENCY (accepted, and deliberately recorded rather than fixed
+here): a hit keeps the entire parsed keydb — every disc's Media Key / VID /
+VUK / Unit Keys, the processing- and device-key pools, and the host-cert
+PRIVATE keys — resident for the process's life, where the pre-cache code
+held one parse transiently. Nothing in this crate zeroizes on drop, so that
+material was always exposed to a core dump or a swapped page; the cache
+widens the WINDOW, it does not create the class. Closing it properly means
+zeroize-on-drop across `KeyDb`, `DiscEntry` and libfreemkv's `aacs::types`
+(which owns most of the buffers) — a cross-crate change, tracked separately,
+NOT smuggled in behind a cache fix.
+
+## cached-db-one-open
+
+Errors are the same ones `KeyDb::load` produces, unchanged: a missing file
+is `NotFound` (the documented benign case), an over-cap or non-UTF-8 file is
+`InvalidData`. The stamp costs an `fstat`, not a 62 MiB read.
+
+ONE OPEN, ONE IDENTITY. The file is opened once and both the stamp (`fstat`
+on that handle) and, on a miss, the bytes come from it. The previous shape —
+`std::fs::metadata(path)` and then a separate `KeyDb::load(path)` — resolved
+the path TWICE with no atomicity between them, so a rename landing in the
+gap cached the new file's stamp beside the old file's contents (or the
+reverse); the pairing was then served, looking perfectly valid, until the
+stamp changed again. The stamp is read BEFORE the bytes on purpose: a write
+racing the read then leaves the OLD stamp cached, which the next call
+detects, whereas stamping afterwards would file torn content under a
+fresh-looking identity and pin it.
+
+## cached-db-reemit
+
+Re-emit the parser's rejection summary on EVERY hit, not just on the parse
+that produced it. `KeyDb::parse` is what logs it, and a hit skips `parse`
+entirely — so a daemon that cached a corrupt keydb logged its rejected-row
+counts exactly once, at whatever moment it first read the file, and then
+served that same damaged database to every later rip in silence. Repetition
+is the correct failure mode here: the line is emitted only when the file
+really does have rejected or duplicate rows, and a warning an operator can
+still see on the tenth rip is worth more than one they had to catch on the
+first.
+
+## load-failure
+
+A MISSING keydb is the documented benign case: the app may simply not have
+one, another source may hold the key, and "no keydb" is genuinely "no key
+here". Everything else is NOT that: `InvalidData` is a keydb over the
+128 MiB cap or one that is not valid UTF-8 — i.e. corrupt, truncated, or a
+half-finished download — and a permission/IO failure means the file could
+not be read at all. Reporting either as an empty result made a broken keydb
+indistinguishable from "this disc has no key" (the same conflation as the
+seven-hour 502), with no log to notice it by. Both are now logged AND
+surfaced as errors so the resolver reports a source failure instead of
+`E7022`.
+
+## save-mirror-parse
+
+Mirror `KeyDb::parse`'s disc-entry rule EXACTLY by CALLING it
+(`is_disc_entry_line`), so `save()` never validates + persists content that
+parses to zero usable entries (e.g. a stray "0xDEADBEEF" comment line) and
+the two can no longer drift — they did drift once already, on the `0X` case
+rule.
+
+## unit-keys-from
+
+CPS-unit numbering: a returned `UnitKey::idx` is the POSITIONAL index
+libfreemkv's `resolve_and_apply` turns into the canonical CPS-unit number
+`idx + 1`. For the terminal per-disc unit-key path we therefore map the
+keydb's stored CPS number `num` to `idx = num - 1`, so the committed number
+is byte-identical to the keydb's `num` (and to what the OLD
+`Key::Unit(entry.unit_keys)` path committed). For the VUK / MK paths the
+boil primitive already yields 0-based positional indices, matching
+`parse_unit_key_ro`'s `(i + 1)` after the resolver's `+ 1`.
+
+## unit-keys-per-disc-hit
+
+Per-disc hit (most specific). `find_disc` normalizes the hash form. Without
+a matched entry this keydb has no per-disc material (Unit Keys / VUK / Media
+Key / stored VID) to anchor a derivation for the disc, so it resolves
+nothing. (The PK and DK pools are global, but the cross-disc MK-pool brute —
+trying OTHER discs' media keys against this disc — stays retired; a PK/DK
+pool only ever resolves a disc reached through its own matched entry below.)
+
+## unit-keys-union
+
+UNION every source of terminal keys, then dedup — never first-hit. A stored
+`unit_keys` list can be PARTIAL (the key-import tool only ever sampled the
+CPS units reachable from a playlist, so an orphan unit's key may be
+missing), while the per-disc VUK boils EVERY declared CPS unit. Taking the
+stored list alone (the old return-at-first-path) would shadow the VUK and
+silently drop the orphan unit's key. So gather both and keep a unique-by-key
+list: the read path tries every key per unit, so an extra or stale key is
+harmless — only a MISSING key hurts.
+
+## unit-keys-vuk-or-mk
+
+2. Per-disc VUK — one step, no VID needed; boils ALL declared units.
+3. Else a Media Key path (stored MK / PK pool / DK pool) → VUK → all
+   declared units. The MK itself carries no VID, but the final `vuk_from_mk`
+   needs one: physical (unlocker) VID first, else the entry's stored VID
+   (`I` field), else cannot derive. Either branch yields the COMPLETE
+   declared set, so we take the first that resolves (VUK preferred —
+   cheapest).
+
+## write-atomic
+
+keydb.cfg is the single source of AACS truth, and save/update run
+unattended (first-boot download + daily-refresh thread, with a container
+restart on every release). A bare in-place `fs::write` truncates the file
+before writing, so a SIGKILL (docker stop's grace window), OOM-kill, power
+loss, or ENOSPC mid-write would leave the keydb half-written — the prior
+good copy already gone. A truncated keydb doesn't error at write time; it
+silently breaks key resolution on every later AACS rip. Writing to a temp
+file then renaming (POSIX rename is atomic within a filesystem) means an
+interrupted update leaves the previous keydb fully intact.
+
+The fsync MUST succeed before the rename: a `sync_all` failure (ENOSPC,
+ESTALE on the bind-mounted volume) means the kernel never guaranteed the
+bytes reached stable storage, so publishing them via rename would defeat
+crash-safety. The temp name is unique per call (pid + monotonic counter) so
+a concurrent update can't share a fixed temp path and rename a mangled file
+over the keydb.
+
+## get-unit-keys-trait
+
+A MISSING keydb is not an error — it simply yields no keys (another source
+may have them), the same as the library's own loader. A keydb that exists
+but cannot be USED (over the size cap, not UTF-8, unreadable) is a source
+failure and returns `Err`: it says nothing about whether this disc has a
+key, and reporting it as an empty result is the same looks-like-success
+failure as a 502 reported as `E7022`.
+
+The keydb carries no AACS 2.1 forensic index keys today, so it does not
+override `get_fmts_indexes` — the default (empty) opts it out, and an FMTS
+disc's forensic set comes from the online source.
+
+## host-certs-trait
+
+Expose the keydb's host certs through the trait — the OEM/AACS cert-auth
+route collects them across every source via this method. Wires the disc's
+MKB generation through for revocation filtering (the keydb parser's
+`; Revoked in MKBv<N>` annotation): a cert revoked at generation `R` is
+withheld once the disc's generation reaches `R`.
+
+## test-pinned-mtime
+
+Force a file's mtime to `PINNED_MTIME`, so two different files can be made
+bit-identical to a `(len, mtime)` stamp on purpose. The cache tests that
+matter all depend on FORCING the indistinguishable case rather than hoping
+the filesystem produces it: on APFS (nanosecond mtimes) a same-second
+replacement is otherwise detected by mtime alone, and a test that relies on
+that passes with the fix reverted — which is worth nothing.
+
+## test-committed
+
+The committed `(cps, key)` pairs libfreemkv's `resolve_and_apply` derives
+from a source's Unit Keys: positional `idx` → canonical CPS number
+`idx + 1`. The KATs compare against THIS to prove byte-identical parity with
+the OLD `Key::Unit` / resolver-derived commit.
+
+## test-mock-ctx-stub
+
+`MockCtx` implements the full `ResolveCtx` trait so it can stand in for a
+real disc, but `KeydbSource` itself never calls `title()`/`samples()` (those
+matter to the online source, not the local keydb). Exercise them directly
+so the mock's trait surface is proven correct too.
+
+## test-kat-a
+
+Here `enc_title_keys` is empty, so the VUK can't derive anything — only the
+stored list contributes, and it commits byte-identically to the stored
+`(cps, key)` pairs.
+
+## test-orphan-union
+
+Orphan-unit completeness (the real keydb bug): an entry stores only `uk1`
+(the key-import tool sampled one reachable CPS unit) but ALSO carries the
+VUK, which boils BOTH declared units. The old return-at-first-path handed
+back just `[uk1]`, shadowing the VUK and silently dropping the orphan unit.
+The union must return BOTH — the stored uk1 AND the VUK-derived second unit.
+
+## test-kat-c
+
+A hash hit with a Media Key and a physical VID (from the unlocker) derives
+`MK → VUK → UK`. The PHYSICAL VID must be used in preference to the keydb's
+stored VID — proven by giving the entry a DIFFERENT stored VID and showing
+the result tracks the physical one.
+
+## test-kat-f
+
+Owner decision #1 (AACS): a keydb Processing Key must be walked against the
+matched disc's own MKB to recover the Media Key, then driven down the full
+chain `PK → MK → VUK → UK`. The disc entry carries NO stored MK/VUK/UK — the
+only key material is a global `PK` row — and the result must be the real
+Unit Keys, byte-identical to deriving from the recovered MK.
+
+The MKB + PK use a known-answer construction (a planted PK whose derived MK
+satisfies the synthetic verify record); the constants are precomputed AES
+vectors so this crate needs no AES primitive of its own. They mirror
+libfreemkv's `boil::mk_from_pk_drives_full_chain_to_uks` KAT.
+
+## test-repeated-lookups
+
+A single AACS-cert rip drives this source at least three times — the
+scan-options builder's inherent `host_certs()`, the trait `host_certs(mkb)`
+during the handshake, then `get_unit_keys` — and each call used to re-read
+and re-parse the WHOLE file, because `KeydbSource` held nothing but a
+`PathBuf`. Measured on the real 62 MiB / 184,860-entry keydb: ~140 ms per
+parse in `--release`, i.e. ~420 ms and ~186 MiB of file reads per rip,
+against ~1 µs for a cache hit.
+
+Catches the mutation that drops the cache (every call re-parses: count 3)
+and the mutation that makes the cache unconditional (see the invalidation
+tests below).
+
+The mtime is pinned into the PAST first, because a cache entry is only
+trusted once the file has been quiet for the mtime granularity
+(`CacheEntry::is_settled`) — a keydb written microseconds ago is
+deliberately re-read. Unix-only: pinning an mtime needs `touch -t` (no
+`filetime` dependency for a test).
+
+## test-changed-keydb
+
+The cache must not go stale: a keydb replaced UNDERNEATH the source (the
+daily-refresh thread writing through a different `KeydbSource`, or an
+operator dropping in a new file) has a different size/mtime stamp and must
+be re-read. Catches the mutation that caches on first load and never
+re-stats — which would pin a rip to a keydb deleted hours earlier.
+
+## test-two-discriminators
+
+The defect both of the inode/settle tests below pin: `(len, mtime)` is not a
+file identity. An external writer — the daily-refresh job, a second freemkv
+process, an editor, a sync tool — that replaces `keydb.cfg` inside one mtime
+granule with a file of the SAME length leaves that pair bit-identical, and
+the source then serves the superseded parse for the rest of the process's
+life: retired or since-corrected keys, reported as success.
+
+Each test disables the OTHER discriminator through `mtime_granularity`, so
+neither can pass on the strength of the one it is not testing, and neither
+depends on the host filesystem's real mtime resolution (on APFS the naive
+scenario is not reproducible at all).
+
+## test-inode-rename
+
+A keydb replaced by ATOMIC RENAME — how every correct publisher, including
+this crate's own `save()`, installs a new file — must be re-read even when
+length and mtime are identical. `mtime_granularity: 0` makes every entry
+settle, so ONLY dev+ino can distinguish the two files here.
+
+Catches the mutation that drops `dev`/`ino` from `FileStamp` (the original
+`(len, modified)` stamp): the source then keeps answering with the previous
+keydb's keys.
+
+## test-inplace-rewrite
+
+A keydb rewritten IN PLACE — same inode, same length, mtime forced back to
+the same value — must still be re-read while the cached entry is too young
+to trust. `mtime_granularity` is set to a decade so nothing ever settles,
+which is the same condition a real same-second, whole-second-mtime
+replacement creates and the only way to reproduce it deterministically on a
+nanosecond-mtime filesystem.
+
+Catches the mutation that deletes the `is_settled` check from `cached_db`
+(every stamp match trusted unconditionally, which is what the original code
+did): dev, ino, len and mtime are ALL identical here, so without the
+freshness rule the source serves the old keys forever.
+
+## test-corrupt-warns
+
+`ParseStats::log` runs inside `KeyDb::parse`, and a cache hit skips `parse`
+— so once the cache landed, the rejected-row summary for a given file was
+emitted exactly ONCE for the entire life of the process, while that same
+damaged file went on being served to every subsequent rip. A long-running
+daemon (autorip) logs it at startup, the operator misses it in the startup
+noise or loses it to log rotation, and there is no second chance for as long
+as the process runs.
+
+Catches the mutation that deletes the `emit_parse_stats` call from the
+cache-HIT arm of `cached_db`: warnings would drop to 1 while the file is
+still served three times.
+
+## test-healthy-silent
+
+Catches the mutation that makes `emit_parse_stats` unconditional (drops
+`ParseStats::log`'s empty-counts early return).
+
+## test-fresh-not-trusted
+
+Catches the mutation that widens the window to something unbounded (a
+granularity of hours would re-parse 62 MiB on every lookup all day) by
+pairing with `repeated_lookups_parse_the_keydb_once`, which proves the entry
+DOES settle once the mtime is in the past.
+
+## test-save-invalidates
+
+`save()` replaces the very file this source reads, so it drops the cached
+parse ITSELF rather than trusting the (size, mtime) stamp: a same-size
+rewrite that lands inside the filesystem's timestamp granularity is the one
+change the stamp cannot see — and it is exactly the change this crate
+performs (the daily-refresh thread re-saving a keydb through the source it
+then reads from).
+
+The test forces that indistinguishable case rather than hoping for it: the
+file is written, stamped to a FIXED mtime, cached, re-saved with different
+keys of the SAME length, and stamped back to the same mtime. To the stamp
+the two files are identical, so only the explicit invalidation can produce
+the new key. Unix-only because pinning an mtime needs `touch -t` (no
+`filetime` dependency for one test).
+
+## test-corrupt-is-error
+
+`KeyDb::load` fails distinguishably for a keydb that is over the 128 MiB cap
+or not valid UTF-8 — a truncated download, a half-written file, a binary
+blob dropped in by mistake. Both used to collapse into `Ok(Vec::new())` with
+ZERO tracing, i.e. into the benign "no key for this disc" the resolver
+reports as `E7022` — the same looks-like-success failure as the seven-hour
+502. Only a MISSING file is genuinely benign.
+
+Catches the mutation that restores `Err(_) => Ok(Vec::new())`.
+</content>

docs/lib-scope.md

@@ -0,0 +1,49 @@
+# freemkv-keysources: scope and design notes
+
+## Mechanism vs. policy
+
+Applications (autorip, the `freemkv` CLI) choose and order the key sources
+from their own config — the local-vs-online policy is just which impls they
+plug in — then resolve and hand the resulting key to `Disc::decrypt_with`.
+
+Each source resolves a disc's terminal **Unit Keys** in one shot via
+`KeySource::get_unit_keys`, driving libfreemkv's boil-down crypto primitives
+for whatever level of material it holds. Compose several with `MultiSource`
+in the caller's chosen order.
+
+Reading the encrypted content-sample units a key server validates on, and
+applying the resolved keys against a disc, is decryption *mechanism* — it
+lives in the library (`libfreemkv::resolve_and_apply`,
+`libfreemkv::read_encrypted_units`), not here.
+
+## `MultiSource` failure semantics
+
+`MultiSource::get_unit_keys` tries each inner source in order and returns the
+first non-empty Unit Key set (and `get_fmts_indexes` does the same for the
+forensic set). The caller supplies the list AND the order — local-first
+`[Keydb, Online]`, online-first `[Online, Keydb]`, etc. — so the "which
+sources, in what order" policy lives entirely with the application, not the
+library. `MultiSource` is itself a `KeySource`, so it nests and composes.
+
+**A source that could not ANSWER is not a source that answered "no key".**
+When no inner source produces keys, the composition returns `Err` if any
+inner source failed, and `Ok(empty)` only when every source genuinely
+answered and none held a key. This mirrors libfreemkv's own resolver
+(`keysource::drive_unit_keys`, which tracks an `errored` flag, and
+`source_failure`, which stamps the first failure onto the disc) — a
+composition that swallowed the `Err` would hand the resolver a clean
+`Ok(empty)`, and an outage would once again be reported to the operator as
+`E7022 No key source has a decryption key for this disc`.
+
+## `first_non_empty` ordering rule
+
+A source failure never blocks the chain (a later source is still tried, and
+a later success still wins outright, error or no error), but it is not
+forgotten either: with no keys anywhere, the FIRST failure is returned.
+First (not last) matches libfreemkv's `source_failure` rule, so the
+most-preferred source's reason is the one the operator is told about, in the
+same order the caller expressed a preference for successes.
+
+`get` selects which trait method to drive, so the base and forensic paths
+share ONE implementation of this rule and cannot drift apart — the forensic
+half carried the identical bug.

docs/multi-source-failure-collapse.md

@@ -0,0 +1,13 @@
+# `MultiSource`: a source that could not answer is not a source that said no
+
+`MultiSource` used to re-collapse the Ok/Err distinction `OnlineSource` was
+engineered to draw: `if let Ok(uks) = s.get_unit_keys(ctx)` discarded every
+`Err`, so a composition whose sources ALL failed returned `Ok(Vec::new())` —
+which the resolver reports as `E7022 No key source has a decryption key for
+this disc`. That is the seven-hour-502 incident reproduced one layer up.
+
+The regression test `multi_source_reports_a_source_failure_instead_of_a_clean_no_key`
+is THE regression check, one layer up from `interpret_reply`: when no source
+holds a key AND a source could not answer, the composition must report the
+FAILURE, never a clean "no key". It catches the mutation that restores
+`if let Ok(uks) = ..` (which drops the `Err` and returns `Ok(empty)`).

docs/online-agent-cache.md

@@ -0,0 +1,39 @@
+# `src/online.rs` — `OnlineSource::agent` caching
+
+## Why cache at all
+
+`query` used to build a FRESH `ureq::Agent` every call, and an FMTS disc
+makes two calls per rip (`get_unit_keys` + `get_fmts_indexes`), so it paid
+two TLS handshakes to the same host for no reason — an agent owns the
+connection pool, and a discarded agent discards the pooled, already
+negotiated TLS connection with it.
+
+## Security invariant
+
+The agent is reused ONLY when the freshly resolved + guarded address SET is
+identical to the one the cached agent is pinned to. Every query still
+re-resolves and re-runs the SSRF guard — that is the whole anti-rebinding
+defence and is deliberately not cached; what is reused is the connection,
+and only to addresses just re-validated this call.
+
+## Why the cache key is a sorted, deduped SET, not an ordered sequence
+
+`to_socket_addrs` hands back whatever order the resolver felt like, and a
+round-robin keyserver reorders its A/AAAA records on essentially every
+lookup — an ordered `==` therefore missed the cache every single time
+against exactly the deployment that most needs it, silently degrading to
+the per-query TLS handshake this field exists to avoid.
+
+Set equality is also the honest predicate: the security property being
+asserted is "these are the same validated addresses", which says nothing
+about their order. The agent itself stays pinned to the order it was BUILT
+with; that is safe because an equal set means every address it holds was
+re-resolved and re-guarded this call.
+
+## `a_reordered_but_identical_address_set_reuses_the_agent`
+
+Catches the mutation that drops the sort/dedup from `agent_for`'s key (the
+reversed order would then build a second agent) and, via the second half of
+the test, the mutation that "fixes" order-sensitivity by comparing only
+lengths or the first element — which would wrongly reuse an agent pinned to
+a DIFFERENT address.

docs/online-guardfail.md

@@ -0,0 +1,41 @@
+# `src/online.rs` — `GuardFail` classification details
+
+## Malformed-port authority
+
+A `host:port` authority whose port is not a u16 is a TYPO in the operator's
+config — `https://example.com:notaport/keys`. This used to fall back to the
+WHOLE authority as the hostname (port text included), which of course never
+resolves, so the URL surfaced as `GuardFail::Unreachable` →
+`Err(KeyServiceUnavailable)`: a standing misconfiguration reported as a
+transient outage, the exact collapse `GuardFail`'s own doc says must not
+happen, because the two halves demand OPPOSITE operator actions (fix the URL
+vs. wait).
+
+Reject it as `Config`, identical to the bracketed-IPv6 branch a few lines
+above, which already returns `Config` for an unparseable port. Silently
+substituting `default_port` would be worse still — it would ship the bearer
+token and key material to a port the operator never configured.
+
+## Guard-blocked address (`Config` arm, `resolve_and_guard`)
+
+Log THAT the URL was rejected, never WHY: the guard's message names the
+resolved address, and an internal address must not reach a log an operator
+may paste into a bug report. The static label is enough to separate
+"misconfigured/blocked key-service URL" from "this disc has no key", which
+is the only distinction the operator needs here.
+
+`error!`, not `warn!`, and `Err`, not `Ok(empty)`. This arm used to return
+`Ok(Vec::new())`, which made the PERMANENT fault quieter than the transient
+one directly above it: the service was never asked, yet the composition was
+handed a clean empty and — with no other source holding the key — told the
+operator `E7022 no key for this disc`. Same lie as the seven-hour 502, from
+a config typo that never self-heals. A failure to ask is never evidence
+about the disc, so it leaves as `Err`; `MultiSource::first_non_empty` still
+lets a later source's real key win outright, so this cannot fail a rip that
+the local keydb could have served.
+
+NOTE (reported upstream): libfreemkv has no `KeyServiceMisconfigured` code,
+so the permanent fault borrows the transient E7028. E7028's contract — "the
+source never got as far as answering the question" — is exactly true here;
+only its "transient, retry later" hint is wrong, and the `error!` log
+carries the correcting detail until a 70xx config code exists.

docs/online-preflight-guard-tests.md

@@ -0,0 +1,20 @@
+# `src/online.rs` tests — pre-flight guards in `query`
+
+Each of the three guards (cleartext scheme, over-cap MKB, under-sampled)
+fires BEFORE any address is resolved and before anything is sent. None was
+exercised before these tests: a reordering that let the cleartext POST
+through would have shipped green, and that POST carries the bearer token
+plus base64 key material.
+
+Their VERDICTS differ on purpose (see docs/online-query-guards.md): the
+cleartext-scheme guard is an operator fault and yields `Err`, while the two
+input-shaped guards yield `Ok(empty)` — this source has nothing for THIS
+disc, permanently, and calling that an outage would send the operator
+waiting for nothing.
+
+The discriminator in all three: the configured host is `.test`, which RFC
+6761 guarantees never resolves. On the guarded path nothing resolves it, so
+the test touches no network and returns `Ok(empty)`. Remove the guard and
+control reaches `resolve_and_guard`, which fails as `Unreachable` ->
+`Err(KeyServiceUnavailable)` — a different Result, so the mutation cannot
+pass.

docs/online-query-contract.md

@@ -0,0 +1,19 @@
+# `src/online.rs` — `OnlineSource::query` return contract
+
+The return type draws THE distinction this source exists to draw:
+
+* `Ok(non-empty)` — the service answered with a key.
+* `Ok(empty)` — the service ANSWERED and holds no key for this disc, or this
+  source had nothing to ask with (no service configured, a misconfigured
+  URL, an over-cap MKB, too few samples). A genuine miss; the resolver moves
+  to the next source and `E7022` is the right verdict.
+* `Err(..)` — the service could not answer: unreachable, timed out, DNS
+  failed, returned 5xx, rejected the token (401/403), rate-limited (429), or
+  replied with something unreadable. NOTHING is known about whether a key
+  exists. Collapsing this into `Ok(empty)` is the bug this signature fixes:
+  a seven-hour run of HTTP 502s was reported to operators as
+  `E7022 No key source has a decryption key for this disc`, and they went
+  hunting for a VUK when the correct action was to wait.
+
+`&self`: one-shot is the resolver's contract (each source's
+`get_unit_keys` is called once), so no per-call latch is needed.

docs/online-query-guards.md

@@ -0,0 +1,57 @@
+# `src/online.rs` — `OnlineSource::query` pre-flight guards
+
+## Refusing cleartext (`http://`)
+
+The request body always carries base64 AACS key material
+(inf/mkb/units/vid) and, when configured, a bearer token in a header — so an
+`http://` POST hands both to any on-path observer, and the token is
+replayable.
+
+There is no legitimate plaintext deployment to preserve here: the address
+guard rejects loopback, RFC1918, ULA and link-local, so an `http://` URL can
+only ever reach the PUBLIC internet — the single worst case for cleartext. A
+self-hosted keyserver on a LAN is already impossible by that guard, whatever
+its scheme.
+
+Still falls THROUGH to the next key source (a local keydb) — an `Err` here
+does not block the chain, because `MultiSource::first_non_empty` returns any
+later source's non-empty key set outright and only surfaces the failure
+when nothing anywhere resolved. What it does stop is the silent case: a
+cleartext URL is the same permanent operator fault as the address-guard
+rejection (see `GuardFail`), so it gets the same verdict. Returning
+`Ok(empty)` here made "we refused to ask" indistinguishable from "the
+service answered and has no key".
+
+## The three-way skip/fault/miss line (MKB cap, sample count)
+
+Written out after the `Config`-was-quieter-than-`Unreachable` bug, in three
+parts:
+
+* NO URL AT ALL is not a fault: the operator did not configure an online
+  source, so there is nothing to report. `Ok(empty)`, silently (the first
+  guard in `query`).
+* A bad URL — wrong scheme, mistyped port, guard-blocked address — is an
+  OPERATOR FAULT. Every disc is affected, nothing was asked, and the fix is
+  to edit the config. That is `Err`.
+* An over-cap MKB or too few content samples is a property of THIS DISC's
+  inputs, not a fault: the request cannot be *formed* for this disc, so this
+  source has nothing for it and never will, whatever the operator does and
+  however long they wait. Reporting that as `KeyServiceUnavailable` would
+  send the operator waiting out an outage that does not exist — the same
+  mislabelling in the opposite direction. These stay `Ok(empty)`, and stay
+  logged.
+
+Logging on the MKB-cap skip matters: a silent empty return there is
+indistinguishable from "no key", so the real cause is surfaced (the cap is
+64 MiB, far above any real trimmed MKB).
+
+## `cleartext_http_url_is_refused_before_anything_is_sent`
+
+Catches two mutations: sending the request anyway (the host is `.test` and
+would surface as `Unreachable`, but the assertion would still hold, which is
+why `too_few_samples_skips_the_request` guards the send-path ordering
+separately) and — the one this test was rewritten for — returning
+`Ok(Vec::new())`, which made a permanent cleartext misconfiguration
+indistinguishable from a genuine miss. A later source's real key still wins
+(`MultiSource::first_non_empty`), so refusing does not fail a rip the local
+keydb can serve.

docs/online-ssrf-guard.md

@@ -0,0 +1,44 @@
+# `src/online.rs` — SSRF guard
+
+## Threat model
+
+The keyserver URL is operator-supplied and the bearer token is confidential.
+After `validate_keyserver_url` checked the host at config time, an attacker
+who controls the keyserver's DNS can rebind it to 169.254.169.254 (cloud
+metadata) or an RFC1918 host in the window between validation and the actual
+POST, exfiltrating the key material and the Authorization token.
+
+## Defence
+
+Resolve the host once just before the POST, reject any blocked IP, and pin
+the ureq connection to those validated addresses so a subsequent DNS flip
+cannot redirect the request. Use `redirects(0)` so a public URL can't
+30x-redirect to an internal host.
+
+## Resolver wiring pitfall (`PinnedResolver`)
+
+ureq 3 replaced v2's resolver closure with the `Resolver` trait. The agent
+MUST be built through `Agent::with_parts` to take a custom one:
+`Agent::new_with_config` silently keeps the default resolver, which would
+send the request to live DNS and quietly reopen the rebinding window this
+module exists to close. That failure is invisible — it has no symptom short
+of an actual attack — so it is pinned by the test
+`hardened_agent_connects_to_the_pinned_address_not_dns`.
+
+## Why that test is structured the way it is
+
+`resolve_and_guard` VALIDATES addresses; `hardened_agent` PINS them, so a
+hostname cannot re-resolve to an internal address between validation and
+connection. The validation half is well covered by other tests — but every
+one of those still passes if the pinned resolver is never consulted, because
+none of them makes a connection. Nothing else in the file proves the agent
+honours the pin, and a mis-wired resolver fails OPEN: the connection quietly
+falls back to live DNS and the rebinding window this module exists to close
+is reopened, with no visible symptom.
+
+The discriminator: pin the agent to a loopback listener the test owns, then
+ask it for a host that CANNOT resolve — `.test` is reserved by RFC 6761 and
+never resolves in the public DNS. Only a consulted resolver can turn that
+name into a connection; a fallback to live DNS fails instead. No network is
+touched, and `query` (whose guard blocks loopback by design) is not
+involved — this drives `hardened_agent` directly.

docs/paths.md

@@ -0,0 +1,13 @@
+# `keydb.cfg` path policy (`src/paths.rs`)
+
+Key-path policy belongs with the key sources (this crate), not the library:
+libfreemkv is handed a path and reads it. The CLI/app asks here for the
+list of locations to *search* (first existing wins) and for the single
+*default* location to write to (e.g. `update-keys`/save).
+
+freemkv is a portable, standalone binary: the `keydb.cfg` lives *next to*
+the executable — `<dir of current exe>/keydb.cfg` — and nowhere else. There
+is no OS-specific config-dir lookup (`%APPDATA%`, `%USERPROFILE%\.config`,
+`$XDG_CONFIG_HOME`, `$HOME/.config`). Drop the exe and its `keydb.cfg` in
+the same folder and it works. Callers needing a custom location pass
+`--keydb PATH`, which bypasses this module entirely.