release: 1.7.0

CI/lint hardening since v1.6.14: clippy on stable with the MSRV single-sourced
from Cargo.toml, cargo-deny dependency-audit gate with first-party git sources
allowed (#15), and audience-based comment-guard (#17). No library code changes.

docs/keydb.md

@@ -0,0 +1,432 @@
+# `keydb.rs` design notes
+
+Long-form rationale moved out of `src/keydb.rs` doc comments to satisfy the
+repo's comment-guard line caps. Each section is pointed to from the file by a
+one-line `// See docs/keydb.md#<anchor>` comment.
+
+## candidate-order
+
+Full derivation path this source walks, cheapest-first:
+
+1. per-disc **Unit Keys** (hash hit) → returned terminal, no derivation.
+2. per-disc **VUK** (hash hit) → `uks_from_vuk` over the disc's encrypted
+   title keys.
+3. a **Media Key**, then `derive_vuk` → `uks_from_vuk`. The MK comes from, in
+   order: the disc's stored MK (hash hit); the keydb's **Processing Key**
+   pool walked against THIS disc's MKB via `derive_media_key_from_pk`; or the
+   device-key pool via `derive_media_key_from_dk`. The PK and DK pools
+   resolve the Media Key WITHOUT a VID; the final `derive_vuk` still needs
+   one. The VID is the unlocker's physical VID (`ResolveCtx::vid`) when
+   present, else the keydb entry's OWN stored VID (the `I` field, `vid`) for
+   the non-physical / ISO path. With no VID from either source the MK path
+   cannot complete — return nothing.
+
+The cross-disc MK-pool brute (trying OTHER discs' stored media keys against
+this disc) stays RETIRED: every MK path here is anchored to the matched
+disc's own MKB or stored material.
+
+The library still OWNS the crypto; this source owns only which primitive to
+call with which material. Returning an empty `Vec` is a genuine "no key for
+this disc here".
+
+## file-identity
+
+`(len, modified)` ALONE IS NOT AN IDENTITY. mtime is stored at whole-second
+resolution on HFS+, on many container overlays and on most network
+filesystems, so a writer that replaces `keydb.cfg` within the same second
+with a file of the same byte length leaves `(len, modified)` bit-identical —
+and the cache then serves the superseded parse for the rest of the process's
+life. For a key database that is not a stale page: it is serving retired or
+since-corrected keys while reporting success. The writers that hit this are
+real and named in this crate's own docs: the daily-refresh job, a second
+freemkv process, an operator's editor, a sync tool.
+
+Two independent discriminators close it:
+
+1. `dev` + `ino`. Every writer that publishes ATOMICALLY (temp file, fsync,
+   rename — what `KeydbSource::save` does, and what any correct publisher
+   does) creates a NEW inode, so the stamp differs immediately and
+   unconditionally, whatever the clock or the mtime resolution did. Zero on
+   platforms without them (see `FileStamp::of`), which costs nothing: rule 2
+   stands alone.
+2. A freshness rule on the cache entry — `CacheEntry::is_settled` — which
+   covers the residual in-place rewrite that reuses the inode.
+
+What this still does NOT catch: a writer that FORGES the timestamp, i.e.
+restores an mtime equal to the cached one (`touch -t`, `rsync --times`, a
+tar/backup restore) into the SAME inode with the SAME length. Nothing short
+of hashing the content can see that, and hashing 62 MiB per lookup is
+precisely the cost the cache exists to remove (~141 ms × ≥3 loads per rip).
+`KeydbSource::save` therefore keeps invalidating the cache directly, so the
+one write path this crate owns never depends on any of the above.
+
+## settle-proof
+
+Whether a cache entry's stamp can be TRUSTED to change if the file changes.
+
+The proof, for a filesystem storing mtime at granularity `G`: suppose this
+entry was stamped at `stamped_at` with `stamped_at - modified >= G`, and some
+writer later rewrites the file in place at wall time `T > stamped_at`. The
+recorded mtime `m'` of that write is within one granule of `T`, so
+`m' >= T - G > stamped_at - G >= modified`. Hence `m' != modified` and the
+stamp DOES change. The ambiguous window is therefore exactly the one where
+the cached entry was stamped less than `G` after the file's own mtime — i.e.
+we looked at the file while it was still "hot" — and in that window the
+entry is refused and the file is re-read. Once the file has been quiet for
+`G`, its entry settles and the cache is fully effective again.
+
+Cost: after every keydb write, lookups re-parse for up to `G` (2 s). The
+published keydb is rewritten daily, so that is ~2 s of re-parsing per day
+against a cache that saves ~420 ms per rip.
+
+`modified: None` (a filesystem that reports no mtime) never settles: the only
+two discriminators left would be size and inode, so the entry is re-read
+every time rather than being trusted on a weaker basis.
+
+CLOCK CAVEAT: `stamped_at` is the local clock and `modified` is the
+filesystem's. On a network share whose server clock runs AHEAD of the
+client's by more than `G`, entries never settle (safe: more re-parsing). A
+server clock BEHIND the client's makes an entry settle early, which re-opens
+the same-second window it closes — the inode rule (1) is what covers the
+realistic writers there.
+
+## keydbsource-cache
+
+`KeydbSource` caches the parsed database behind the file's identity stamp.
+The published keydb is ~62 MiB / ~181k entries and a single AACS-cert rip
+parses it repeatedly — `host_certs()` for the drive handshake (freemkv's
+`pipe.rs`/`engine.rs`, autorip's `keysource.rs` each build their own source),
+then the trait `host_certs(mkb)` during resolution, then `get_unit_keys` —
+and every one of those calls used to re-read all 62 MiB from disk and
+re-parse it from scratch, because the struct held nothing but a `PathBuf`.
+
+A keydb replaced underneath this source is picked up on the next call, with
+ONE stated exception: a writer that rewrites the file in place, into the
+same inode, at the same byte length, having restored the previous mtime
+exactly (`touch -t` / `rsync --times` / a backup restore). Everything else —
+any atomic rename, any length change, any real clock advance — changes the
+stamp. `FileStamp` and `CacheEntry::is_settled` carry the full argument and
+the clock caveats; do not weaken either without re-reading them. (The
+earlier wording here — "an externally replaced keydb is still picked up" —
+was simply false for a same-second, same-length replacement, which is the
+common shape of a daily-refresh job on a whole-second-mtime filesystem.)
+
+MEMORY RESIDENCY (accepted, and deliberately recorded rather than fixed
+here): a hit keeps the entire parsed keydb — every disc's Media Key / VID /
+VUK / Unit Keys, the processing- and device-key pools, and the host-cert
+PRIVATE keys — resident for the process's life, where the pre-cache code
+held one parse transiently. Nothing in this crate zeroizes on drop, so that
+material was always exposed to a core dump or a swapped page; the cache
+widens the WINDOW, it does not create the class. Closing it properly means
+zeroize-on-drop across `KeyDb`, `DiscEntry` and libfreemkv's `aacs::types`
+(which owns most of the buffers) — a cross-crate change, tracked separately,
+NOT smuggled in behind a cache fix.
+
+## cached-db-one-open
+
+Errors are the same ones `KeyDb::load` produces, unchanged: a missing file
+is `NotFound` (the documented benign case), an over-cap or non-UTF-8 file is
+`InvalidData`. The stamp costs an `fstat`, not a 62 MiB read.
+
+ONE OPEN, ONE IDENTITY. The file is opened once and both the stamp (`fstat`
+on that handle) and, on a miss, the bytes come from it. The previous shape —
+`std::fs::metadata(path)` and then a separate `KeyDb::load(path)` — resolved
+the path TWICE with no atomicity between them, so a rename landing in the
+gap cached the new file's stamp beside the old file's contents (or the
+reverse); the pairing was then served, looking perfectly valid, until the
+stamp changed again. The stamp is read BEFORE the bytes on purpose: a write
+racing the read then leaves the OLD stamp cached, which the next call
+detects, whereas stamping afterwards would file torn content under a
+fresh-looking identity and pin it.
+
+## cached-db-reemit
+
+Re-emit the parser's rejection summary on EVERY hit, not just on the parse
+that produced it. `KeyDb::parse` is what logs it, and a hit skips `parse`
+entirely — so a daemon that cached a corrupt keydb logged its rejected-row
+counts exactly once, at whatever moment it first read the file, and then
+served that same damaged database to every later rip in silence. Repetition
+is the correct failure mode here: the line is emitted only when the file
+really does have rejected or duplicate rows, and a warning an operator can
+still see on the tenth rip is worth more than one they had to catch on the
+first.
+
+## load-failure
+
+A MISSING keydb is the documented benign case: the app may simply not have
+one, another source may hold the key, and "no keydb" is genuinely "no key
+here". Everything else is NOT that: `InvalidData` is a keydb over the
+128 MiB cap or one that is not valid UTF-8 — i.e. corrupt, truncated, or a
+half-finished download — and a permission/IO failure means the file could
+not be read at all. Reporting either as an empty result made a broken keydb
+indistinguishable from "this disc has no key" (the same conflation as the
+seven-hour 502), with no log to notice it by. Both are now logged AND
+surfaced as errors so the resolver reports a source failure instead of
+`E7022`.
+
+## save-mirror-parse
+
+Mirror `KeyDb::parse`'s disc-entry rule EXACTLY by CALLING it
+(`is_disc_entry_line`), so `save()` never validates + persists content that
+parses to zero usable entries (e.g. a stray "0xDEADBEEF" comment line) and
+the two can no longer drift — they did drift once already, on the `0X` case
+rule.
+
+## unit-keys-from
+
+CPS-unit numbering: a returned `UnitKey::idx` is the POSITIONAL index
+libfreemkv's `resolve_and_apply` turns into the canonical CPS-unit number
+`idx + 1`. For the terminal per-disc unit-key path we therefore map the
+keydb's stored CPS number `num` to `idx = num - 1`, so the committed number
+is byte-identical to the keydb's `num` (and to what the OLD
+`Key::Unit(entry.unit_keys)` path committed). For the VUK / MK paths the
+boil primitive already yields 0-based positional indices, matching
+`parse_unit_key_ro`'s `(i + 1)` after the resolver's `+ 1`.
+
+## unit-keys-per-disc-hit
+
+Per-disc hit (most specific). `find_disc` normalizes the hash form. Without
+a matched entry this keydb has no per-disc material (Unit Keys / VUK / Media
+Key / stored VID) to anchor a derivation for the disc, so it resolves
+nothing. (The PK and DK pools are global, but the cross-disc MK-pool brute —
+trying OTHER discs' media keys against this disc — stays retired; a PK/DK
+pool only ever resolves a disc reached through its own matched entry below.)
+
+## unit-keys-union
+
+UNION every source of terminal keys, then dedup — never first-hit. A stored
+`unit_keys` list can be PARTIAL (the key-import tool only ever sampled the
+CPS units reachable from a playlist, so an orphan unit's key may be
+missing), while the per-disc VUK boils EVERY declared CPS unit. Taking the
+stored list alone (the old return-at-first-path) would shadow the VUK and
+silently drop the orphan unit's key. So gather both and keep a unique-by-key
+list: the read path tries every key per unit, so an extra or stale key is
+harmless — only a MISSING key hurts.
+
+## unit-keys-vuk-or-mk
+
+2. Per-disc VUK — one step, no VID needed; boils ALL declared units.
+3. Else a Media Key path (stored MK / PK pool / DK pool) → VUK → all
+   declared units. The MK itself carries no VID, but the final `vuk_from_mk`
+   needs one: physical (unlocker) VID first, else the entry's stored VID
+   (`I` field), else cannot derive. Either branch yields the COMPLETE
+   declared set, so we take the first that resolves (VUK preferred —
+   cheapest).
+
+## write-atomic
+
+keydb.cfg is the single source of AACS truth, and save/update run
+unattended (first-boot download + daily-refresh thread, with a container
+restart on every release). A bare in-place `fs::write` truncates the file
+before writing, so a SIGKILL (docker stop's grace window), OOM-kill, power
+loss, or ENOSPC mid-write would leave the keydb half-written — the prior
+good copy already gone. A truncated keydb doesn't error at write time; it
+silently breaks key resolution on every later AACS rip. Writing to a temp
+file then renaming (POSIX rename is atomic within a filesystem) means an
+interrupted update leaves the previous keydb fully intact.
+
+The fsync MUST succeed before the rename: a `sync_all` failure (ENOSPC,
+ESTALE on the bind-mounted volume) means the kernel never guaranteed the
+bytes reached stable storage, so publishing them via rename would defeat
+crash-safety. The temp name is unique per call (pid + monotonic counter) so
+a concurrent update can't share a fixed temp path and rename a mangled file
+over the keydb.
+
+## get-unit-keys-trait
+
+A MISSING keydb is not an error — it simply yields no keys (another source
+may have them), the same as the library's own loader. A keydb that exists
+but cannot be USED (over the size cap, not UTF-8, unreadable) is a source
+failure and returns `Err`: it says nothing about whether this disc has a
+key, and reporting it as an empty result is the same looks-like-success
+failure as a 502 reported as `E7022`.
+
+The keydb carries no AACS 2.1 forensic index keys today, so it does not
+override `get_fmts_indexes` — the default (empty) opts it out, and an FMTS
+disc's forensic set comes from the online source.
+
+## host-certs-trait
+
+Expose the keydb's host certs through the trait — the OEM/AACS cert-auth
+route collects them across every source via this method. Wires the disc's
+MKB generation through for revocation filtering (the keydb parser's
+`; Revoked in MKBv<N>` annotation): a cert revoked at generation `R` is
+withheld once the disc's generation reaches `R`.
+
+## test-pinned-mtime
+
+Force a file's mtime to `PINNED_MTIME`, so two different files can be made
+bit-identical to a `(len, mtime)` stamp on purpose. The cache tests that
+matter all depend on FORCING the indistinguishable case rather than hoping
+the filesystem produces it: on APFS (nanosecond mtimes) a same-second
+replacement is otherwise detected by mtime alone, and a test that relies on
+that passes with the fix reverted — which is worth nothing.
+
+## test-committed
+
+The committed `(cps, key)` pairs libfreemkv's `resolve_and_apply` derives
+from a source's Unit Keys: positional `idx` → canonical CPS number
+`idx + 1`. The KATs compare against THIS to prove byte-identical parity with
+the OLD `Key::Unit` / resolver-derived commit.
+
+## test-mock-ctx-stub
+
+`MockCtx` implements the full `ResolveCtx` trait so it can stand in for a
+real disc, but `KeydbSource` itself never calls `title()`/`samples()` (those
+matter to the online source, not the local keydb). Exercise them directly
+so the mock's trait surface is proven correct too.
+
+## test-kat-a
+
+Here `enc_title_keys` is empty, so the VUK can't derive anything — only the
+stored list contributes, and it commits byte-identically to the stored
+`(cps, key)` pairs.
+
+## test-orphan-union
+
+Orphan-unit completeness (the real keydb bug): an entry stores only `uk1`
+(the key-import tool sampled one reachable CPS unit) but ALSO carries the
+VUK, which boils BOTH declared units. The old return-at-first-path handed
+back just `[uk1]`, shadowing the VUK and silently dropping the orphan unit.
+The union must return BOTH — the stored uk1 AND the VUK-derived second unit.
+
+## test-kat-c
+
+A hash hit with a Media Key and a physical VID (from the unlocker) derives
+`MK → VUK → UK`. The PHYSICAL VID must be used in preference to the keydb's
+stored VID — proven by giving the entry a DIFFERENT stored VID and showing
+the result tracks the physical one.
+
+## test-kat-f
+
+Owner decision #1 (AACS): a keydb Processing Key must be walked against the
+matched disc's own MKB to recover the Media Key, then driven down the full
+chain `PK → MK → VUK → UK`. The disc entry carries NO stored MK/VUK/UK — the
+only key material is a global `PK` row — and the result must be the real
+Unit Keys, byte-identical to deriving from the recovered MK.
+
+The MKB + PK use a known-answer construction (a planted PK whose derived MK
+satisfies the synthetic verify record); the constants are precomputed AES
+vectors so this crate needs no AES primitive of its own. They mirror
+libfreemkv's `boil::mk_from_pk_drives_full_chain_to_uks` KAT.
+
+## test-repeated-lookups
+
+A single AACS-cert rip drives this source at least three times — the
+scan-options builder's inherent `host_certs()`, the trait `host_certs(mkb)`
+during the handshake, then `get_unit_keys` — and each call used to re-read
+and re-parse the WHOLE file, because `KeydbSource` held nothing but a
+`PathBuf`. Measured on the real 62 MiB / 184,860-entry keydb: ~140 ms per
+parse in `--release`, i.e. ~420 ms and ~186 MiB of file reads per rip,
+against ~1 µs for a cache hit.
+
+Catches the mutation that drops the cache (every call re-parses: count 3)
+and the mutation that makes the cache unconditional (see the invalidation
+tests below).
+
+The mtime is pinned into the PAST first, because a cache entry is only
+trusted once the file has been quiet for the mtime granularity
+(`CacheEntry::is_settled`) — a keydb written microseconds ago is
+deliberately re-read. Unix-only: pinning an mtime needs `touch -t` (no
+`filetime` dependency for a test).
+
+## test-changed-keydb
+
+The cache must not go stale: a keydb replaced UNDERNEATH the source (the
+daily-refresh thread writing through a different `KeydbSource`, or an
+operator dropping in a new file) has a different size/mtime stamp and must
+be re-read. Catches the mutation that caches on first load and never
+re-stats — which would pin a rip to a keydb deleted hours earlier.
+
+## test-two-discriminators
+
+The defect both of the inode/settle tests below pin: `(len, mtime)` is not a
+file identity. An external writer — the daily-refresh job, a second freemkv
+process, an editor, a sync tool — that replaces `keydb.cfg` inside one mtime
+granule with a file of the SAME length leaves that pair bit-identical, and
+the source then serves the superseded parse for the rest of the process's
+life: retired or since-corrected keys, reported as success.
+
+Each test disables the OTHER discriminator through `mtime_granularity`, so
+neither can pass on the strength of the one it is not testing, and neither
+depends on the host filesystem's real mtime resolution (on APFS the naive
+scenario is not reproducible at all).
+
+## test-inode-rename
+
+A keydb replaced by ATOMIC RENAME — how every correct publisher, including
+this crate's own `save()`, installs a new file — must be re-read even when
+length and mtime are identical. `mtime_granularity: 0` makes every entry
+settle, so ONLY dev+ino can distinguish the two files here.
+
+Catches the mutation that drops `dev`/`ino` from `FileStamp` (the original
+`(len, modified)` stamp): the source then keeps answering with the previous
+keydb's keys.
+
+## test-inplace-rewrite
+
+A keydb rewritten IN PLACE — same inode, same length, mtime forced back to
+the same value — must still be re-read while the cached entry is too young
+to trust. `mtime_granularity` is set to a decade so nothing ever settles,
+which is the same condition a real same-second, whole-second-mtime
+replacement creates and the only way to reproduce it deterministically on a
+nanosecond-mtime filesystem.
+
+Catches the mutation that deletes the `is_settled` check from `cached_db`
+(every stamp match trusted unconditionally, which is what the original code
+did): dev, ino, len and mtime are ALL identical here, so without the
+freshness rule the source serves the old keys forever.
+
+## test-corrupt-warns
+
+`ParseStats::log` runs inside `KeyDb::parse`, and a cache hit skips `parse`
+— so once the cache landed, the rejected-row summary for a given file was
+emitted exactly ONCE for the entire life of the process, while that same
+damaged file went on being served to every subsequent rip. A long-running
+daemon (autorip) logs it at startup, the operator misses it in the startup
+noise or loses it to log rotation, and there is no second chance for as long
+as the process runs.
+
+Catches the mutation that deletes the `emit_parse_stats` call from the
+cache-HIT arm of `cached_db`: warnings would drop to 1 while the file is
+still served three times.
+
+## test-healthy-silent
+
+Catches the mutation that makes `emit_parse_stats` unconditional (drops
+`ParseStats::log`'s empty-counts early return).
+
+## test-fresh-not-trusted
+
+Catches the mutation that widens the window to something unbounded (a
+granularity of hours would re-parse 62 MiB on every lookup all day) by
+pairing with `repeated_lookups_parse_the_keydb_once`, which proves the entry
+DOES settle once the mtime is in the past.
+
+## test-save-invalidates
+
+`save()` replaces the very file this source reads, so it drops the cached
+parse ITSELF rather than trusting the (size, mtime) stamp: a same-size
+rewrite that lands inside the filesystem's timestamp granularity is the one
+change the stamp cannot see — and it is exactly the change this crate
+performs (the daily-refresh thread re-saving a keydb through the source it
+then reads from).
+
+The test forces that indistinguishable case rather than hoping for it: the
+file is written, stamped to a FIXED mtime, cached, re-saved with different
+keys of the SAME length, and stamped back to the same mtime. To the stamp
+the two files are identical, so only the explicit invalidation can produce
+the new key. Unix-only because pinning an mtime needs `touch -t` (no
+`filetime` dependency for one test).
+
+## test-corrupt-is-error
+
+`KeyDb::load` fails distinguishably for a keydb that is over the 128 MiB cap
+or not valid UTF-8 — a truncated download, a half-written file, a binary
+blob dropped in by mistake. Both used to collapse into `Ok(Vec::new())` with
+ZERO tracing, i.e. into the benign "no key for this disc" the resolver
+reports as `E7022` — the same looks-like-success failure as the seven-hour
+502. Only a MISSING file is genuinely benign.
+
+Catches the mutation that restores `Err(_) => Ok(Vec::new())`.
+</content>