Two defect classes dominate this change, and both are the same shape: something
that should have been loud was silent.
KEY MATERIAL IN A PUBLIC `Debug`. `KeyDb` and `DiscEntry` both carried a derived
`Debug` over raw key bytes — every processing key, and every disc's Media Key,
VID, VUK and Unit Keys. Both are public API, so a single `{:?}` in a caller's
log line, a `tracing` field, or an `assert_eq!` panic message published the
whole keydb: on the real file that is 184,860 discs' key material plus the
processing-key pool. libfreemkv's equivalent types (`aacs::types`) all
hand-write a redacting `Debug` for exactly this reason; this crate's copy of the
parser dropped the convention when it was relocated. Both impls are restored,
field-for-field in the same style, and pinned by tests that fail against the
derive.
A SOURCE THAT COULD NOT ANSWER IS NOT A SOURCE THAT SAID "NO KEY". `OnlineSource`
was engineered at length to separate `Err` (the service could not answer) from
`Ok(empty)` (it answered and holds nothing) after a seven-hour run of HTTP 502s
was reported to operators as E7022 "no key for this disc". Three places
re-collapsed that distinction:
* `MultiSource` discarded every inner `Err` (`if let Ok(uks) = ..`), so a
composition whose sources had all failed returned a clean `Ok(empty)` — the
incident reproduced one layer up, in both the base and the forensic path. It
now returns the FIRST source failure when nothing resolved, matching
libfreemkv's own resolver, while a later success still wins outright.
* `KeydbSource` turned every `KeyDb::load` failure into `Ok(empty)` with no
log at all. A keydb over the 128 MiB cap or one that is not valid UTF-8 is a
corrupt, truncated or half-written file, not a disc without a key; those are
now logged and returned as errors. A MISSING keydb stays the documented
benign empty.
* A mistyped port in the key-service URL (`https://host:notaport/`) reused the
whole authority as the hostname, so it failed DNS and surfaced as
`Unreachable` — a standing misconfiguration reported as a transient outage,
the one collapse `GuardFail`'s doc says must never happen because the two
halves demand opposite operator actions. It is now a `Config` rejection, the
same verdict the bracketed-IPv6 branch already gave.
The keydb parser was equally silent: every malformed `| DK` / `| PK` / `| HC` /
`| HC2` / `0x…` row was dropped with no counter and no log, and so was a row
lost to the entry cap or overwritten by a duplicate hash. A byte-shifted mirror
download therefore passed `save()`'s validation, was atomically committed as the
new keydb, and resolved nothing for every disc past the damage — indistinguishably
from a healthy file. Rejections are now counted by reason and summarised in one
log line per parse. The real published keydb parses with zero rejections and zero
duplicates, so the summary stays quiet on healthy input.
Also fixed, with tests:
* The 62 MiB / 184,860-entry keydb was re-read and re-parsed on EVERY call —
~140 ms each in release, three or more times per AACS-cert rip. `KeydbSource`
now caches the parse behind the file's (size, mtime) stamp (~1 µs on a hit),
re-reads a file replaced underneath it, and drops its own cache on `save()`
so a same-size rewrite inside the timestamp resolution cannot be missed.
Peak RSS is unchanged; the parse simply is not repeated.
* A disc row prefixed `0X` was dropped silently, though `parse_hex` in the same
file deliberately accepts both cases. `save()`'s entry count now CALLS the
parser's own predicate instead of restating it, so the two cannot drift again.
* Each disc hash was stored twice — once as the map key, once in the entry —
~13 MB of duplicate text at real scale, now one shared allocation.
* `OnlineSource` built a fresh agent per query, paying a second TLS handshake
on every FMTS disc. The agent is reused only while the freshly re-resolved
and re-guarded address set is identical, so the anti-rebinding guarantee is
untouched.
Test coverage for defences that were never exercised: the cleartext-`http://`
refusal, the over-cap MKB skip and the too-few-samples skip (a reordering that
let a plaintext POST through — carrying the bearer token and base64 key material
— would have shipped green), and the 1 MiB reply cap. The real-scale tests used
to `return` early when `KEYDB_PATH` was unset, so CI ran them, asserted nothing
about 181k-entry behaviour and reported success; they are `#[ignore]`d with a
documented run command and now fail loudly if run without their fixture.
Bump to 1.6.0 (workspace sync, no source change). Fix stale get_uk ->
get_unit_keys references in the crate description + comments, correct the
libfreemkv version note, and drop a phantom Mapfile mention from the tests.
Base per-CPS-unit keys and the AACS 2.1 forensic index set are now two explicit
trait operations instead of one overloaded get_uk whose return count was implicit.
keydb serves unit keys (forensic opts out via the default); online implements both;
MultiSource drives both in order. Teed up for 1.4.5. Local WIP baseline.
leak-guard bans the .invalid TLD (grouped with private .internal/.local/etc);
example.test is the RFC 6761 reserved test TLD — equally unreachable, not on
the ban list — so the leak-guard CI check passes.
libfreemkv deleted the aacs::boil veneer, so switch the resolve path to the
raw primitives: derive_media_key_from_{pk,dk}, derive_vuk, decrypt_unit_key
from aacs::derive, and the newtypes from aacs::types. Add a local uks_from_vuk
helper composing decrypt_unit_key. No behaviour change; fmt/clippy/test green
on Rust 1.86.
unit_keys_from now resolves the Media Key in order: stored per-disc MK
-> keydb Processing Key pool (mk_from_pk vs this disc's own MKB) ->
device-key pool (mk_from_dk), then MK+VID -> VUK -> UK. MK/VUK entries
still honored directly; cross-disc MK-pool brute stays retired. Fixes the
factually-wrong justifying comment + adds PK-pool KATs.
Add tests/key_sources.rs exercising the published KeySource impls
end-to-end over real fixture files and the libfreemkv parsers:
- KeydbSource: disc-hash lookup from a real keydb.cfg (VUK hit ranks
ahead of the universal DK pool), hash miss yields only the pool,
missing file is silent/not errored, label + needs_samples, and
host-cert serving from a | HC | row (inherent and trait paths).
- paths: exe-local search list, default == search head, existing path
reflects on-disk state (local-only, no OS fallback).
- MapfileSource: persisted # freemkv-uk: keys read back as a terminal
Key::Unit, one-shot exhaustion, missing/keyless mapfile offers nothing.
- OnlineSource: unconfigured no-op (no network), one-shot latch,
metadata, and validate_keyserver_url scheme/SSRF gating.
- MultiSource: caller-supplied order/precedence (and its reverse),
empty-source skip, needs_samples/errored OR-aggregation, nesting, and
a real keydb-then-mapfile precedence chain over fixtures.
Matthew Jackson