Compare commits

...
68 Commits
Author SHA1 Message Date
Matthew Jackson 3c6ef54cf3 tests: loopback idle bound T_IDLE is 1 s, so a slow macOS runner's scheduling gaps stay inside every margin 2026-10-01 18:06:06 -07:00
Matthew Jackson 73c7f05b4b tests: Stop tests against a silent stub idle T_STOP_IDLE (2 s), past the Stop 2026-10-01 17:20:54 -07:00
Matthew Jackson e6320d52c9 tests: the stopped reachability query idles 2 s and gets 10 s to drain 2026-10-01 16:54:40 -07:00
Matthew Jackson 49cf396c07 tests: the stopped reachability query idles past the Stop, so only the Stop ends it 2026-10-01 16:53:58 -07:00
Matthew Jackson 940bb215ab online: follow libfreemkv's halt::Progress rename to Liveness 2026-09-30 20:36:55 -07:00
Matthew Jackson 18b590407c Cargo.lock: restore (accidentally changed by local patch) 2026-09-30 20:24:57 -07:00
Matthew Jackson 116b33739b tests: halt-landing tests assert Halted, not a tight wall-clock bound 2026-09-30 20:24:46 -07:00
Matthew Jackson d935fa6ffd online: allow LAN/loopback/link-local key-service hosts, reuse libfreemkv is_blocked_ip 2026-09-30 20:03:38 -07:00
Matthew Jackson b65959d3a1 ci: run cargo-deny from the repo dir so the .cargo patches apply 2026-09-30 16:09:03 -07:00
Matthew Jackson 3f08e980a0 ci: path-patch freemkv-unlock; dependabot ignores org workflow refs
libfreemkv now deps freemkv-unlock by version with a git-tag override, so every root that reaches it patches it. Dependabot config unified; it no longer proposes freemkv/.github v1->v2.
2026-09-30 15:53:05 -07:00
Matthew Jackson 6c8e2c5c46 Allow forensic phase fallback after definitive key service misses 2026-09-29 17:20:43 -07:00
Matthew Jackson ba357e46f5 online: wire ctx.halt()/ctx.progress() into the query worker (ST-K1b) 2026-09-28 08:18:38 -07:00
Matthew Jackson a815a0a578 online: ST-K1b tests for ctx.halt()/ctx.progress() wiring (RED) 2026-09-28 08:15:20 -07:00
Matthew Jackson ba6f213be3 KU-E1b: the key-service query is not yet Stop-aware (wording)
check_keyserver_url_static's rustdoc and its test comment no longer
call the first query halt-aware: OnlineSource runs it under its own
Halt until ST-K1b wires ctx.halt() (J10).
2026-09-28 07:18:27 -07:00
Matthew Jackson 37568b1002 test: use the RFC 2606 .test TLD for the unresolvable host (leak-guard flags .invalid) 2026-09-28 07:04:48 -07:00
Matthew Jackson 40389acd3f KU-E1 review minor 4: partial stored keys plus a Km still report NoVid
The keydb keeps a NoVid miss reason beside the keys it did return, so
resolve sets vid_would_help for the CPS units only the VID could
derive. Other miss reasons are still reported only with no key.
2026-09-28 06:28:54 -07:00
Matthew Jackson 54508d8ec2 KU-E1 review minor 4 red: partial stored keys plus a Km still report NoVid
A keydb entry with stored unit keys for some CPS units and a Media Key
but no VID returns the stored keys with an empty miss path, so
resolve cannot tell that the VID would derive the rest (J23).
2026-09-28 06:28:47 -07:00
Matthew Jackson 2c93420970 KU-E1: a DNS-free key-service URL check for factory builds
check_keyserver_url_static rejects every Config fault (scheme, host,
port, a literal non-public address) with no lookup. A factory build,
which no Stop can reach, uses it; the host lookup and its SSRF guard
run at the first query on the halt-aware worker, as they already did.
A test-only counter proves the check starts no lookup.
2026-09-28 06:24:52 -07:00
Matthew Jackson c48faa042b KU-E1 red: a DNS-free key-service URL check for factory builds
The engine checks the key-service URL, a DNS lookup with a fixed 10 s
timeout that Stop cannot interrupt, on every factory build, and no
Halt reaches a factory. The test asks for a static check that rejects
Config faults with no lookup.
2026-09-28 05:45:58 -07:00
Matthew Jackson ab93818012 KU-E1 J23: OnlineSource and MultiSource report uses_vid
OnlineSource::uses_vid is true (it sends vid_b64); MultiSource is
any(inner); KeydbSource keeps the default false.
2026-09-28 05:31:51 -07:00
Matthew Jackson f3b66307cc KU-E1 J23 red: which key sources consume the disc's VID
OnlineSource sends the VID (vid_b64), so it uses it; MultiSource uses
it if any inner source does; KeydbSource does not (its Km-without-VID
case is reported per disc, as a NoVid miss path).
2026-09-28 05:31:23 -07:00
Matthew Jackson ad59f3ce45 KU-K1 review: peek the reachability slot, MultiSource forwards last_failure
record_last_failure_transport now reads LAST_DECODE_REACHABILITY.with(|c|
c.get()) directly instead of take_last_decode_reachability(), so it no longer
erases the slot freemkv-library's server reads after resolve
(server/ripper/mod.rs:505, :1361) to avoid a second key-service call. Green:
the regression test now finds the verdict still in the slot after
get_unit_keys/get_fmts_indexes. MultiSource::last_failure_was_transport
forwards any(inner), matching answer_depends_on_samples (KU3-6).
2026-09-28 04:19:04 -07:00
Matthew Jackson 4e83b3c1f8 KU-K1 review: red — record_last_failure_transport erases the reachability slot
Opus's review: freemkv-library's server calls take_online_decode_reachability()
after resolve (server/ripper/mod.rs:505, :1361) to classify a no-key without a
second probe. record_last_failure_transport currently TAKES the thread-local
slot, so the server sees None and fires probe_online_reachability — a second
key-service call, and a definite 422 misread as an outage. Red: the slot is
gone after get_unit_keys/get_fmts_indexes. Also adds the forensic-path,
mid-body-stall, transport-then-5xx-reset and MultiSource forwarding tests
requested in review; the latter fails since MultiSource has no override yet.
2026-09-28 04:18:02 -07:00
Matthew Jackson c5985d1c87 KU-K1: keydb/multi sample-dependence and online transport-class failures
Per keys-upfront-design KU-K1 (§8.2): KeydbSource::answer_depends_on_samples
is false (keyed by disc hash, asked once per resolve, N-KU10); MultiSource's
is any(inner) (KU3-6), so a nested sample-dependent source still gets the
per-piece ask. OnlineSource::last_failure_was_transport (J15) is true only
when the most recent query recorded DecodeReachability::Transport (DNS,
connect refused, or an idle stall — never an HTTP answer, 5xx included), and
resets on the next success, so resolve's J13 retry (stop-design-v5 T31)
targets exactly the requests that never got an answer.
2026-09-28 04:13:03 -07:00
Matthew Jackson 8f923fd5f9 KU-K1: tests for answer_depends_on_samples and last_failure_was_transport
KeydbSource::answer_depends_on_samples (KSK1) and MultiSource::answer_depends_on_samples
(KSK2, KU3-6 "any(inner)") both default to `true` today, so a keydb-then-online
composition would be re-asked per piece instead of once. OnlineSource::last_failure_was_transport
(J15) defaults to `false`, so a transient DNS/connect/idle failure would never be
retried by `resolve` (J13). Red: these six assertions fail against today's defaults.
2026-09-28 04:12:12 -07:00
Matthew Jackson 18fdab6b77 KU-P1: decrypt the real-unit test through test_util::decrypt_unit
aacs::content::decrypt_unit becomes pub(crate) at KU-X2 (keys-upfront design
v3.4 §2.2); the ignored real-unit test now uses the test-util door. The
feature comes in as a dev-dependency (resolved through the sibling path patch).
2026-09-28 00:22:08 -07:00
Matthew Jackson dfa02adbe2 online: a stopped key query never sends and builds no body
Review fixes, green half:
- PostJob carries the caller's Halt and re-checks it after the host lookup,
  before connecting. A Stop during DNS then never uploads the key material
  or bearer token, so a reopen's query is the only one the service sees.
- query_with returns Halted before reading or base64-encoding the MKB when
  a Stop is already pending.
2026-09-27 23:50:33 -07:00
Matthew Jackson cbab284c85 online: tests that a stopped key query sends nothing and reads no disc material
Review fixes, red half: a Stop during the host lookup must never reach the
service (never call the key service twice), and a pending Stop returns
before the body is built. KT7a's u64::MAX check now asserts saturation.
2026-09-27 23:49:08 -07:00
Matthew Jackson 0ffdb2a9f6 online: stall-only key-service timeouts and mid-flight Stop (ST-K1a)
Commit 2 of 2 (stop-design-v5 §2.7, T15-T17, D3, D4):

- query_with builds the body on the caller's thread and runs DNS + POST on
  a worker, capped at 4 per host. The caller waits for a slot and for the
  result halt-aware in 20 ms slices, so Stop returns Halted within a slice
  at any stage. The abandoned worker frees its slot at its own stall bound.
- Reachability is recorded on the caller's thread, only when the worker's
  result arrives. A Stop records nothing.
- IdleConnector, chained after DefaultConnector, owns every post-connect
  bound. Each write and read gets a fresh 60 s stall bound (T16), and the
  wait for a reply's first byte gets 60 s + request bytes / 256 KiB/s (T17).
  The 180 s phase totals are gone: no total is configured.
- A reply body that stalls or drops records Transport reachability (T16).
  An over-cap body still records its Status.

The trait path passes a fresh Halt. ctx.halt()/ctx.progress() is ST-K1b,
after KU-X2. No public API change.
2026-09-27 23:41:43 -07:00
Matthew Jackson adbb1a12a0 online: ST-K1a tests for stall-only key-service timeouts and mid-flight Stop
Commit 1 of 2 (stop-design-v5 §5.0): KT1-KT10 plus guards G12/G13, with
signatures that keep today's behaviour (query_with ignores the halt,
hardened_agent_with maps idle onto the old phase totals, the first-byte
budget does not scale), so each test compiles and fails on its assertion.
KT9 replaces the 180 s totals test. The ctx.halt()/ctx.progress() wiring
is ST-K1b (after KU-X2).
2026-09-27 23:37:22 -07:00
Matthew Jackson 05281249d1 keydb: hand out host certs best first, with no disc input
Unannotated certs in keydb order, then revoked ones by latest revocation.
The trait ignores its mkb hint; KeyDb::host_certs(mkb) is deprecated.
2026-09-27 17:24:04 -07:00
Matthew Jackson cedbc4b1fa online: offline Temporary-verdict test; proxy wording
Drop the real-DNS .invalid test for an offline check of every GuardFail::Unreachable reason; say every key lookup failed.
2026-09-27 16:44:34 -07:00
Matthew Jackson 1606ff3ca3 online: review nits for the proxy fix and keyserver URL verdict
Correct the rationale: the env proxy was resolved to the key service's pinned address, so every lookup failed.
Gate set_last_decode_reachability behind a test-hooks feature, mark the verdict types non_exhaustive, test a .invalid host, add CHANGELOG.
2026-09-27 12:40:09 -07:00
Matthew Jackson 10b0348db8 online: never use an env proxy; typed keyserver URL verdict and reachability test hook
The key-service agent disables ureq's env proxy, which would re-resolve the host and bypass the pinned, SSRF-guarded addresses.
check_keyserver_url returns a Permanent/Temporary KeyserverUrlRejection; set_last_decode_reachability plants a verdict for caller tests.
2026-09-27 12:26:39 -07:00
Matthew Jackson d0a3ab852b v1.7.7: bump version (unified release) 2026-09-26 21:16:59 +00:00
Matthew Jackson 91b906fa61 Prepare 1.7.7 with concise comments and shared CI policy 2026-09-26 11:46:35 -07:00
Matthew Jackson e987506d96 v1.7.6: bump version (unified release) 2026-09-26 04:05:43 +00:00
Matthew Jackson 60cff0ead6 chore: absorb qa into dev for lockstep promote [skip-media] 2026-09-25 12:25:47 -07:00
Matthew Jackson c333c6000b v1.7.6: bump version (unified release) 2026-09-25 05:32:01 +00:00
Matthew Jackson adf2401c06 chore(qa): gate 1.7.6 promote — media matrix not needed [skip-media]
The 1.7.6 unified release ships no rip-path change (Linux desktop shell,
notifications, menu_layout refactor, XDG paths). Real-media QA on EC2
adds ~2h + real money to the release cycle to prove code that never
enters the rip pipeline. This empty commit puts `[skip-media]` in qa's
HEAD so the media matrix in qa.yml auto-skips per the documented bypass;
the full-platform builds/tests still run.
2026-09-24 21:37:29 -07:00
Matthew Jackson b89b7c9660 chore(qa-sync): absorb qa's post-1.7.5 release commits into dev
Release-orchestrate committed to qa (v1.7.5 version bump, etc.) but did
not push those commits back to dev, so promote's fast-forward gate
refused (qa was diverged from dev). This is an ours-merge: the commit
records qa as an ancestor of dev without applying its content, since
dev has already moved past 1.7.5 to 1.7.6 with the equivalent
information in place.
2026-09-24 21:21:11 -07:00
Matthew Jackson 1121522534 chore(release): bump to 1.7.6, follow the CI-required release-notes shape
Bumps Cargo.toml + Cargo.lock to 1.7.6. Also fixes a CI-shape defect the
previous release left behind: dev's CHANGELOG still said
`## [1.7.5] — UNRELEASED` after 1.7.5 shipped (release-orchestrate dated
main but did not push a fresh UNRELEASED heading back to dev). Corrects
that here so the changelog job accepts 1.7.6 on the way to qa:

  ## [1.7.6] — UNRELEASED
    - Version aligned; unified release driven by freemkv 1.7.6.
  ## [1.7.5] — 2026-09-23
    - Version aligned (or actual content, per the shipped main entry).

The driver for 1.7.6 is the freemkv Linux desktop shell (GTK4 +
libadwaita) + rip-finished desktop notification (issue #56).
2026-09-24 20:23:08 -07:00
Matthew Jackson 146f53873d v1.7.5: bump version (unified release) 2026-09-23 04:40:56 +00:00
Matthew Jackson b925a4fa50 docs: 1.7.5 changelog entry for the unified release [skip-media]
Version aligned to 1.7.5. No functional change to this crate — the release
is driven by freemkv-unlock mirroring the freemkv-firmware 0.9.0 ABI, where
the drive's separate Ake (0x06) and Bus (0x07) levers are retired into a
single Encryption (0x06) lever.
2026-09-22 21:05:01 -07:00
Matthew Jackson 4f169a4881 v1.7.4: bump version (unified release) 2026-09-21 23:03:38 +00:00
Matthew Jackson e251e0bf6f docs: mark 1.7.4 changelog heading UNRELEASED (release workflow dates it) 2026-09-21 15:07:11 -07:00
Matthew Jackson 6704c1e05b docs: 1.7.4 changelog + consumer release notes 2026-09-21 11:47:20 -07:00
Matthew Jackson a34980970b docs/ci: drop stale local-precommit/release.sh references — CI is the only gate 2026-09-21 09:11:35 -07:00
Matthew Jackson 948511853e test(online): trim interpret_reply reachability aside to 3 lines for comment-guard 2026-09-21 08:51:04 -07:00
Matthew Jackson 373cdc4cf2 ci: adopt central freemkv/.github guards + checkout-siblings composite
Thin-caller comment/leak/identity guards → freemkv/.github reusable
workflows; drop local ci/comment-guard.py and ci/leak-guard.sh. Replace
per-job sibling checkout + .cargo/config heredoc with the shared
checkout-siblings@v1 composite (checkout libfreemkv+freemkv-unlock,
patch libfreemkv). Add cancel-in-progress concurrency to ci.yml/qa.yml;
standardize actions/checkout to @v5.
2026-09-21 08:48:58 -07:00
Matthew Jackson a60b021c35 Merge: ureq 3.4.2 + bounded reply body (supersedes Dependabot #12) 2026-09-20 20:15:19 -07:00
Matthew Jackson fc8b6e3a05 bump ureq 3.4.2 + preserve rolling-idle body timeout
ureq 3.4.1 (#1194, "Fix timeout budgets restarting and applying to later
phases") made timeout_recv_response bound only the response HEADERS and
turned timeout_recv_body into an ABSOLUTE total-body deadline. The online
key-service POST set recv_response only, so under 3.4.2 the reply body read
had NO deadline at all — a server that sent headers then stalled mid-reply
would hang the key request forever.

The /decode reply is a small JSON key answer that arrives in one read (a
few hundred bytes; the 1 MiB cap is a defensive anti-OOM ceiling, not an
expected size), so this is a fine total-cap case, not a large-body slow-link
download: no rolling-idle connector is warranted. Set timeout_recv_body to
the same TIMEOUT_SECS total ceiling so the POST can never hang forever.

Regression guard the_reply_body_read_is_bounded_not_only_the_headers asserts
recv_body is now set. Full suite + precommit (fmt/clippy/deny/tests) green.
2026-09-20 20:03:54 -07:00
Matthew Jackson bdbc805ba5 keydb: report match state + entry shape to de-conflate "no entry" (#46)
Implement KeySource::resolve_unit_keys for KeydbSource by splitting
unit_keys_from into resolve_from, which reports: whether find_disc
matched; the specific miss reason (NoVid when a Media Key has no VID,
else NoDerivableKey); the matched entry's shape (vuk / unit_keys count /
media_key / keydb_vid / enc_title_keys len / vid_available); and the
loaded entry count for a self-diagnosing true-miss verdict.

MultiSource forwards resolve_unit_keys so a matched inner source's richer
verdict survives composition. No derivation-logic change: the locked
VID-per-path rule is intentional and the investigation found no
correctness bug there — only the conflated verdict. Tests cover
matched-no-VID, matched-no-material, true miss, and the success path.
2026-09-20 19:16:06 -07:00
Matthew Jackson fa75d28441 freemkv-keysources 1.7.4: keydb loads every entry — drop entry cap, strip BOM, count malformed rows
- Remove the MAX_DISC_ENTRIES cap: a keydb is a database and must hold every
  disc row it contains (a count-based drop silently lost real discs).
- Strip a leading UTF-8 BOM before parsing so a Windows-saved keydb's FIRST
  disc row survives (U+FEFF is not trim()-able whitespace).
- Gate the disc branch on the 0x/0X prefix alone and COUNT a malformed 0x row
  (missing " = ") as rejected instead of silently ignoring it — surfaces the
  "hash is in the file but nothing resolves" class of bug.
- Tests: BOM, CRLF, leading whitespace, case/prefix-insensitive lookup,
  duplicate-hash last-wins, unit-key-only rows, 600k-entry no-drop, and the
  load_failure verdicts (permission / invalid / missing).

[skip-media]
2026-09-20 09:43:41 -07:00
Matthew Jackson 5d2e11a4dd freemkv-keysources 1.7.4: version bump (unified 1.7.4) 2026-09-19 14:03:36 -07:00
Matthew Jackson 50bd0e4be2 v1.7.3: bump version (unified release) 2026-09-19 05:31:15 +00:00
Matthew Jackson 148419c2bb changelog: 1.7.3 unified-release entry (freemkv --share + libfreemkv multi-angle UHD fix) 2026-09-18 22:18:41 -07:00
Matthew Jackson 1b97e6186d merge: record v1.7.2 release into dev (-s ours; keep path deps + dev's next-cycle work) 2026-09-18 21:39:15 -07:00
Matthew Jackson 23988922eb v1.7.2: bump version (unified release) 2026-09-18 18:37:25 +00:00
Matthew Jackson e4ecc3cf83 merge: record v1.7.1 main into dev (dev already at 1.7.1; keep path deps) 2026-09-17 23:08:12 -07:00
Matthew Jackson 55ef44e74e comment-guard: trim over-cap comment blocks to caps (comments only)
Restructure 10 over-cap comment blocks in keydb.rs and online.rs to fit
comment-guard's caps (inline // asides <=3 lines, /// item-doc prose <=8)
by tightening wording. No code changes; behavior unchanged.
2026-09-17 22:37:38 -07:00
Matthew Jackson 173a532a6e deps: revert ureq to 3.4.0 (3.4.2 body-deadline regression; keep the rest of this repo's dep changes) 2026-09-17 22:29:32 -07:00
Matthew Jackson 6879df55eb deps: rustls 0.23.43→0.23.45 (RUSTSEC TLS-1.3 advisory) + ureq 3.4.2 2026-09-17 22:18:07 -07:00
Matthew Jackson 0a735498fb online: block NAT64 well-known prefix in SSRF guard; fix https-only doc 2026-09-15 14:06:04 -07:00
Matthew Jackson 531e3a40bf fix(keysources): resolve own-repo audit findings (v1.6.0->dev)
online.rs:
- validate_keyserver_url/resolve_and_guard now reject any non-https:// scheme
  as a Config fault, so a cleartext misconfig is caught ONCE at config time
  instead of failing every rip via query()'s runtime https check.
- is_blocked_ip: add 198.18.0.0/15 (RFC 2544 benchmarking) and 192.0.0.0/24
  (RFC 6890 IETF protocol assignments, incl. 192.0.0.170/171) to the SSRF guard.
- DNS in-flight cap is now PER HOST, not process-global, so black-holed lookups
  to one dead keyserver can't starve resolution of a healthy different keyserver.
- reject a non-empty, non-":port" tail after a bracketed IPv6 authority
  (e.g. https://[::1]extra/) instead of silently dropping it.

keydb.rs / keydb_format.rs:
- MAX_KEYDB_BYTES defined once in keydb_format and re-used by keydb.rs.
- CacheEntry::is_settled falls back to inode identity + stamped_at age when the
  mtime is absent (mtime-less FS) or in the future (clock skew), so the ~62 MiB
  keydb no longer re-parses on every lookup.
- save()'s entry-count guard counts DK/PK/HC/HC2 rows via the real parsers
  (is_parseable_entry_line), so a prefixed-but-malformed row can't pass entries>0.

Tests added for each fix.
2026-09-15 10:30:50 -07:00
Matthew Jackson ae8e5ade3c fix(online): record decode POST reachability for the caller to classify
OnlineSource now records the HTTP outcome of its actual /decode POST — a
Status(code) for any answer (200/404/422/429/5xx) or Transport for a
connect/timeout/DNS/TLS failure — in a per-thread slot, readable via the new
take_last_decode_reachability() / DecodeReachability. This lets a caller
(autorip) classify a no-key result from the REAL decode instead of firing a
second, redundant probe against the POST-only endpoint. No change to key
resolution itself. Targets 1.7.2.
2026-09-14 14:55:12 -07:00
Matthew Jackson 8b914e7121 v1.7.1: bump version (unified release) 2026-09-14 03:34:52 +00:00
Matthew Jackson 62ebd61ff1 docs(changelog): 1.7.1 release notes (LibreDrive unlock fix) 2026-09-13 20:01:03 -07:00
30 changed files with 3261 additions and 2171 deletions
+15 -26
View File
@@ -2,10 +2,11 @@ version: 2
# Dependency updates land on `dev`, never on `main`.
#
# `main` here is a RELEASE POINTER that release.sh moves to each tag. A bot
# commit on it would put work there that no tag contains, which is exactly the
# state that aborted the 1.6.2 cascade at the last step -- so pointing
# Dependabot at main would recreate that failure on a schedule.
# `main` here is a RELEASE POINTER that freemkv's release-orchestrate.yml
# moves to each tag. A bot commit on it would put work there that no tag
# contains, which is exactly the state that aborted the 1.6.2 cascade at the
# last step -- so pointing Dependabot at main would recreate that failure on a
# schedule.
updates:
- package-ecosystem: cargo
directory: /
@@ -13,9 +14,7 @@ updates:
schedule:
interval: weekly
open-pull-requests-limit: 5
# One PR per week for the routine bumps instead of one per crate. Eight
# repos times a handful of crates is a volume nobody reads, and an
# unread PR queue is indistinguishable from no updates at all.
# One PR per week for the routine bumps instead of one per crate.
groups:
minor-and-patch:
update-types:
@@ -23,18 +22,14 @@ updates:
- patch
ignore:
# The freemkv crates depend on each other by GIT TAG, re-pinned by
# release.sh as part of the release commit. Dependabot cannot see that
# cascade, so a PR bumping one of these would fight the release process
# and could pin a version whose tag does not exist yet.
# release-orchestrate.yml in the release commit. A bot bump would fight
# the release cascade and could pin a tag that does not exist yet.
- dependency-name: freemkv-unlock
- dependency-name: libfreemkv
- dependency-name: freemkv-keysources
- dependency-name: freemkv-i18n
- dependency-name: freemkv-engine
# The workflows are now real infrastructure -- the release cascade, the
# cross-platform hash matrix, the disc gate -- so their actions need the same
# attention as the crates.
- package-ecosystem: github-actions
directory: /
target-branch: dev
@@ -47,17 +42,11 @@ updates:
- minor
- patch
ignore:
# NOT a dependency: `dtolnay/rust-toolchain` is versioned by the RUST
# release it installs, and the tag we pin is the toolchain CI is pinned
# to on purpose -- precommit.sh runs the same one locally so a lint that
# passes on a developer's newer default cannot pass CI by accident.
#
# Dependabot reads those tags as semver and proposed 1.97.0 -> 1.100.0,
# a Rust version that does not exist. Every such PR 404s on toolchain
# download across all eight repos, and they regenerate weekly -- eight
# permanently-red PRs that promote.yml then has to special-case when it
# decides whether dev is green.
#
# Bumping the toolchain is a deliberate, all-eight-repos change, made by
# hand together with precommit.sh. There is nothing here for a bot.
# Versioned by the Rust release it installs; CI picks the toolchain via
# `toolchain:` on @master/@stable. Dependabot proposes Rust versions that
# do not exist. Toolchain bumps are a deliberate all-repo change.
- dependency-name: dtolnay/rust-toolchain
# The org's own reusable workflows/actions. @v1 and @v2 are independent
# moving tags (v2 is only the comment guard), not semver releases: a
# "bump" from v1 to v2 would move a caller onto an OLDER guard.
- dependency-name: "freemkv/.github*"
+50 -130
View File
@@ -10,16 +10,22 @@ on:
branches: [main, dev, qa]
pull_request:
# Rapid dev pushes supersede their own in-progress runs instead of stacking.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
# WHY THE SIBLING CHECKOUTS — this is NOT a monorepo.
#
# The committed Cargo.toml resolves libfreemkv from a git TAG (it is off
# crates.io). That means CI was building this branch against the LAST RELEASE
# of libfreemkv, never against libfreemkv's dev tip — so a coordinated change
# across the two repos could not be verified until release.sh cut the tags.
# across the two repos could not be verified until release-orchestrate.yml cut the tags.
# Green here meant "compiles against last month's library".
#
# The fix mirrors the developer's local layout. Every repo is checked out into
# a subdirectory of $GITHUB_WORKSPACE, reproducing
# The fix mirrors the developer's local layout via the shared composite
# freemkv/.github//actions/checkout-siblings: every sibling repo is checked out
# into a subdirectory of $GITHUB_WORKSPACE, reproducing
# ~/Developer/freemkv/{freemkv-keysources,libfreemkv,freemkv-unlock}, and a
# generated .cargo/config.toml redirects the git-tag patch to those paths.
# Config-level [patch.crates-io] wins over the manifest one, which is the same
@@ -36,7 +42,7 @@ on:
# the `deny` job audits the resolved graph, but CI's sibling path-patch layout
# resolves the freemkv crates as path packages, which `--locked` can't satisfy —
# the same reason the binary crates omit it. The lock that has to hold is the one
# release.sh regenerates in each consuming binary.
# release-orchestrate.yml regenerates in each consuming binary.
jobs:
# Single source of truth for the MSRV — read rust-version from Cargo.toml once
@@ -47,7 +53,7 @@ jobs:
outputs:
v: ${{ steps.msrv.outputs.v }}
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
@@ -61,7 +67,7 @@ jobs:
changelog:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
- name: Require a CHANGELOG entry for the crate version on main
shell: bash
run: |
@@ -82,28 +88,14 @@ jobs:
needs: resolve-msrv
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
with:
path: freemkv-keysources
- uses: actions/checkout@v7
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
repository: freemkv/libfreemkv
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- name: Resolve siblings from their dev tips
shell: bash
working-directory: freemkv-keysources
run: |
mkdir -p .cargo
cat > .cargo/config.toml <<'EOF'
[patch.crates-io]
libfreemkv = { path = "../libfreemkv" }
EOF
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
# fmt runs on the MSRV pin; clippy runs on stable so new-lint debt is a
# blocking gate the pinned toolchain would hide. Stable is installed first
# for clippy; the MSRV pin goes second so it stays fmt's default toolchain.
@@ -127,28 +119,14 @@ jobs:
needs: resolve-msrv
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
with:
path: freemkv-keysources
- uses: actions/checkout@v7
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
repository: freemkv/libfreemkv
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- name: Resolve siblings from their dev tips
shell: bash
working-directory: freemkv-keysources
run: |
mkdir -p .cargo
cat > .cargo/config.toml <<'EOF'
[patch.crates-io]
libfreemkv = { path = "../libfreemkv" }
EOF
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
@@ -165,33 +143,19 @@ jobs:
deny:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
with:
path: freemkv-keysources
- uses: actions/checkout@v7
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
repository: freemkv/libfreemkv
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- name: Resolve siblings from their dev tips
shell: bash
working-directory: freemkv-keysources
run: |
mkdir -p .cargo
cat > .cargo/config.toml <<'EOF'
[patch.crates-io]
libfreemkv = { path = "../libfreemkv" }
EOF
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@stable
- uses: EmbarkStudios/cargo-deny-action@v2
with:
command: check advisories bans licenses sources
manifest-path: freemkv-keysources/Cargo.toml
- uses: taiki-e/install-action@cargo-deny
# Run from the repo dir so the generated .cargo/config.toml path patches apply.
- run: cargo deny --all-features check advisories bans licenses sources
working-directory: freemkv-keysources
# This crate ships wherever freemkv and autorip ship, and it drags in all of
# libfreemkv's platform code (SCSI transports included). Build the tests, not
@@ -206,35 +170,20 @@ jobs:
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
# than a slow one: a never-scheduled job leaves the run pending forever, so
# the branch's required checks never report complete.
needs: resolve-msrv
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: macos-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
with:
path: freemkv-keysources
- uses: actions/checkout@v7
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
repository: freemkv/libfreemkv
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- name: Resolve siblings from their dev tips
shell: bash
working-directory: freemkv-keysources
run: |
mkdir -p .cargo
cat > .cargo/config.toml <<'EOF'
[patch.crates-io]
libfreemkv = { path = "../libfreemkv" }
EOF
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
@@ -254,28 +203,14 @@ jobs:
needs: resolve-msrv
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
with:
path: freemkv-keysources
- uses: actions/checkout@v7
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
repository: freemkv/libfreemkv
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- name: Resolve siblings from their dev tips
shell: bash
working-directory: freemkv-keysources
run: |
mkdir -p .cargo
cat > .cargo/config.toml <<'EOF'
[patch.crates-io]
libfreemkv = { path = "../libfreemkv" }
EOF
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
@@ -303,35 +238,20 @@ jobs:
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
# than a slow one: a never-scheduled job leaves the run pending forever, so
# the branch's required checks never report complete.
needs: resolve-msrv
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: windows-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
with:
path: freemkv-keysources
- uses: actions/checkout@v7
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
repository: freemkv/libfreemkv
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- name: Resolve siblings from their dev tips
shell: bash
working-directory: freemkv-keysources
run: |
mkdir -p .cargo
cat > .cargo/config.toml <<'EOF'
[patch.crates-io]
libfreemkv = { path = "../libfreemkv" }
EOF
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
+7 -12
View File
@@ -1,17 +1,12 @@
name: comment-guard
# Caps explanatory prose by COMMENT INTENT (not visibility):
# //! / /*! module docs — unbounded (module-scoped, can't hide per-item text)
# /// / /** */ item docs — 8 prose lines (fenced ``` examples excluded)
# // / /* */ asides — 3 lines (single-blank-separated runs count as one)
# Overflow belongs in the module //! or docs/<topic>.md with a one-line // pointer.
# Thin caller → the canonical reusable guard in freemkv/.github (one definition
# for every repo). Fork PRs still run it; same-repo pushes aren't double-run.
on: [push, pull_request]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
comment-guard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Run comment-guard
run: python3 ci/comment-guard.py
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork
uses: freemkv/.github/.github/workflows/guard-comment.yml@v2
+7 -34
View File
@@ -1,41 +1,14 @@
name: identity-guard
# Prevention: no commit may carry an unrelated-business or non-canonical author/
# committer email. dev@example.com (a dead corp address) once slipped onto the
# release cascade + a batch of local commits; this fails the build if it recurs.
# Thin caller → the canonical reusable guard in freemkv/.github.
on:
push:
branches: [dev, qa, main]
pull_request:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
identity:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Reject banned author/committer emails
env:
BEFORE: ${{ github.event.before }}
AFTER: ${{ github.sha }}
EVENT: ${{ github.event_name }}
PR_BASE: ${{ github.event.pull_request.base.sha }}
PR_HEAD: ${{ github.event.pull_request.head.sha }}
run: |
set -euo pipefail
# Unrelated business domains + a personal address normalized to the
# GitHub noreply. Real external contributors are unaffected.
BANNED='@example-a\.invalid|@example-b\.invalid|owner@example\.invalid'
if [ "$EVENT" = "pull_request" ]; then
range="$PR_BASE..$PR_HEAD"
elif [ -n "${BEFORE:-}" ] && [ "$BEFORE" != "0000000000000000000000000000000000000000" ] && git cat-file -e "$BEFORE" 2>/dev/null; then
range="$BEFORE..$AFTER"
else
range="$AFTER~1..$AFTER"
fi
bad=$(git log --format='%an <%ae>%n%cn <%ce>' "$range" 2>/dev/null | sort -u | grep -iE "$BANNED" || true)
if [ -n "$bad" ]; then
echo "::error::commit(s) carry a banned author/committer identity:"; echo "$bad" | sed 's/^/ /'
echo "::error::Commit as your GitHub noreply (1085847+MattJackson@users.noreply.github.com). Fix: git config user.email <noreply>; git commit --amend --reset-author."
exit 1
fi
echo "identity-guard: clean"
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork
uses: freemkv/.github/.github/workflows/guard-identity.yml@v1
+6 -30
View File
@@ -1,35 +1,11 @@
name: leak-guard
# Self-contained public-repo leak gate. Public CI cannot reach the private
# tooling, so this encodes only the generic net: internal-infra references,
# tracked CLAUDE.md/.claude paths, and AI-attribution in commit messages.
# No project-specific reverse-engineering vocabulary lives here.
# Thin caller → the canonical reusable guard in freemkv/.github.
on: [push, pull_request]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
leak-guard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Compute commit range
id: range
run: |
if [ "${{ github.event_name }}" = "pull_request" ]; then
base="${{ github.event.pull_request.base.sha }}"
head="${{ github.event.pull_request.head.sha }}"
echo "range=$base..$head" >> "$GITHUB_OUTPUT"
else
before="${{ github.event.before }}"
after="${{ github.sha }}"
# New branch / first push: github.event.before is all-zeros.
if [ -z "$before" ] || [ "$before" = "0000000000000000000000000000000000000000" ]; then
echo "range=$after" >> "$GITHUB_OUTPUT"
else
echo "range=$before..$after" >> "$GITHUB_OUTPUT"
fi
fi
- name: Run leak-guard
run: bash ci/leak-guard.sh "${{ steps.range.outputs.range }}"
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork
uses: freemkv/.github/.github/workflows/guard-leak.yml@v1
+25 -62
View File
@@ -13,16 +13,21 @@ name: qa
# Sibling repos are checked out at `qa`, NOT `dev`. A qa run that resolved its
# dependencies from dev tips would be validating a combination that is not the
# one being released, which is the exact failure this branch exists to prevent.
# The shared checkout-siblings composite resolves qa->qa from the caller branch.
#
# What this gate CANNOT cover: `disc://` and real `iso://` need physical media,
# and no hosted runner has an optical drive or the image hoard. Those run on a
# self-hosted runner (see the media job at the end) and are the one leg that
# stays on hardware.
# and no hosted runner has an optical drive or the image hoard. That coverage
# lives in freemkv's qa.yml (the real-media EC2 matrix), not in this file.
on:
push:
branches: [qa]
workflow_dispatch:
# Supersede an in-progress qa run when a newer candidate lands on the branch.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
# Single source of truth for the MSRV — see ci.yml. Jobs that build/test AT
# the MSRV read needs.resolve-msrv.outputs.v from Cargo.toml's rust-version;
@@ -32,7 +37,7 @@ jobs:
outputs:
v: ${{ steps.msrv.outputs.v }}
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
@@ -56,7 +61,7 @@ jobs:
permissions:
contents: write
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Stamp the next rc
@@ -85,28 +90,14 @@ jobs:
os: ['ubuntu-latest', 'macos-latest']
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
with:
path: freemkv-keysources
- uses: actions/checkout@v7
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
repository: freemkv/libfreemkv
ref: qa
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- name: Resolve siblings from their qa tips
shell: bash
working-directory: freemkv-keysources
run: |
mkdir -p .cargo
cat > .cargo/config.toml <<'EOF'
[patch.crates-io]
libfreemkv = { path = "../libfreemkv" }
EOF
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
@@ -122,28 +113,14 @@ jobs:
cross-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
with:
path: freemkv-keysources
- uses: actions/checkout@v7
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
repository: freemkv/libfreemkv
ref: qa
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- name: Resolve siblings from their qa tips
shell: bash
working-directory: freemkv-keysources
run: |
mkdir -p .cargo
cat > .cargo/config.toml <<'EOF'
[patch.crates-io]
libfreemkv = { path = "../libfreemkv" }
EOF
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
@@ -162,28 +139,14 @@ jobs:
needs: resolve-msrv
runs-on: windows-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
with:
path: freemkv-keysources
- uses: actions/checkout@v7
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
repository: freemkv/libfreemkv
ref: qa
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- name: Resolve siblings from their qa tips
shell: bash
working-directory: freemkv-keysources
run: |
mkdir -p .cargo
cat > .cargo/config.toml <<'EOF'
[patch.crates-io]
libfreemkv = { path = "../libfreemkv" }
EOF
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
+9 -11
View File
@@ -17,7 +17,7 @@ jobs:
verify:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
- name: Verify Cargo.toml version matches tag
run: |
CARGO_VER="v$(grep '^version' Cargo.toml | head -1 | sed 's/.*"\(.*\)"/\1/')"
@@ -29,19 +29,19 @@ jobs:
# Tests run as a PARALLEL TRIPWIRE: they fail the run if they fail, but the
# release job does NOT `needs:` this job. The tag decision was already gated by
# the local precommit (same MSRV, same commit).
# CI (ci.yml on dev, then a green qa.yml on the same commit).
#
# Cargo.lock is now committed (issue #15), but --locked is still omitted: on
# the runner the libfreemkv dep resolves from crates.io, not the local path
# patch, which --locked can't reconcile. Its
# libfreemkv dep resolves from crates.io (the dev .cargo/config.toml patch is
# gitignored and absent on the runner), so libfreemkv must already be published
# at this version — the release script enforces that ordering before tagging.
# at this version — release-orchestrate.yml enforces that ordering before tagging.
test:
needs: verify
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
@@ -51,12 +51,10 @@ jobs:
- uses: Swatinem/rust-cache@v2
- run: cargo test
# NOTE: the crates.io publish is driven by the maintainer's release tooling
# from local credentials, not from CI — this repo has no CARGO_REGISTRY_TOKEN
# secret. release.sh runs `cargo publish --no-verify` locally. CI here only
# verifies the version/tag match, runs tests (tripwire), and cuts the GitHub
# Release. The binaries git-tag-pin keysources, so they don't wait on the
# crates.io publish — it serves external consumers in parallel.
# NOTE: there is no crates.io publish — keysources is `publish = false` and
# git-tag-only. Consumers git-tag-pin it via a committed [patch.crates-io], so
# the git tag itself is the release artifact. CI here only verifies the
# version/tag match, runs tests (tripwire), and cuts the GitHub Release.
release:
# Only needs `verify`; the GitHub Release is cut as soon as the version
@@ -64,7 +62,7 @@ jobs:
needs: verify
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
- name: Create GitHub Release
uses: softprops/action-gh-release@v3
with:
+66
View File
@@ -1,5 +1,71 @@
# Changelog
## [Unreleased]
### Fixed
- The online key service ignores `HTTP_PROXY`/`HTTPS_PROXY`/`ALL_PROXY`: with one set, every key lookup failed (the pinned resolver also answered the proxy's address, so the proxy request went to the key service).
### Changed
- `KeydbSource` hands out host certs best first, with no disc input: certs with no `Revoked in MKBv<N>` note in keydb order, then revoked ones by latest revocation. The inherent `host_certs()` and the `KeySource::host_certs` trait method both return this order; the trait ignores its `mkb` argument.
### Deprecated
- `KeyDb::host_certs(mkb)`: use `KeyDb::host_certs_ranked`.
### Added
- `KeyDb::host_certs_ranked`.
- `check_keyserver_url` with `KeyserverUrlRejection`/`KeyserverUrlFault`: a typed Permanent vs Temporary verdict, so callers need not match `validate_keyserver_url`'s text.
- `test-hooks` feature exposing `set_last_decode_reachability`, to test decode-verdict handling without a network.
## [1.7.7] — 2026-09-26
### Maintenance
- Replace comment-overflow documentation with concise source contracts and README instructions; enforce the shared comment policy in CI.
## [1.7.6] — 2026-09-26
### Changed
- Version aligned to 1.7.6 for the unified release. No functional changes to this crate; the release is driven by the freemkv 1.7.6 Linux desktop shell (GTK4 + libadwaita) and the rip-finished desktop notification (issue #56).
## [1.7.5] — 2026-09-23
### Changed
- Version aligned to 1.7.5 for the unified release. No functional changes to this crate; the release is driven by freemkv-unlock mirroring the freemkv-firmware 0.9.0 ABI (the drive's `Ake` and `Bus` levers retired into a single `Encryption` lever).
## [1.7.4] — 2026-09-21
### Fixed
- keydb lookups now report match state and entry shape, so a "no entry" result tells you whether the disc was truly absent or matched-but-underivable (no usable VID) (#46).
- ureq 3.4.2 with a bounded, rolling-idle reply body, so a slow-but-progressing download is no longer killed by an absolute deadline.
### Maintenance
- CI moved to the central reusable workflows.
## [1.7.3] — 2026-09-19
### Changed
- Unified release with freemkv 1.7.3 (`info --share` disc-structure capture for bug reports + the multi-angle UHD main-title fix in libfreemkv; see the libfreemkv/freemkv 1.7.3 notes). No functional changes to this crate.
## [1.7.2] — UNRELEASED
### Added
- `OnlineSource` now records the reachability outcome of its actual `/decode` POST (an HTTP `Status(code)` for any answer, including 200/404/422/429/5xx, or `Transport` for a connect/timeout/DNS/TLS failure) in a per-thread slot, readable via the new `take_last_decode_reachability()` / `DecodeReachability`. This lets a caller (autorip) classify a no-key result from the REAL decode's outcome instead of firing a second, redundant probe against the POST-only endpoint. No behavioural change to key resolution itself.
## [1.7.1] — UNRELEASED
### Changed
- Version aligned to 1.7.1 for the unified release (freemkv-unlock 1.7.1 LibreDrive unlock fix); no functional changes.
## [1.7.0] — 2026-09-02
### Changed
Generated
+101 -68
View File
@@ -16,7 +16,7 @@ checksum = "f8eb277bec05f56a0e0591f155a484cbd0f4f07ff2905051a48c72f004f7ed58"
dependencies = [
"cipher",
"cpubits",
"cpufeatures 0.3.1",
"cpufeatures",
]
[[package]]
@@ -32,13 +32,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5"
[[package]]
name = "block-buffer"
version = "0.10.4"
name = "bitflags"
version = "2.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71"
dependencies = [
"generic-array",
]
checksum = "3ded4057c258ba199e2d26386d3af3780957ecaee6c4ef4041c6b4b8b97c0b06"
[[package]]
name = "block-buffer"
@@ -84,7 +81,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "65c35e4b699c7e15ccbe7ee35c005e4fc0a278d22238a2857e6ce2dadeda1b06"
dependencies = [
"cfg-if",
"cpufeatures 0.3.1",
"cpufeatures",
"rand_core",
]
@@ -94,7 +91,7 @@ version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8cf2a2c93cd704877c0858356ed03480ff301ee950b43f1cbe4573b088bfa6c"
dependencies = [
"crypto-common 0.2.2",
"crypto-common",
"inout",
]
@@ -139,15 +136,6 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "15b85f9c39137c3a891689859392b1bd49812121d0d61c9caf00d46ed5ce06ae"
[[package]]
name = "cpufeatures"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280"
dependencies = [
"libc",
]
[[package]]
name = "cpufeatures"
version = "0.3.1"
@@ -200,16 +188,6 @@ version = "0.8.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17"
[[package]]
name = "crypto-common"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
dependencies = [
"generic-array",
"typenum",
]
[[package]]
name = "crypto-common"
version = "0.2.2"
@@ -225,25 +203,15 @@ version = "0.5.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7cd812cc2bc1d69d4764bd80df88b4317eaef9e773c75226407d9bc0876b211c"
[[package]]
name = "digest"
version = "0.10.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
dependencies = [
"block-buffer 0.10.4",
"crypto-common 0.1.7",
]
[[package]]
name = "digest"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2"
dependencies = [
"block-buffer 0.12.1",
"block-buffer",
"const-oid",
"crypto-common 0.2.2",
"crypto-common",
]
[[package]]
@@ -278,6 +246,16 @@ version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "errno"
version = "0.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
dependencies = [
"libc",
"windows-sys 0.52.0",
]
[[package]]
name = "find-msvc-tools"
version = "0.1.11"
@@ -306,7 +284,7 @@ dependencies = [
[[package]]
name = "freemkv-keysources"
version = "1.7.1"
version = "1.7.7"
dependencies = [
"base64",
"flate2",
@@ -319,7 +297,7 @@ dependencies = [
[[package]]
name = "freemkv-unlock"
version = "1.7.1"
version = "1.7.7"
dependencies = [
"aes",
"base64",
@@ -331,16 +309,7 @@ dependencies = [
"sha1",
"sha2",
"tracing",
]
[[package]]
name = "generic-array"
version = "0.14.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a"
dependencies = [
"typenum",
"version_check",
"zeroize",
]
[[package]]
@@ -534,7 +503,7 @@ checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
[[package]]
name = "libfreemkv"
version = "1.7.1"
version = "1.7.7"
dependencies = [
"aes",
"base64",
@@ -542,16 +511,24 @@ dependencies = [
"freemkv-unlock",
"libc",
"memchr",
"rand",
"rayon",
"roxmltree",
"rustix",
"serde",
"serde_json",
"sha1",
"sha2",
"socket2",
"tracing",
"zip",
]
[[package]]
name = "linux-raw-sys"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
[[package]]
name = "litemap"
version = "0.8.3"
@@ -725,7 +702,7 @@ dependencies = [
"getrandom 0.2.17",
"libc",
"untrusted",
"windows-sys",
"windows-sys 0.52.0",
]
[[package]]
@@ -735,10 +712,23 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c20b6793b5c2fa6553b250154b78d6d0db37e72700ae35fad9387a46f487c97"
[[package]]
name = "rustls"
version = "0.23.43"
name = "rustix"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0283386ce02abc0151e1761d08802dfe86c173b0b494af5cbc086574e453da06"
checksum = "891efababe418670775f199f0d233d84843c227a0949a883ce15b37c78d6629d"
dependencies = [
"bitflags",
"errno",
"libc",
"linux-raw-sys",
"windows-sys 0.52.0",
]
[[package]]
name = "rustls"
version = "0.23.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0d41d731c7d2f962d1ccc364cec258de3c0e93b38c2fb3ba97ac74513048d634"
dependencies = [
"log",
"once_cell",
@@ -814,13 +804,13 @@ dependencies = [
[[package]]
name = "sha1"
version = "0.10.7"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a978451301f4db1d02937a4ab3ccce137717b81826e79b7d49ffe3244a13c3b8"
checksum = "aacc4cc499359472b4abe1bf11d0b12e688af9a805fa5e3016f9a386dc2d0214"
dependencies = [
"cfg-if",
"cpufeatures 0.2.17",
"digest 0.10.7",
"cpufeatures",
"digest",
]
[[package]]
@@ -830,8 +820,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4"
dependencies = [
"cfg-if",
"cpufeatures 0.3.1",
"digest 0.11.3",
"cpufeatures",
"digest",
]
[[package]]
@@ -852,6 +842,16 @@ version = "1.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9be42f50aa861c555654aa3a37f52f4b1074bacf4e48fe0ef7fa584e80f1f0f"
[[package]]
name = "socket2"
version = "0.6.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4"
dependencies = [
"libc",
"windows-sys 0.61.2",
]
[[package]]
name = "stable_deref_trait"
version = "1.2.1"
@@ -994,9 +994,9 @@ checksum = "8ecb6da28b8a351d773b68d5825ac39017e680750f980f3a1a85cd8dd28a47c1"
[[package]]
name = "ureq"
version = "3.4.1"
version = "3.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "af5546be8f5378d5414f83733f5c9a2526f4645829edbc1c41790aeef1b38e8b"
checksum = "9a7ac20be9b7726e0bbdbf974c059676d9acb1cd414961f570a4e8231cacd7fc"
dependencies = [
"base64",
"cookie_store",
@@ -1069,6 +1069,12 @@ dependencies = [
"rustls-pki-types",
]
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-sys"
version = "0.52.0"
@@ -1078,6 +1084,15 @@ dependencies = [
"windows-targets",
]
[[package]]
name = "windows-sys"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
dependencies = [
"windows-link",
]
[[package]]
name = "windows-targets"
version = "0.52.6"
@@ -1197,6 +1212,20 @@ name = "zeroize"
version = "1.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e13c156562582aa81c60cb29407084cdb54c4164760106ab78e6c5b0858cf64e"
dependencies = [
"zeroize_derive",
]
[[package]]
name = "zeroize_derive"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3c50655cbb0fe3fc43170059e702f1ce5e19b84cec58dc87b037a09935c2f328"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "zerotrie"
@@ -1271,8 +1300,12 @@ dependencies = [
[[patch.unused]]
name = "freemkv-engine"
version = "1.7.1"
version = "1.7.7"
[[patch.unused]]
name = "freemkv-i18n"
version = "1.7.1"
version = "1.7.7"
[[patch.unused]]
name = "freemkv-keysources"
version = "1.7.7"
+13 -3
View File
@@ -1,6 +1,6 @@
[package]
name = "freemkv-keysources"
version = "1.7.1"
version = "1.7.7"
edition = "2024"
rust-version = "1.88"
license = "MIT"
@@ -17,7 +17,7 @@ publish = false
# The crate provides the `KeySource` trait + `Key`/`DiscInputs` types these impls fill.
# libfreemkv is git-only now; the committed [patch.crates-io] below redirects this
# bare version req to the libfreemkv git tag (local dev overrides it to a path).
libfreemkv = "1.6"
libfreemkv = "1.7"
# OnlineSource: POST disc inputs + samples to a key service over HTTP.
ureq = { version = "3", features = ["json"] }
serde_json = "1"
@@ -30,6 +30,15 @@ zip = { version = "8", default-features = false, features = ["deflate"] }
# key material).
tracing = "0.1"
[features]
# `set_last_decode_reachability`, for consumers' tests only (enable in dev-dependencies).
test-hooks = []
[dev-dependencies]
# `test_util` fixtures (`decrypt_unit`, …) for this crate's tests (keys-upfront
# KU-P1). CI resolves it through the sibling path patch.
libfreemkv = { version = "1.7", features = ["test-util"] }
[profile.release]
lto = "thin"
codegen-units = 1
@@ -40,7 +49,8 @@ codegen-units = 1
# overrides it with a path patch via the gitignored .cargo/config.toml (a
# config-level [patch.crates-io] wins over this manifest one for the same crate).
[patch.crates-io]
libfreemkv = { git = "https://github.com/freemkv/libfreemkv", tag = "v1.6.14" }
libfreemkv = { git = "https://github.com/freemkv/libfreemkv", tag = "v1.7.7" }
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.7.7" }
[lints.clippy]
uninlined_format_args = "allow"
+7
View File
@@ -9,6 +9,13 @@ Pluggable AACS key sources (keydb, online key service) for
and hands libfreemkv its terminal Unit Keys via `get_unit_keys`; the library
does all derivation.
## Development
Build API documentation with `cargo doc --no-deps --open` for source construction,
keydb paths and online-query contracts. Run `cargo test --tests` to validate
parsing, caching and request guards. Tests use synthetic keys and local fixtures;
never commit real key material or credentials.
## License
MIT — see [LICENSE](LICENSE).
-252
View File
@@ -1,252 +0,0 @@
#!/usr/bin/env python3
"""comment-guard: cap explanatory prose by COMMENT INTENT, not visibility.
Three tiers, keyed on the syntax the author reached for:
* `//!` / `/*!` — a module (or crate) design doc. UNBOUNDED. There is one per
file, at the top, documenting the module as a whole, so it can't be used to
smuggle per-item rationale the way item docs can. Long correctness / design
essays (e.g. a memory-ordering proof) legitimately live here.
* `///` / `/** */` — item documentation. Up to DOC_CAP *prose* lines. Fenced
``` code blocks do NOT count, so real examples aren't punished. This holds
whether or not the item is `pub`: a `pub(crate)` fn deserves a real doc
paragraph just like a `pub` one. Capping `///` here (rather than exempting
it, as the old guard did — issue #17) is what stops rationale being hidden
behind a doc sigil, while still leaving room for a genuine paragraph.
* `//` / `/* */` — a quick internal aside. Up to INLINE_CAP lines. This is the
shape that sprawls into walls, so it's kept tight: stanzas separated by a
single blank line count CUMULATIVELY, so one essay can't be split into runs.
Overflow belongs in the module `//!`, or in docs/<topic>.md with a one-line `//`
pointer — both show up in review.
Scans src/ and tests/ under the given root (default cwd). `--selftest` runs
built-in cases. Exit non-zero if any block exceeds its cap.
"""
import re
import sys
from pathlib import Path
INLINE_CAP = 3
DOC_CAP = 8
_RAW_OPEN = re.compile(r'r(#*)"') # raw string opener: r"…", r#"…"#, br##"…"## …
def _strip_line_comment(s):
"""Return (kind, body) for a whole-line comment, else (None, None).
kind is 'doc' for `///`/`//!`, 'plain' for other `//`. A trailing comment on
a code line is not a whole-line comment and is ignored (single-line, cheap).
"""
if s.startswith("//!"):
return "doc", s[3:]
if s.startswith("///") and not s.startswith("////"):
return "doc", s[3:]
if s.startswith("//"):
return "plain", s[2:]
return None, None
def _fence_toggle(body):
"""True if this doc body line opens/closes a ``` fenced code block."""
return body.strip().startswith("```")
def _classify(lines):
"""Yield per-line dicts: comment kind ('doc'/'plain'/None), doc-ness of
block comments, and code info. Block comments are tracked across lines.
"""
info = []
in_block = False
block_is_doc = False
raw_hashes = None # None = not in a raw string; int N = open, needs '"' + N*'#'
for raw in lines:
s = raw.lstrip()
rec = {"raw": raw, "s": s, "comment": None, "is_doc": False, "code": False}
# Inside a raw string literal (e.g. an embedded HTML/CSS/JS asset): the
# bytes are string data, not Rust comments, so the guard is blind to them
# until the string closes. Prevents `/* */` or `//` lines in an embedded
# page from being counted as comment blocks.
if raw_hashes is not None:
if ('"' + "#" * raw_hashes) in raw:
raw_hashes = None
info.append(rec)
continue
if in_block:
rec["comment"] = "doc" if block_is_doc else "plain"
rec["is_doc"] = block_is_doc
if "*/" in raw:
in_block = False
info.append(rec)
continue
# A block comment only starts a comment-line if it's the first token.
if s.startswith("/*"):
rec["is_doc"] = s.startswith("/**") or s.startswith("/*!")
rec["comment"] = "doc" if rec["is_doc"] else "plain"
if "*/" not in s[2:]:
in_block = True
block_is_doc = rec["is_doc"]
info.append(rec)
continue
kind, _ = _strip_line_comment(s)
if kind is not None:
rec["comment"] = kind
rec["is_doc"] = kind == "doc"
info.append(rec)
continue
# Code (or blank). Blank = empty after strip.
rec["code"] = s != ""
# A code line may OPEN a raw string that spans following lines. Detected
# only on code (comments were handled above), so `// see r"x"` is safe.
m = _RAW_OPEN.search(raw)
if m and ('"' + "#" * len(m.group(1))) not in raw[m.end():]:
raw_hashes = len(m.group(1))
info.append(rec)
return info
def violations(path):
lines = path.read_text(encoding="utf-8", errors="replace").splitlines()
info = _classify(lines)
n = len(info)
out = []
i = 0
while i < n:
rec = info[i]
if rec["comment"] is None:
i += 1
continue
start = i
if rec["is_doc"]:
# Doc run: contiguous doc lines. `//!` / `/*!` module docs are
# UNBOUNDED; any other doc run caps PROSE (fenced ``` excluded) at
# DOC_CAP, regardless of the documented item's visibility.
j = i
prose = 0
in_fence = False
module_doc = False
while j < n and info[j]["comment"] == "doc":
s = info[j]["s"]
if s.startswith("//!") or s.startswith("/*!"):
module_doc = True
body = (
s[3:] if (s.startswith("///") or s.startswith("//!")) else s
)
if _fence_toggle(body):
in_fence = not in_fence
elif not in_fence and body.strip() != "":
prose += 1
j += 1
if not module_doc and prose > DOC_CAP:
out.append((start + 1, prose, DOC_CAP, "doc prose"))
i = j
continue
# Internal (plain) run, with single-blank bridging (cumulative).
j = i
count = 0
while j < n:
r = info[j]
if r["comment"] == "plain":
count += 1
j += 1
continue
if r["comment"] == "doc":
break
if r["comment"] is None and not r["code"]:
# blank: bridge only a SINGLE blank, else stop
if j + 1 < n and info[j + 1]["comment"] == "plain":
j += 1
continue
break
break # code
if count > INLINE_CAP:
out.append((start + 1, count, INLINE_CAP, "inline comment"))
i = j
return out
def main(argv):
if "--selftest" in argv:
return _selftest()
root = Path(argv[1]) if len(argv) > 1 else Path.cwd()
files = sorted(p for d in ("src", "tests") for p in (root / d).rglob("*.rs"))
total = 0
for f in files:
for start, length, cap, kind in violations(f):
rel = f.relative_to(root)
print(f"{rel}:{start}: {kind} block is {length} lines (max {cap})")
total += 1
if total:
print(f"\ncomment-guard: {total} comment block(s) exceed their cap.")
print(
f"Inline // asides cap at {INLINE_CAP}; /// item-doc prose caps at "
f"{DOC_CAP} (fenced examples excluded); //! module docs are unbounded. "
"Trim, move the rationale into the module //!, or docs/<topic>.md "
"with a // pointer."
)
return 1
return 0
def _selftest():
import tempfile
cases = [
# (rust source, expected number of violations)
# short /// doc -> ok
("/// short doc\npub fn a() {}\n", 0),
# /// on a PRIVATE item now gets the doc cap (8), so 4 lines is fine
("/// l1\n/// l2\n/// l3\n/// l4\nfn priv_a() {}\n", 0),
# ...but /// is NOT unlimited: 9 prose lines on a private item fails
("".join(f"/// l{k}\n" for k in range(9)) + "fn priv_b() {}\n", 1),
# 8 prose /// on a pub item -> ok (boundary)
("".join(f"/// l{k}\n" for k in range(8)) + "pub fn c() {}\n", 0),
# 9 prose /// on a pub item -> violation
("".join(f"/// l{k}\n" for k in range(9)) + "pub fn d() {}\n", 1),
# /// with a long fenced example -> example excluded, ok
(
"/// summary\n/// # Examples\n/// ```\n"
+ "".join("/// code\n" for _ in range(12))
+ "/// ```\npub fn e() {}\n",
0,
),
# pub(crate) gets the SAME doc cap as pub (no pub/private split anymore)
("".join(f"/// l{k}\n" for k in range(6)) + "pub(crate) fn f() {}\n", 0),
# bare inline 4 lines -> violation
("// a\n// b\n// c\n// d\nlet x = 1;\n", 1),
# blank-split dodge: 3 + blank + 3 inline -> cumulative -> violation
("// a\n// b\n// c\n\n// d\n// e\n// f\nfn z() {}\n", 1),
# plain /* */ block, 5 lines -> inline cap -> violation
("/* one\n two\n three\n four\n five */\nfn q() {}\n", 1),
# comment-like lines INSIDE a raw string are string data, not comments
(
'const H: &str = r##"<style>\n'
+ "".join("/* css note */\n" for _ in range(8))
+ '</style>"##;\nfn h() {}\n',
0,
),
# //! module doc may run long -> EXEMPT (no cap)
("".join(f"//! l{k}\n" for k in range(40)) + "\npub fn m() {}\n", 0),
# /*! block module doc may run long -> EXEMPT
("/*! l0\n" + "".join(f" l{k}\n" for k in range(30)) + "*/\nfn n() {}\n", 0),
]
ok = True
with tempfile.TemporaryDirectory() as d:
for idx, (src, want) in enumerate(cases):
p = Path(d) / f"case{idx}.rs"
p.write_text(src)
got = len(violations(p))
flag = "ok" if got == want else "FAIL"
if got != want:
ok = False
print(f" selftest {idx}: want {want} got {got} [{flag}]")
print("selftest PASSED" if ok else "selftest FAILED")
return 0 if ok else 1
if __name__ == "__main__":
sys.exit(main(sys.argv))
-120
View File
@@ -1,120 +0,0 @@
#!/usr/bin/env bash
#
# leak-guard.sh — self-contained public-repo leak gate.
#
# This is the LAST line of defense in CI. It is intentionally self-contained:
# public CI cannot reach the private tooling, so this script encodes ONLY the
# generic net — internal infrastructure references, agent-context files, and
# AI-attribution in commit messages. It deliberately contains NO project-
# specific reverse-engineering vocabulary (those words would themselves be a
# leak). The richer private scanner stays private.
#
# Fails (exit 1) if any of the following appear in the repo:
# 1. a tracked CLAUDE.md or .claude/ path (agent context — never public),
# 2. tracked file content matching the internal-infra net,
# 3. a commit message (in the given range) with AI attribution.
#
# Usage:
# leak-guard.sh [<commit-range>]
# <commit-range> optional git rev-list range to scan commit messages
# (e.g. "abc..def"). If omitted, commit-message scan is
# skipped (path + content checks always run).
set -euo pipefail
# Absolute path to this script, resolved before any cd, so we can exclude it
# from the content scan (it necessarily contains the detection patterns).
SELF_ABS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/$(basename "${BASH_SOURCE[0]}")"
REPO="$(git rev-parse --show-toplevel)"
cd "$REPO"
fail=0
note() { printf ' ✗ %s\n' "$1"; fail=1; }
# Internal-infra net — GENERIC ONLY. This script ships in the public repo, so
# the patterns themselves must not name any org-specific identifier (doing so
# would itself leak the infra they guard). We catch the leak *class*:
# - RFC1918 private IPv4 ranges (10/8, 172.16/12, 192.168/16),
# - private/internal/non-routable TLDs (.internal/.local/.lan/.corp/.invalid),
# - docker.internal.
# The full org-specific net (literal hostnames, service names, repo paths,
# vendor tooling, …) lives ONLY in the private scanner and never ships here.
INFRA_RE='\b10\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}|\b172\.(1[6-9]|2[0-9]|3[01])\.[0-9]{1,3}\.[0-9]{1,3}|\b192\.168\.[0-9]{1,3}\.[0-9]{1,3}|\.internal\b|\.local\b|\.lan\b|\.corp\b|\.invalid\b|docker\.internal'
# Home-path net — GENERIC ONLY. Catches an absolute developer home path
# committed into a tracked file (a macOS /Users/<user>/… or Linux /home/<user>/…
# path). This names NO specific user — it matches the leak *class* (any home
# path), so the pattern itself reveals nothing org- or person-specific. A real
# leak (e.g. /Users/alice/Developer/x slipping into a public RELEASE.md) trips
# this regardless of whose machine it came from. The username segment is a
# literal-username class ([A-Za-z0-9._-]) so dynamic/templated paths that build
# the user at runtime — shell `/home/$USER/`, doc `/home/<rip>/`, Rust
# `/home/{user}/` — do NOT false-positive; only a baked-in literal home leaks.
HOMEPATH_RE='/Users/[A-Za-z0-9._-]+/|/home/[A-Za-z0-9._-]+/'
# AI-attribution net (case-insensitive). "claude" matches only as a standalone
# word — NOT preceded by a dot/slash/alnum and NOT followed by .md — so legit
# mentions of CLAUDE.md / .claude/ in a commit message don't false-positive.
ATTR_RE='co-authored-by|generated with|🤖|(?<![.\/A-Za-z0-9])claude(?!\.md)'
echo "── leak-guard: tracked agent-context paths ──"
while IFS= read -r f; do
case "$f" in
CLAUDE.md|*/CLAUDE.md|.claude|.claude/*|*/.claude|*/.claude/*)
note "tracked agent-context file: $f (CLAUDE.md/.claude must never be tracked in a public repo)" ;;
esac
done < <(git ls-files)
# Match a PCRE against a file, emitting "LINE: MATCH". The pattern is passed as
# an argument (not interpolated into a //) so metacharacters like the "/" in a
# path-style token can't break the regex. Reads raw bytes so non-UTF-8 blobs
# don't abort the scan.
pcre_matches() {
perl -e '
my ($file, $re) = @ARGV;
open(my $fh, "<:raw", $file) or exit 0;
my $rx; eval { $rx = qr/$re/i }; exit 0 if $@;
while (my $l = <$fh>) { if ($l =~ /$rx/) { print "$.: $&\n"; } }
' "$1" "$2" 2>/dev/null
}
# This script's own source necessarily contains the detection patterns (e.g.
# the regex tokens in INFRA_RE), so scanning it would always self-flag. Skip it.
SELF="$(git ls-files --full-name -- "$SELF_ABS" 2>/dev/null | head -1)"
echo "── leak-guard: internal-infra references in tracked files ──"
while IFS= read -r f; do
case "$f" in *.png|*.jpg|*.jpeg|*.ico|*.gif|*.bin|*.crate|*.gz|*.zip|*.pdf) continue ;; esac
[ -n "$SELF" ] && [ "$f" = "$SELF" ] && continue
[ -f "$f" ] || continue
while IFS= read -r hit; do
[ -z "$hit" ] && continue
note "internal-infra reference: $f:$hit"
done < <(pcre_matches "$f" "$INFRA_RE")
while IFS= read -r hit; do
[ -z "$hit" ] && continue
note "[HOME-PATH] absolute home path: $f:$hit (no local home path may be committed to a public repo)"
done < <(pcre_matches "$f" "$HOMEPATH_RE")
done < <(git ls-files)
RANGE="${1:-}"
if [ -n "$RANGE" ]; then
echo "── leak-guard: AI-attribution in commit messages ($RANGE) ──"
while IFS= read -r sha; do
[ -z "$sha" ] && continue
msg="$(git log -1 --format='%B' "$sha" 2>/dev/null || true)"
# Pass the pattern as an argument (not interpolated into a //) so the
# lookbehind char class and "/" don't break the regex.
hit="$(printf '%s' "$msg" | perl -e '
my $re = $ARGV[0]; my $rx = qr/$re/i;
while (my $l = <STDIN>) { if ($l =~ /($rx)/) { print "$1\n"; last; } }
' "$ATTR_RE" | head -1 || true)"
[ -n "$hit" ] && note "commit ${sha:0:12}: message contains \"$hit\" (owner rule: zero AI attribution, ever)"
done < <(git rev-list "$RANGE" 2>/dev/null || true)
fi
echo
if [ "$fail" -ne 0 ]; then
echo "✗ leak-guard: blocking finding(s) above — DO NOT MERGE/PUBLISH"
exit 1
fi
echo "✓ leak-guard: clean"
-159
View File
@@ -1,159 +0,0 @@
# keydb_format.rs — design notes
## Full parser API retained on purpose
The parser is copied verbatim, so it carries the full KeyDb/DiscEntry API
even though this crate's consumer (`keydb.rs`) only exercises a subset
(`load`, `find_disc`, `iter_disc_entries`, and the public fields read by
`KeydbSource::unit_keys_from` / `KeydbSource::host_certs`). NOTE: an earlier
revision of this comment cited a `candidates_from` function — no such
function has ever existed in this crate; the real consumer is
`KeydbSource::unit_keys_from` (keydb.rs), which reads `unit_keys`, `vuk`,
`media_key` AND `vid` (`vid` is load-bearing for the MK+VID derivation the
kat_c / kat_d KATs pin — it is NOT unused). `DiscEntry::title` is likewise
read — by `to_keydb_cfg` and the redacting `Debug` impl — so it is not
unused either. The genuinely unused items — `empty` and `find_vuk` — are
part of the faithful copy and are retained rather than pruned; allow
dead_code so the byte-for-byte copy compiles clean without diverging from
the libfreemkv original.
## ParseStats — what `KeyDb::parse` threw away
The parser is fed a THIRD-PARTY file: a mirror can serve a truncated or
byte-shifted download that is still valid UTF-8, still contains enough
recognisable rows to pass `KeydbSource::save`'s "at least one entry" check,
and is then atomically committed as the new keydb — after which every disc
past the corruption point silently resolves nothing. Before this counter,
each malformed `| DK` / `| PK` / `| HC` / `| HC2` / `0x…` row was dropped by
an `if let Some(..) = .. { }` with no `else`, no counter and no log, so that
file was indistinguishable from a healthy one at every layer above.
The counts are logged ONCE per parse (a per-line log would be 181k lines on
a real keydb, which is why the rate limit is "one summary", never "hide the
count").
## `KeyDb::load_counted` — same-descriptor stamp + parse
Two callers need this seam, both in `crate::KeydbSource`:
* The identity stamp and the bytes must come from the SAME descriptor.
Stat-the-path-then-load-the-path is two independent resolutions of one
name, so an atomic rename landing between them stored a stamp
describing one file next to the parsed contents of another — and the
cache then served that pairing until the stamp changed again. `fstat`
on the open handle cannot be pointed at a different inode.
* The rejection counts have to be RETAINED, not just logged, so a cache
hit can re-emit them.
## `to_keydb_cfg` — trailing-comment placement
The trailing `; <comment>` (MKB version / volume size / UHD) is emitted
ONLY after a `U` (unit-keys) field — that is the one place the parser
splits the value on `;`. Gluing a comment onto an `M`/`I`/`V` value would
make `parse_hex16` reject the whole field, so a comment-bearing entry that
has no unit keys drops its comment (keys always survive; the metadata is a
derivable hint). Real per-disc rows that carry metadata also carry keys.
## `parse_disc_entry` title handling
Title = everything between `= ` and the first ` | ` field (or the trailing
`;` comment), kept VERBATIM (trimmed). This is a FAITHFUL copy of the keydb
title, so it must round-trip exactly: a previous version extracted a `(...)`
substring as a "display title", but that TRUNCATED real titles that
legitimately contain parentheses ("Lawrence of Arabia (Restored Version) –
Disc 2 …" → "Restored Version") and broke serialize→parse idempotence.
Display prettification, if wanted, belongs in the title-display layer, NOT
this codec.
## Real-data test notes
`to_keydb_cfg_is_idempotent_on_real_keydb`: parse the full keydb -> serialize
(S1) -> parse S1 -> serialize again (S2). S1 MUST equal S2 byte-for-byte.
This is the right invariant: a raw keydb.cfg has formatting variance
(whitespace, optional fields, comment style) that our CANONICAL serializer
normalizes, so `text == to_keydb_cfg` is NOT expected — but once normalized,
a re-load+re-serialize must be stable. Idempotence here proves `parse` is
lossless on its own output and `to_keydb_cfg` is deterministic. Also asserts
no rows are dropped. It used to carry no `#[ignore]` and `return` early when
`KEYDB_PATH` was unset — so CI ran it, asserted NOTHING about 181k-entry
behaviour, and reported green. A skip has to be visible in the test output
to mean anything, and an ignored test that is RUN without its fixture must
fail loudly rather than silently pass.
`real_keydb_path`: the old pattern (`match keydb_path() { Some(p) => p, None
=> return }`) on a non-ignored test is the worst of both worlds: CI ran the
test, asserted nothing, and reported success. These tests are `#[ignore]`d,
so running one at all is an explicit request — and an explicit request with
no fixture must fail, not quietly pass.
## `load_counted_reads_the_handle_not_the_path` test
`load_counted` must read the HANDLE it was given, never re-resolve the
path. `KeydbSource::cached_db` stamps the open file with `fstat` and then
parses from that same descriptor precisely so the recorded identity and
the parsed bytes cannot describe two different files; the moment this
function re-opens the path, that guarantee is gone and an atomic rename
landing in between stores one file's stamp beside another file's keys —
a mismatch the cache then serves as if it were valid.
The race itself cannot be scheduled deterministically in-process, so the
SEAM is what is pinned: with the path already replaced, the handle's
contents must still win. Catches the mutation that restores an internal
`File::open(path)`.
## `host_cert_debug_is_redacted_including_the_sibling_wrapper` test
The SIBLING type. `KeyDb`'s hand-written `Debug` prints `host_certs_len`, so
it protects the whole-db rendering and nothing else — `KeyDb::host_certs` is
a public field, and `{:?}` on ONE element of it walks a completely different
code path: `KeydbHostCert`'s derive, then `HostCert`'s impl. That path
carries the AACS host PRIVATE key, the only key material in the crate that
authenticates US rather than opening a disc.
It is safe today only because `libfreemkv::HostCert` hand-writes a redacting
`Debug` (`aacs::types`, `private_key: "<redacted>"` + `certificate_len`) —
an EXTERNAL guarantee this crate silently inherits through the derive, with
nothing on this side pinning it. This test pins it: it catches a libfreemkv
upgrade that swaps that impl for `#[derive(Debug)]`, which would leak the
host private key out of this crate's public API with no diff in this
repository at all.
Rendered three ways because callers reach the cert three ways: the whole
`Vec` (`db.host_certs`), one element, and the inner `HostCert` a caller
gets back from `KeyDb::host_certs(mkb)`.
## KEYDB-parser integration tests relocated from libfreemkv
These exercise the parser (`KeyDb::load`) end-to-end against a real
keydb.cfg and feed its material into libfreemkv's AACS crypto
(`derive_vuk`, then `decrypt_unit` + `is_clean_ts`). They live here now
that the parser lives here. All are `KEYDB_PATH`-env-gated and no-op in CI
when the env is unset; they must still COMPILE.
## `test_decrypt_real_unit` — no-decrypt outcome
Reaching the fallback path is the EXPECTED outcome for the AACS 2.0 (BEE)
sample this test was written against: the unit is still bus-encrypted, so
no unit key alone can open it. That makes "no key worked" an unusable
assertion on its own — which is exactly why this test used to end in an
`eprintln!` and fall off the end, i.e. pass unconditionally. A no-op
`decrypt_unit`, an `is_clean` stuck at `false`, or a parser that yielded
zero keys would all have produced this same clean exit.
What CAN be asserted regardless of the sample: the pipeline actually ran,
and the decrypt primitive actually transformed the bytes.
## Hand-written redacting `Debug`
Key material must never reach a log, a panic message, or a bug report. Both
`KeyDb` and `DiscEntry` carry raw AACS key bytes, and both are public API, so
a `#[derive(Debug)]` on either is a leak of the crown jewels: `{:?}` on a
loaded `KeyDb` printed every processing key plus every disc's Media Key,
VID, VUK and Unit Keys. libfreemkv's equivalent types (`aacs::types`:
`DeviceKey`, `HostCert`, `Vid`, `MediaKey`, `Vuk`, `ProcessingKey`,
`UnitKey`, its own `DiscEntry`) all hand-write a redacting `Debug` for
exactly this reason; this crate's copy of the parser dropped that
convention when it was relocated. These restore it, field-for-field in the
same style (`<redacted>` for key bytes, a `_len` for key-bearing
collections). Pinned by `keydb_debug_is_redacted` /
`disc_entry_debug_is_redacted`.
-432
View File
@@ -1,432 +0,0 @@
# `keydb.rs` design notes
Long-form rationale moved out of `src/keydb.rs` doc comments to satisfy the
repo's comment-guard line caps. Each section is pointed to from the file by a
one-line `// See docs/keydb.md#<anchor>` comment.
## candidate-order
Full derivation path this source walks, cheapest-first:
1. per-disc **Unit Keys** (hash hit) → returned terminal, no derivation.
2. per-disc **VUK** (hash hit) → `uks_from_vuk` over the disc's encrypted
title keys.
3. a **Media Key**, then `derive_vuk` → `uks_from_vuk`. The MK comes from, in
order: the disc's stored MK (hash hit); the keydb's **Processing Key**
pool walked against THIS disc's MKB via `derive_media_key_from_pk`; or the
device-key pool via `derive_media_key_from_dk`. The PK and DK pools
resolve the Media Key WITHOUT a VID; the final `derive_vuk` still needs
one. The VID is the unlocker's physical VID (`ResolveCtx::vid`) when
present, else the keydb entry's OWN stored VID (the `I` field, `vid`) for
the non-physical / ISO path. With no VID from either source the MK path
cannot complete — return nothing.
The cross-disc MK-pool brute (trying OTHER discs' stored media keys against
this disc) stays RETIRED: every MK path here is anchored to the matched
disc's own MKB or stored material.
The library still OWNS the crypto; this source owns only which primitive to
call with which material. Returning an empty `Vec` is a genuine "no key for
this disc here".
## file-identity
`(len, modified)` ALONE IS NOT AN IDENTITY. mtime is stored at whole-second
resolution on HFS+, on many container overlays and on most network
filesystems, so a writer that replaces `keydb.cfg` within the same second
with a file of the same byte length leaves `(len, modified)` bit-identical —
and the cache then serves the superseded parse for the rest of the process's
life. For a key database that is not a stale page: it is serving retired or
since-corrected keys while reporting success. The writers that hit this are
real and named in this crate's own docs: the daily-refresh job, a second
freemkv process, an operator's editor, a sync tool.
Two independent discriminators close it:
1. `dev` + `ino`. Every writer that publishes ATOMICALLY (temp file, fsync,
rename — what `KeydbSource::save` does, and what any correct publisher
does) creates a NEW inode, so the stamp differs immediately and
unconditionally, whatever the clock or the mtime resolution did. Zero on
platforms without them (see `FileStamp::of`), which costs nothing: rule 2
stands alone.
2. A freshness rule on the cache entry — `CacheEntry::is_settled` — which
covers the residual in-place rewrite that reuses the inode.
What this still does NOT catch: a writer that FORGES the timestamp, i.e.
restores an mtime equal to the cached one (`touch -t`, `rsync --times`, a
tar/backup restore) into the SAME inode with the SAME length. Nothing short
of hashing the content can see that, and hashing 62 MiB per lookup is
precisely the cost the cache exists to remove (~141 ms × ≥3 loads per rip).
`KeydbSource::save` therefore keeps invalidating the cache directly, so the
one write path this crate owns never depends on any of the above.
## settle-proof
Whether a cache entry's stamp can be TRUSTED to change if the file changes.
The proof, for a filesystem storing mtime at granularity `G`: suppose this
entry was stamped at `stamped_at` with `stamped_at - modified >= G`, and some
writer later rewrites the file in place at wall time `T > stamped_at`. The
recorded mtime `m'` of that write is within one granule of `T`, so
`m' >= T - G > stamped_at - G >= modified`. Hence `m' != modified` and the
stamp DOES change. The ambiguous window is therefore exactly the one where
the cached entry was stamped less than `G` after the file's own mtime — i.e.
we looked at the file while it was still "hot" — and in that window the
entry is refused and the file is re-read. Once the file has been quiet for
`G`, its entry settles and the cache is fully effective again.
Cost: after every keydb write, lookups re-parse for up to `G` (2 s). The
published keydb is rewritten daily, so that is ~2 s of re-parsing per day
against a cache that saves ~420 ms per rip.
`modified: None` (a filesystem that reports no mtime) never settles: the only
two discriminators left would be size and inode, so the entry is re-read
every time rather than being trusted on a weaker basis.
CLOCK CAVEAT: `stamped_at` is the local clock and `modified` is the
filesystem's. On a network share whose server clock runs AHEAD of the
client's by more than `G`, entries never settle (safe: more re-parsing). A
server clock BEHIND the client's makes an entry settle early, which re-opens
the same-second window it closes — the inode rule (1) is what covers the
realistic writers there.
## keydbsource-cache
`KeydbSource` caches the parsed database behind the file's identity stamp.
The published keydb is ~62 MiB / ~181k entries and a single AACS-cert rip
parses it repeatedly — `host_certs()` for the drive handshake (freemkv's
`pipe.rs`/`engine.rs`, autorip's `keysource.rs` each build their own source),
then the trait `host_certs(mkb)` during resolution, then `get_unit_keys` —
and every one of those calls used to re-read all 62 MiB from disk and
re-parse it from scratch, because the struct held nothing but a `PathBuf`.
A keydb replaced underneath this source is picked up on the next call, with
ONE stated exception: a writer that rewrites the file in place, into the
same inode, at the same byte length, having restored the previous mtime
exactly (`touch -t` / `rsync --times` / a backup restore). Everything else —
any atomic rename, any length change, any real clock advance — changes the
stamp. `FileStamp` and `CacheEntry::is_settled` carry the full argument and
the clock caveats; do not weaken either without re-reading them. (The
earlier wording here — "an externally replaced keydb is still picked up" —
was simply false for a same-second, same-length replacement, which is the
common shape of a daily-refresh job on a whole-second-mtime filesystem.)
MEMORY RESIDENCY (accepted, and deliberately recorded rather than fixed
here): a hit keeps the entire parsed keydb — every disc's Media Key / VID /
VUK / Unit Keys, the processing- and device-key pools, and the host-cert
PRIVATE keys — resident for the process's life, where the pre-cache code
held one parse transiently. Nothing in this crate zeroizes on drop, so that
material was always exposed to a core dump or a swapped page; the cache
widens the WINDOW, it does not create the class. Closing it properly means
zeroize-on-drop across `KeyDb`, `DiscEntry` and libfreemkv's `aacs::types`
(which owns most of the buffers) — a cross-crate change, tracked separately,
NOT smuggled in behind a cache fix.
## cached-db-one-open
Errors are the same ones `KeyDb::load` produces, unchanged: a missing file
is `NotFound` (the documented benign case), an over-cap or non-UTF-8 file is
`InvalidData`. The stamp costs an `fstat`, not a 62 MiB read.
ONE OPEN, ONE IDENTITY. The file is opened once and both the stamp (`fstat`
on that handle) and, on a miss, the bytes come from it. The previous shape —
`std::fs::metadata(path)` and then a separate `KeyDb::load(path)` — resolved
the path TWICE with no atomicity between them, so a rename landing in the
gap cached the new file's stamp beside the old file's contents (or the
reverse); the pairing was then served, looking perfectly valid, until the
stamp changed again. The stamp is read BEFORE the bytes on purpose: a write
racing the read then leaves the OLD stamp cached, which the next call
detects, whereas stamping afterwards would file torn content under a
fresh-looking identity and pin it.
## cached-db-reemit
Re-emit the parser's rejection summary on EVERY hit, not just on the parse
that produced it. `KeyDb::parse` is what logs it, and a hit skips `parse`
entirely — so a daemon that cached a corrupt keydb logged its rejected-row
counts exactly once, at whatever moment it first read the file, and then
served that same damaged database to every later rip in silence. Repetition
is the correct failure mode here: the line is emitted only when the file
really does have rejected or duplicate rows, and a warning an operator can
still see on the tenth rip is worth more than one they had to catch on the
first.
## load-failure
A MISSING keydb is the documented benign case: the app may simply not have
one, another source may hold the key, and "no keydb" is genuinely "no key
here". Everything else is NOT that: `InvalidData` is a keydb over the
128 MiB cap or one that is not valid UTF-8 — i.e. corrupt, truncated, or a
half-finished download — and a permission/IO failure means the file could
not be read at all. Reporting either as an empty result made a broken keydb
indistinguishable from "this disc has no key" (the same conflation as the
seven-hour 502), with no log to notice it by. Both are now logged AND
surfaced as errors so the resolver reports a source failure instead of
`E7022`.
## save-mirror-parse
Mirror `KeyDb::parse`'s disc-entry rule EXACTLY by CALLING it
(`is_disc_entry_line`), so `save()` never validates + persists content that
parses to zero usable entries (e.g. a stray "0xDEADBEEF" comment line) and
the two can no longer drift — they did drift once already, on the `0X` case
rule.
## unit-keys-from
CPS-unit numbering: a returned `UnitKey::idx` is the POSITIONAL index
libfreemkv's `resolve_and_apply` turns into the canonical CPS-unit number
`idx + 1`. For the terminal per-disc unit-key path we therefore map the
keydb's stored CPS number `num` to `idx = num - 1`, so the committed number
is byte-identical to the keydb's `num` (and to what the OLD
`Key::Unit(entry.unit_keys)` path committed). For the VUK / MK paths the
boil primitive already yields 0-based positional indices, matching
`parse_unit_key_ro`'s `(i + 1)` after the resolver's `+ 1`.
## unit-keys-per-disc-hit
Per-disc hit (most specific). `find_disc` normalizes the hash form. Without
a matched entry this keydb has no per-disc material (Unit Keys / VUK / Media
Key / stored VID) to anchor a derivation for the disc, so it resolves
nothing. (The PK and DK pools are global, but the cross-disc MK-pool brute —
trying OTHER discs' media keys against this disc — stays retired; a PK/DK
pool only ever resolves a disc reached through its own matched entry below.)
## unit-keys-union
UNION every source of terminal keys, then dedup — never first-hit. A stored
`unit_keys` list can be PARTIAL (the key-import tool only ever sampled the
CPS units reachable from a playlist, so an orphan unit's key may be
missing), while the per-disc VUK boils EVERY declared CPS unit. Taking the
stored list alone (the old return-at-first-path) would shadow the VUK and
silently drop the orphan unit's key. So gather both and keep a unique-by-key
list: the read path tries every key per unit, so an extra or stale key is
harmless — only a MISSING key hurts.
## unit-keys-vuk-or-mk
2. Per-disc VUK — one step, no VID needed; boils ALL declared units.
3. Else a Media Key path (stored MK / PK pool / DK pool) → VUK → all
declared units. The MK itself carries no VID, but the final `vuk_from_mk`
needs one: physical (unlocker) VID first, else the entry's stored VID
(`I` field), else cannot derive. Either branch yields the COMPLETE
declared set, so we take the first that resolves (VUK preferred —
cheapest).
## write-atomic
keydb.cfg is the single source of AACS truth, and save/update run
unattended (first-boot download + daily-refresh thread, with a container
restart on every release). A bare in-place `fs::write` truncates the file
before writing, so a SIGKILL (docker stop's grace window), OOM-kill, power
loss, or ENOSPC mid-write would leave the keydb half-written — the prior
good copy already gone. A truncated keydb doesn't error at write time; it
silently breaks key resolution on every later AACS rip. Writing to a temp
file then renaming (POSIX rename is atomic within a filesystem) means an
interrupted update leaves the previous keydb fully intact.
The fsync MUST succeed before the rename: a `sync_all` failure (ENOSPC,
ESTALE on the bind-mounted volume) means the kernel never guaranteed the
bytes reached stable storage, so publishing them via rename would defeat
crash-safety. The temp name is unique per call (pid + monotonic counter) so
a concurrent update can't share a fixed temp path and rename a mangled file
over the keydb.
## get-unit-keys-trait
A MISSING keydb is not an error — it simply yields no keys (another source
may have them), the same as the library's own loader. A keydb that exists
but cannot be USED (over the size cap, not UTF-8, unreadable) is a source
failure and returns `Err`: it says nothing about whether this disc has a
key, and reporting it as an empty result is the same looks-like-success
failure as a 502 reported as `E7022`.
The keydb carries no AACS 2.1 forensic index keys today, so it does not
override `get_fmts_indexes` — the default (empty) opts it out, and an FMTS
disc's forensic set comes from the online source.
## host-certs-trait
Expose the keydb's host certs through the trait — the OEM/AACS cert-auth
route collects them across every source via this method. Wires the disc's
MKB generation through for revocation filtering (the keydb parser's
`; Revoked in MKBv<N>` annotation): a cert revoked at generation `R` is
withheld once the disc's generation reaches `R`.
## test-pinned-mtime
Force a file's mtime to `PINNED_MTIME`, so two different files can be made
bit-identical to a `(len, mtime)` stamp on purpose. The cache tests that
matter all depend on FORCING the indistinguishable case rather than hoping
the filesystem produces it: on APFS (nanosecond mtimes) a same-second
replacement is otherwise detected by mtime alone, and a test that relies on
that passes with the fix reverted — which is worth nothing.
## test-committed
The committed `(cps, key)` pairs libfreemkv's `resolve_and_apply` derives
from a source's Unit Keys: positional `idx` → canonical CPS number
`idx + 1`. The KATs compare against THIS to prove byte-identical parity with
the OLD `Key::Unit` / resolver-derived commit.
## test-mock-ctx-stub
`MockCtx` implements the full `ResolveCtx` trait so it can stand in for a
real disc, but `KeydbSource` itself never calls `title()`/`samples()` (those
matter to the online source, not the local keydb). Exercise them directly
so the mock's trait surface is proven correct too.
## test-kat-a
Here `enc_title_keys` is empty, so the VUK can't derive anything — only the
stored list contributes, and it commits byte-identically to the stored
`(cps, key)` pairs.
## test-orphan-union
Orphan-unit completeness (the real keydb bug): an entry stores only `uk1`
(the key-import tool sampled one reachable CPS unit) but ALSO carries the
VUK, which boils BOTH declared units. The old return-at-first-path handed
back just `[uk1]`, shadowing the VUK and silently dropping the orphan unit.
The union must return BOTH — the stored uk1 AND the VUK-derived second unit.
## test-kat-c
A hash hit with a Media Key and a physical VID (from the unlocker) derives
`MK → VUK → UK`. The PHYSICAL VID must be used in preference to the keydb's
stored VID — proven by giving the entry a DIFFERENT stored VID and showing
the result tracks the physical one.
## test-kat-f
Owner decision #1 (AACS): a keydb Processing Key must be walked against the
matched disc's own MKB to recover the Media Key, then driven down the full
chain `PK → MK → VUK → UK`. The disc entry carries NO stored MK/VUK/UK — the
only key material is a global `PK` row — and the result must be the real
Unit Keys, byte-identical to deriving from the recovered MK.
The MKB + PK use a known-answer construction (a planted PK whose derived MK
satisfies the synthetic verify record); the constants are precomputed AES
vectors so this crate needs no AES primitive of its own. They mirror
libfreemkv's `boil::mk_from_pk_drives_full_chain_to_uks` KAT.
## test-repeated-lookups
A single AACS-cert rip drives this source at least three times — the
scan-options builder's inherent `host_certs()`, the trait `host_certs(mkb)`
during the handshake, then `get_unit_keys` — and each call used to re-read
and re-parse the WHOLE file, because `KeydbSource` held nothing but a
`PathBuf`. Measured on the real 62 MiB / 184,860-entry keydb: ~140 ms per
parse in `--release`, i.e. ~420 ms and ~186 MiB of file reads per rip,
against ~1 µs for a cache hit.
Catches the mutation that drops the cache (every call re-parses: count 3)
and the mutation that makes the cache unconditional (see the invalidation
tests below).
The mtime is pinned into the PAST first, because a cache entry is only
trusted once the file has been quiet for the mtime granularity
(`CacheEntry::is_settled`) — a keydb written microseconds ago is
deliberately re-read. Unix-only: pinning an mtime needs `touch -t` (no
`filetime` dependency for a test).
## test-changed-keydb
The cache must not go stale: a keydb replaced UNDERNEATH the source (the
daily-refresh thread writing through a different `KeydbSource`, or an
operator dropping in a new file) has a different size/mtime stamp and must
be re-read. Catches the mutation that caches on first load and never
re-stats — which would pin a rip to a keydb deleted hours earlier.
## test-two-discriminators
The defect both of the inode/settle tests below pin: `(len, mtime)` is not a
file identity. An external writer — the daily-refresh job, a second freemkv
process, an editor, a sync tool — that replaces `keydb.cfg` inside one mtime
granule with a file of the SAME length leaves that pair bit-identical, and
the source then serves the superseded parse for the rest of the process's
life: retired or since-corrected keys, reported as success.
Each test disables the OTHER discriminator through `mtime_granularity`, so
neither can pass on the strength of the one it is not testing, and neither
depends on the host filesystem's real mtime resolution (on APFS the naive
scenario is not reproducible at all).
## test-inode-rename
A keydb replaced by ATOMIC RENAME — how every correct publisher, including
this crate's own `save()`, installs a new file — must be re-read even when
length and mtime are identical. `mtime_granularity: 0` makes every entry
settle, so ONLY dev+ino can distinguish the two files here.
Catches the mutation that drops `dev`/`ino` from `FileStamp` (the original
`(len, modified)` stamp): the source then keeps answering with the previous
keydb's keys.
## test-inplace-rewrite
A keydb rewritten IN PLACE — same inode, same length, mtime forced back to
the same value — must still be re-read while the cached entry is too young
to trust. `mtime_granularity` is set to a decade so nothing ever settles,
which is the same condition a real same-second, whole-second-mtime
replacement creates and the only way to reproduce it deterministically on a
nanosecond-mtime filesystem.
Catches the mutation that deletes the `is_settled` check from `cached_db`
(every stamp match trusted unconditionally, which is what the original code
did): dev, ino, len and mtime are ALL identical here, so without the
freshness rule the source serves the old keys forever.
## test-corrupt-warns
`ParseStats::log` runs inside `KeyDb::parse`, and a cache hit skips `parse`
— so once the cache landed, the rejected-row summary for a given file was
emitted exactly ONCE for the entire life of the process, while that same
damaged file went on being served to every subsequent rip. A long-running
daemon (autorip) logs it at startup, the operator misses it in the startup
noise or loses it to log rotation, and there is no second chance for as long
as the process runs.
Catches the mutation that deletes the `emit_parse_stats` call from the
cache-HIT arm of `cached_db`: warnings would drop to 1 while the file is
still served three times.
## test-healthy-silent
Catches the mutation that makes `emit_parse_stats` unconditional (drops
`ParseStats::log`'s empty-counts early return).
## test-fresh-not-trusted
Catches the mutation that widens the window to something unbounded (a
granularity of hours would re-parse 62 MiB on every lookup all day) by
pairing with `repeated_lookups_parse_the_keydb_once`, which proves the entry
DOES settle once the mtime is in the past.
## test-save-invalidates
`save()` replaces the very file this source reads, so it drops the cached
parse ITSELF rather than trusting the (size, mtime) stamp: a same-size
rewrite that lands inside the filesystem's timestamp granularity is the one
change the stamp cannot see — and it is exactly the change this crate
performs (the daily-refresh thread re-saving a keydb through the source it
then reads from).
The test forces that indistinguishable case rather than hoping for it: the
file is written, stamped to a FIXED mtime, cached, re-saved with different
keys of the SAME length, and stamped back to the same mtime. To the stamp
the two files are identical, so only the explicit invalidation can produce
the new key. Unix-only because pinning an mtime needs `touch -t` (no
`filetime` dependency for one test).
## test-corrupt-is-error
`KeyDb::load` fails distinguishably for a keydb that is over the 128 MiB cap
or not valid UTF-8 — a truncated download, a half-written file, a binary
blob dropped in by mistake. Both used to collapse into `Ok(Vec::new())` with
ZERO tracing, i.e. into the benign "no key for this disc" the resolver
reports as `E7022` — the same looks-like-success failure as the seven-hour
502. Only a MISSING file is genuinely benign.
Catches the mutation that restores `Err(_) => Ok(Vec::new())`.
</content>
-49
View File
@@ -1,49 +0,0 @@
# freemkv-keysources: scope and design notes
## Mechanism vs. policy
Applications (autorip, the `freemkv` CLI) choose and order the key sources
from their own config — the local-vs-online policy is just which impls they
plug in — then resolve and hand the resulting key to `Disc::decrypt_with`.
Each source resolves a disc's terminal **Unit Keys** in one shot via
`KeySource::get_unit_keys`, driving libfreemkv's boil-down crypto primitives
for whatever level of material it holds. Compose several with `MultiSource`
in the caller's chosen order.
Reading the encrypted content-sample units a key server validates on, and
applying the resolved keys against a disc, is decryption *mechanism* — it
lives in the library (`libfreemkv::resolve_and_apply`,
`libfreemkv::read_encrypted_units`), not here.
## `MultiSource` failure semantics
`MultiSource::get_unit_keys` tries each inner source in order and returns the
first non-empty Unit Key set (and `get_fmts_indexes` does the same for the
forensic set). The caller supplies the list AND the order — local-first
`[Keydb, Online]`, online-first `[Online, Keydb]`, etc. — so the "which
sources, in what order" policy lives entirely with the application, not the
library. `MultiSource` is itself a `KeySource`, so it nests and composes.
**A source that could not ANSWER is not a source that answered "no key".**
When no inner source produces keys, the composition returns `Err` if any
inner source failed, and `Ok(empty)` only when every source genuinely
answered and none held a key. This mirrors libfreemkv's own resolver
(`keysource::drive_unit_keys`, which tracks an `errored` flag, and
`source_failure`, which stamps the first failure onto the disc) — a
composition that swallowed the `Err` would hand the resolver a clean
`Ok(empty)`, and an outage would once again be reported to the operator as
`E7022 No key source has a decryption key for this disc`.
## `first_non_empty` ordering rule
A source failure never blocks the chain (a later source is still tried, and
a later success still wins outright, error or no error), but it is not
forgotten either: with no keys anywhere, the FIRST failure is returned.
First (not last) matches libfreemkv's `source_failure` rule, so the
most-preferred source's reason is the one the operator is told about, in the
same order the caller expressed a preference for successes.
`get` selects which trait method to drive, so the base and forensic paths
share ONE implementation of this rule and cannot drift apart — the forensic
half carried the identical bug.
-13
View File
@@ -1,13 +0,0 @@
# `MultiSource`: a source that could not answer is not a source that said no
`MultiSource` used to re-collapse the Ok/Err distinction `OnlineSource` was
engineered to draw: `if let Ok(uks) = s.get_unit_keys(ctx)` discarded every
`Err`, so a composition whose sources ALL failed returned `Ok(Vec::new())` —
which the resolver reports as `E7022 No key source has a decryption key for
this disc`. That is the seven-hour-502 incident reproduced one layer up.
The regression test `multi_source_reports_a_source_failure_instead_of_a_clean_no_key`
is THE regression check, one layer up from `interpret_reply`: when no source
holds a key AND a source could not answer, the composition must report the
FAILURE, never a clean "no key". It catches the mutation that restores
`if let Ok(uks) = ..` (which drops the `Err` and returns `Ok(empty)`).
-39
View File
@@ -1,39 +0,0 @@
# `src/online.rs` — `OnlineSource::agent` caching
## Why cache at all
`query` used to build a FRESH `ureq::Agent` every call, and an FMTS disc
makes two calls per rip (`get_unit_keys` + `get_fmts_indexes`), so it paid
two TLS handshakes to the same host for no reason — an agent owns the
connection pool, and a discarded agent discards the pooled, already
negotiated TLS connection with it.
## Security invariant
The agent is reused ONLY when the freshly resolved + guarded address SET is
identical to the one the cached agent is pinned to. Every query still
re-resolves and re-runs the SSRF guard — that is the whole anti-rebinding
defence and is deliberately not cached; what is reused is the connection,
and only to addresses just re-validated this call.
## Why the cache key is a sorted, deduped SET, not an ordered sequence
`to_socket_addrs` hands back whatever order the resolver felt like, and a
round-robin keyserver reorders its A/AAAA records on essentially every
lookup — an ordered `==` therefore missed the cache every single time
against exactly the deployment that most needs it, silently degrading to
the per-query TLS handshake this field exists to avoid.
Set equality is also the honest predicate: the security property being
asserted is "these are the same validated addresses", which says nothing
about their order. The agent itself stays pinned to the order it was BUILT
with; that is safe because an equal set means every address it holds was
re-resolved and re-guarded this call.
## `a_reordered_but_identical_address_set_reuses_the_agent`
Catches the mutation that drops the sort/dedup from `agent_for`'s key (the
reversed order would then build a second agent) and, via the second half of
the test, the mutation that "fixes" order-sensitivity by comparing only
lengths or the first element — which would wrongly reuse an agent pinned to
a DIFFERENT address.
-41
View File
@@ -1,41 +0,0 @@
# `src/online.rs` — `GuardFail` classification details
## Malformed-port authority
A `host:port` authority whose port is not a u16 is a TYPO in the operator's
config — `https://example.com:notaport/keys`. This used to fall back to the
WHOLE authority as the hostname (port text included), which of course never
resolves, so the URL surfaced as `GuardFail::Unreachable` →
`Err(KeyServiceUnavailable)`: a standing misconfiguration reported as a
transient outage, the exact collapse `GuardFail`'s own doc says must not
happen, because the two halves demand OPPOSITE operator actions (fix the URL
vs. wait).
Reject it as `Config`, identical to the bracketed-IPv6 branch a few lines
above, which already returns `Config` for an unparseable port. Silently
substituting `default_port` would be worse still — it would ship the bearer
token and key material to a port the operator never configured.
## Guard-blocked address (`Config` arm, `resolve_and_guard`)
Log THAT the URL was rejected, never WHY: the guard's message names the
resolved address, and an internal address must not reach a log an operator
may paste into a bug report. The static label is enough to separate
"misconfigured/blocked key-service URL" from "this disc has no key", which
is the only distinction the operator needs here.
`error!`, not `warn!`, and `Err`, not `Ok(empty)`. This arm used to return
`Ok(Vec::new())`, which made the PERMANENT fault quieter than the transient
one directly above it: the service was never asked, yet the composition was
handed a clean empty and — with no other source holding the key — told the
operator `E7022 no key for this disc`. Same lie as the seven-hour 502, from
a config typo that never self-heals. A failure to ask is never evidence
about the disc, so it leaves as `Err`; `MultiSource::first_non_empty` still
lets a later source's real key win outright, so this cannot fail a rip that
the local keydb could have served.
NOTE (reported upstream): libfreemkv has no `KeyServiceMisconfigured` code,
so the permanent fault borrows the transient E7028. E7028's contract — "the
source never got as far as answering the question" — is exactly true here;
only its "transient, retry later" hint is wrong, and the `error!` log
carries the correcting detail until a 70xx config code exists.
-20
View File
@@ -1,20 +0,0 @@
# `src/online.rs` tests — pre-flight guards in `query`
Each of the three guards (cleartext scheme, over-cap MKB, under-sampled)
fires BEFORE any address is resolved and before anything is sent. None was
exercised before these tests: a reordering that let the cleartext POST
through would have shipped green, and that POST carries the bearer token
plus base64 key material.
Their VERDICTS differ on purpose (see docs/online-query-guards.md): the
cleartext-scheme guard is an operator fault and yields `Err`, while the two
input-shaped guards yield `Ok(empty)` — this source has nothing for THIS
disc, permanently, and calling that an outage would send the operator
waiting for nothing.
The discriminator in all three: the configured host is `.test`, which RFC
6761 guarantees never resolves. On the guarded path nothing resolves it, so
the test touches no network and returns `Ok(empty)`. Remove the guard and
control reaches `resolve_and_guard`, which fails as `Unreachable` ->
`Err(KeyServiceUnavailable)` — a different Result, so the mutation cannot
pass.
-19
View File
@@ -1,19 +0,0 @@
# `src/online.rs` — `OnlineSource::query` return contract
The return type draws THE distinction this source exists to draw:
* `Ok(non-empty)` — the service answered with a key.
* `Ok(empty)` — the service ANSWERED and holds no key for this disc, or this
source had nothing to ask with (no service configured, a misconfigured
URL, an over-cap MKB, too few samples). A genuine miss; the resolver moves
to the next source and `E7022` is the right verdict.
* `Err(..)` — the service could not answer: unreachable, timed out, DNS
failed, returned 5xx, rejected the token (401/403), rate-limited (429), or
replied with something unreadable. NOTHING is known about whether a key
exists. Collapsing this into `Ok(empty)` is the bug this signature fixes:
a seven-hour run of HTTP 502s was reported to operators as
`E7022 No key source has a decryption key for this disc`, and they went
hunting for a VUK when the correct action was to wait.
`&self`: one-shot is the resolver's contract (each source's
`get_unit_keys` is called once), so no per-call latch is needed.
-57
View File
@@ -1,57 +0,0 @@
# `src/online.rs` — `OnlineSource::query` pre-flight guards
## Refusing cleartext (`http://`)
The request body always carries base64 AACS key material
(inf/mkb/units/vid) and, when configured, a bearer token in a header — so an
`http://` POST hands both to any on-path observer, and the token is
replayable.
There is no legitimate plaintext deployment to preserve here: the address
guard rejects loopback, RFC1918, ULA and link-local, so an `http://` URL can
only ever reach the PUBLIC internet — the single worst case for cleartext. A
self-hosted keyserver on a LAN is already impossible by that guard, whatever
its scheme.
Still falls THROUGH to the next key source (a local keydb) — an `Err` here
does not block the chain, because `MultiSource::first_non_empty` returns any
later source's non-empty key set outright and only surfaces the failure
when nothing anywhere resolved. What it does stop is the silent case: a
cleartext URL is the same permanent operator fault as the address-guard
rejection (see `GuardFail`), so it gets the same verdict. Returning
`Ok(empty)` here made "we refused to ask" indistinguishable from "the
service answered and has no key".
## The three-way skip/fault/miss line (MKB cap, sample count)
Written out after the `Config`-was-quieter-than-`Unreachable` bug, in three
parts:
* NO URL AT ALL is not a fault: the operator did not configure an online
source, so there is nothing to report. `Ok(empty)`, silently (the first
guard in `query`).
* A bad URL — wrong scheme, mistyped port, guard-blocked address — is an
OPERATOR FAULT. Every disc is affected, nothing was asked, and the fix is
to edit the config. That is `Err`.
* An over-cap MKB or too few content samples is a property of THIS DISC's
inputs, not a fault: the request cannot be *formed* for this disc, so this
source has nothing for it and never will, whatever the operator does and
however long they wait. Reporting that as `KeyServiceUnavailable` would
send the operator waiting out an outage that does not exist — the same
mislabelling in the opposite direction. These stay `Ok(empty)`, and stay
logged.
Logging on the MKB-cap skip matters: a silent empty return there is
indistinguishable from "no key", so the real cause is surfaced (the cap is
64 MiB, far above any real trimmed MKB).
## `cleartext_http_url_is_refused_before_anything_is_sent`
Catches two mutations: sending the request anyway (the host is `.test` and
would surface as `Unreachable`, but the assertion would still hold, which is
why `too_few_samples_skips_the_request` guards the send-path ordering
separately) and — the one this test was rewritten for — returning
`Ok(Vec::new())`, which made a permanent cleartext misconfiguration
indistinguishable from a genuine miss. A later source's real key still wins
(`MultiSource::first_non_empty`), so refusing does not fail a rip the local
keydb can serve.
-44
View File
@@ -1,44 +0,0 @@
# `src/online.rs` — SSRF guard
## Threat model
The keyserver URL is operator-supplied and the bearer token is confidential.
After `validate_keyserver_url` checked the host at config time, an attacker
who controls the keyserver's DNS can rebind it to 169.254.169.254 (cloud
metadata) or an RFC1918 host in the window between validation and the actual
POST, exfiltrating the key material and the Authorization token.
## Defence
Resolve the host once just before the POST, reject any blocked IP, and pin
the ureq connection to those validated addresses so a subsequent DNS flip
cannot redirect the request. Use `redirects(0)` so a public URL can't
30x-redirect to an internal host.
## Resolver wiring pitfall (`PinnedResolver`)
ureq 3 replaced v2's resolver closure with the `Resolver` trait. The agent
MUST be built through `Agent::with_parts` to take a custom one:
`Agent::new_with_config` silently keeps the default resolver, which would
send the request to live DNS and quietly reopen the rebinding window this
module exists to close. That failure is invisible — it has no symptom short
of an actual attack — so it is pinned by the test
`hardened_agent_connects_to_the_pinned_address_not_dns`.
## Why that test is structured the way it is
`resolve_and_guard` VALIDATES addresses; `hardened_agent` PINS them, so a
hostname cannot re-resolve to an internal address between validation and
connection. The validation half is well covered by other tests — but every
one of those still passes if the pinned resolver is never consulted, because
none of them makes a connection. Nothing else in the file proves the agent
honours the pin, and a mis-wired resolver fails OPEN: the connection quietly
falls back to live DNS and the rebinding window this module exists to close
is reopened, with no visible symptom.
The discriminator: pin the agent to a loopback listener the test owns, then
ask it for a host that CANNOT resolve — `.test` is reserved by RFC 6761 and
never resolves in the public DNS. Only a consulted resolver can turn that
name into a connection; a fallback to live DNS fails instead. No network is
touched, and `query` (whose guard blocks loopback by design) is not
involved — this drives `hardened_agent` directly.
-13
View File
@@ -1,13 +0,0 @@
# `keydb.cfg` path policy (`src/paths.rs`)
Key-path policy belongs with the key sources (this crate), not the library:
libfreemkv is handed a path and reads it. The CLI/app asks here for the
list of locations to *search* (first existing wins) and for the single
*default* location to write to (e.g. `update-keys`/save).
freemkv is a portable, standalone binary: the `keydb.cfg` lives *next to*
the executable — `<dir of current exe>/keydb.cfg` — and nowhere else. There
is no OS-specific config-dir lookup (`%APPDATA%`, `%USERPROFILE%\.config`,
`$XDG_CONFIG_HOME`, `$HOME/.config`). Drop the exe and its `keydb.cfg` in
the same folder and it works. Callers needing a custom location pass
`--keydb PATH`, which bypasses this module entirely.
+450 -127
View File
@@ -1,12 +1,10 @@
//! `keydb.cfg` key source (source #1).
//!
//! Parses a local `keydb.cfg`, looks the disc up by hash, and derives the
//! disc's terminal **Unit Keys** by composing libfreemkv's raw
//! `aacs::derive` primitives — never re-implementing AES. The path mirrors
//! the OLD candidate order EXACTLY, cheapest-first: per-disc Unit Keys, then
//! VUK, then a Media Key (stored / PK pool / DK pool) via `derive_vuk`. See
//! docs/keydb.md#candidate-order for the full path and VID rules. The
//! library owns the crypto; this source owns only which primitive to call.
//! Parses a local `keydb.cfg`, looks the disc up by hash, and derives the disc's terminal
//! **Unit Keys** by composing libfreemkv's raw `aacs::derive` primitives — never
//! re-implementing AES. The path mirrors the OLD candidate order EXACTLY, cheapest-first:
//! per-disc Unit Keys, then VUK, then a Media Key (stored / PK pool / DK pool) via
//! `derive_vuk`.
use std::io::{Read, Write};
use std::path::{Path, PathBuf};
@@ -15,16 +13,16 @@ use std::sync::{Arc, Mutex};
use crate::uks_from_vuk;
use libfreemkv::aacs::derive::{derive_media_key_from_dk, derive_media_key_from_pk, derive_vuk};
use libfreemkv::aacs::trace::{KeyNode, MatchedEntry};
use libfreemkv::aacs::types::{HostCert, MediaKey, UnitKey, Vid};
use libfreemkv::keysource::ResolveCtx;
use libfreemkv::keysource::{ResolveCtx, UnitKeyResolution};
use libfreemkv::{Error, KeySource};
use crate::keydb_format::KeyDb;
// Upper bound on decompressed keydb size (decompression-bomb cap): a tiny
// zip/gz could otherwise inflate to GiB and OOM the refresh thread. Mirrors
// keydb_format::MAX_KEYDB_BYTES (the on-disk load cap); keep the two equal.
const MAX_KEYDB_BYTES: u64 = 128 * 1024 * 1024;
// Decompression-bomb cap on decompressed keydb size: a tiny zip/gz could
// inflate to GiB and OOM the refresh thread. Same constant `KeyDb::load`
// uses on-disk, defined once in keydb_format so the two can't drift.
use crate::keydb_format::MAX_KEYDB_BYTES;
/// Result of a KEYDB save/update -- path written, entry count, and byte size.
#[derive(Debug)]
@@ -34,14 +32,12 @@ pub struct UpdateResult {
pub bytes: usize,
}
// Widest observed granularity of a filesystem's stored mtime (HFS+, many
// container/network filesystems record whole seconds); 2 s leaves rounding
// room. See docs/keydb.md#settle-proof for CacheEntry::is_settled's argument.
// Widest observed granularity of a filesystem's stored mtime (HFS+, many container/network
// filesystems record whole seconds); 2 s leaves rounding room.
const MTIME_GRANULARITY: std::time::Duration = std::time::Duration::from_secs(2);
// The identity of the keydb file a cache entry was parsed from. `(len,
// modified)` alone is not an identity (see docs/keydb.md#file-identity for
// why); `dev`+`ino` plus CacheEntry::is_settled close the gap.
// The identity of the keydb file a cache entry was parsed from. `(len, modified)` alone is not
// an identity; `dev`+`ino` plus CacheEntry::is_settled close the gap.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
struct FileStamp {
/// Filesystem device id. `0` where unavailable.
@@ -92,26 +88,35 @@ struct CacheEntry {
}
impl CacheEntry {
// Whether this entry's stamp can be TRUSTED to change if the file changes.
// Proof, cost, and the clock caveat are in docs/keydb.md#settle-proof;
// don't weaken this without re-reading them.
// Require a settled identity before reusing a cached parse: coarse mtimes can hide
// in-place edits within one timestamp tick. Missing/future mtimes use the age of
// the observation; inode identity detects atomic replacement.
fn is_settled(&self, granularity: std::time::Duration) -> bool {
self.stamp
match self
.stamp
.modified
.and_then(|m| self.stamped_at.duration_since(m).ok())
.is_some_and(|age| age >= granularity)
{
// The normal case: a real mtime that was already `granularity` in
// the past when we stamped it, so any later write bumps the mtime
// past our stamp and is detected. Trust it.
Some(age) => age >= granularity,
// No mtime (mtime-less FS) or a future mtime (clock skew): `duration_since`
// yields None, so the mtime can't discriminate change — settle via inode
// identity once this observation has itself aged past the granularity.
None => std::time::SystemTime::now()
.duration_since(self.stamped_at)
.is_ok_and(|age| age >= granularity),
}
}
}
/// A [`KeySource`] backed by a local `keydb.cfg` file.
///
/// The parsed database is CACHED behind the file's identity stamp: a single
/// AACS-cert rip calls `host_certs()`, the trait `host_certs(mkb)`, and
/// `get_unit_keys`, and each used to re-read + re-parse the whole
/// ~62 MiB file. A replaced keydb is picked up on the next call, with one
/// narrow exception; see docs/keydb.md#keydbsource-cache for that exception
/// and the memory-residency tradeoff — do not weaken [`FileStamp`] or
/// [`CacheEntry::is_settled`] without re-reading it.
/// The parsed database is CACHED behind the file's identity stamp: a single AACS-cert rip calls
/// `host_certs()`, the trait `host_certs(mkb)`, and `get_unit_keys`, and each used to re-read +
/// re-parse the whole ~62 MiB file. A replaced keydb is picked up on the next call, with one
/// narrow exception.
pub struct KeydbSource {
path: PathBuf,
// Mutex, not RwLock: the guarded section is a stamp compare + Arc clone,
@@ -122,9 +127,8 @@ pub struct KeydbSource {
// cache from a test, since timing is flaky and the parsed value is
// identical either way.
parses: AtomicUsize,
// The `G` of CacheEntry::is_settled, as a field so tests can isolate the
// two staleness discriminators one at a time (0 vs. a huge duration). See
// docs/keydb.md#test-two-discriminators.
// The `G` of CacheEntry::is_settled, as a field so tests can isolate the two staleness
// discriminators one at a time (0 vs. a huge duration).
mtime_granularity: std::time::Duration,
// Corruption summaries emitted (see emit_parse_stats). Same rationale as
// `parses`: avoids pulling a `tracing` subscriber into dev-dependencies.
@@ -167,9 +171,8 @@ impl KeydbSource {
self
}
// The parsed keydb, from cache when unchanged (errors mirror KeyDb::load).
// ONE OPEN, ONE IDENTITY: stamp (fstat) and bytes share one handle. See
// docs/keydb.md#cached-db-one-open for why a separate metadata()+load() pair was unsound.
// The parsed keydb, from cache when unchanged (errors mirror KeyDb::load). ONE OPEN, ONE
// IDENTITY: stamp (fstat) and bytes share one handle.
fn cached_db(&self) -> std::io::Result<Arc<KeyDb>> {
let file = std::fs::File::open(&self.path)?;
let stamp = FileStamp::of(&file.metadata()?);
@@ -227,9 +230,9 @@ impl KeydbSource {
self.parses.load(Ordering::Relaxed)
}
// Turn a KeyDb::load failure into this source's verdict: MISSING is the
// documented benign case (Ok/None); everything else is logged and
// surfaced as an error, not silently reported as "no key". See docs/keydb.md#load-failure.
// Turn a KeyDb::load failure into this source's verdict: MISSING is the documented benign
// case (Ok/None); everything else is logged and surfaced as an error, not silently reported
// as "no key".
fn load_failure(&self, e: &std::io::Error) -> Option<Error> {
match e.kind() {
std::io::ErrorKind::NotFound => None,
@@ -278,16 +281,10 @@ impl KeydbSource {
let entries = text
.lines()
.filter(|l| {
let t = l.trim();
// Mirror KeyDb::parse's disc-entry rule EXACTLY by CALLING it,
// so save() never persists content that parses to zero usable
// entries. See docs/keydb.md#save-mirror-parse.
crate::keydb_format::is_disc_entry_line(t)
|| t.starts_with("| DK")
|| t.starts_with("| PK")
|| t.starts_with("| HC")
})
// Mirror KeyDb::parse by running the same parsers, not a prefix check: a
// right-prefix row with bad hex / short cert parses to nothing, so a prefix count
// let junk past `entries > 0`.
.filter(|l| crate::keydb_format::is_parseable_entry_line(l))
.count();
if entries == 0 {
@@ -327,13 +324,11 @@ impl KeydbSource {
/// live-drive scan as `DriveCredentials` for the AACS handshake. Empty if
/// the keydb is missing/unreadable or carries no host cert.
///
/// Inherent, no-MKB form: this is used by the **scan-options** builder,
/// which runs before the disc's MKB generation is known, so no revocation
/// filtering is applied (passes `None`). The [`KeySource::host_certs`] TRAIT
/// method wires the real MKB generation through for revocation filtering.
/// Best first ([`KeyDb::host_certs_ranked`]): this is what the scan-options
/// builder hands the live-drive handshake as `DriveCredentials`.
pub fn host_certs(&self) -> Vec<HostCert> {
match self.cached_db() {
Ok(db) => db.host_certs(None),
Ok(db) => db.host_certs_ranked(),
// No error channel here (the scan-options builder wants a list), so
// the failure can only be LOGGED — but it must not be invisible.
Err(e) => {
@@ -343,20 +338,25 @@ impl KeydbSource {
}
}
// Derive this disc's terminal Unit Keys from a parsed keydb. Pure (no
// I/O). Empty Vec = no key for this disc from this keydb. CPS-unit
// numbering (idx = num - 1) is explained in docs/keydb.md#unit-keys-from.
// Return terminal Unit Keys from a parsed keydb without I/O.
// CPS unit numbers are one-based; the returned vector uses zero-based indexes.
fn unit_keys_from(db: &KeyDb, ctx: &dyn ResolveCtx) -> Vec<UnitKey> {
// Per-disc hit (most specific); find_disc normalizes the hash form.
// Without a match there is no per-disc anchor, so the global PK/DK
// pools are never consulted. See docs/keydb.md#unit-keys-per-disc-hit.
Self::resolve_from(db, ctx).keys
}
// De-conflated resolution: the keys PLUS whether the disc matched and — on a
// keyless match — WHY nothing derived, so a matched-but-underivable disc is
// never reported as a flat "no entry" (issue #46). Pure (no I/O).
fn resolve_from(db: &KeyDb, ctx: &dyn ResolveCtx) -> KeydbResolution {
// Per-disc hit (most specific); find_disc normalizes the hash form. Without a match
// there is no per-disc anchor, so the global PK/DK pools are never consulted.
let entries_loaded = db.disc_entries.len();
let Some(entry) = db.find_disc(ctx.disc_hash()) else {
return Vec::new();
return KeydbResolution::miss(entries_loaded);
};
// UNION every source of terminal keys, then dedup — never first-hit,
// since a stored `unit_keys` list can be PARTIAL while the VUK boils
// every declared unit. See docs/keydb.md#unit-keys-union.
// UNION every source of terminal keys, then dedup — never first-hit, since a stored
// `unit_keys` list can be PARTIAL while the VUK boils every declared unit.
let mut keys: Vec<UnitKey> = Vec::new();
// 1. Terminal Unit Keys stored in the entry — directly usable, no
@@ -386,9 +386,13 @@ impl KeydbSource {
&[]
}
};
// Why the keyless derivation could not finish, for the trace's miss
// path. `None` once any key lands (or when there was no material to try).
let mut miss_reason: Option<KeyNode> = None;
if !enc_title_keys.is_empty() {
// VUK path, else MK path (stored/PK/DK) → VUK. See
// docs/keydb.md#unit-keys-vuk-or-mk for the VID rules.
// VUK path, else MK path (stored/PK/DK) → VUK.
let derived = if let Some(vuk) = entry.vuk {
uks_from_vuk(&vuk, enc_title_keys)
} else {
@@ -420,7 +424,15 @@ impl KeydbSource {
uks_from_vuk(&derive_vuk(&mk.0, &vid.0), enc_title_keys)
}
// Locked VID-per-path rule: an MK with no VID cannot derive.
_ => Vec::new(),
// De-conflate the two reasons so the trace can say WHICH.
(Some(_), None) => {
miss_reason = Some(KeyNode::NoVid);
Vec::new()
}
(None, _) => {
miss_reason = Some(KeyNode::NoDerivableKey);
Vec::new()
}
}
};
keys.extend(derived);
@@ -429,7 +441,60 @@ impl KeydbSource {
// Unique by key value, first occurrence wins (stored numbering kept).
let mut seen = std::collections::HashSet::new();
keys.retain(|u| seen.insert(u.key));
keys
// Booleans-and-lengths shape of the matched entry, for the app to log —
// answers "what did the matched entry actually carry?" (issue #46).
let shape = MatchedEntry {
has_vuk: entry.vuk.is_some(),
has_unit_keys: !entry.unit_keys.is_empty(),
unit_keys_len: entry.unit_keys.len(),
has_media_key: entry.media_key.is_some(),
has_keydb_vid: entry.vid.is_some(),
enc_title_keys_len: enc_title_keys.len(),
vid_available: ctx.vid().is_some() || entry.vid.is_some(),
};
// On a match with no key: carry the specific reason if we have one, else
// leave it empty for the library to render a bare `NoDerivableKey`.
// KU J23: `NoVid` stays beside partial keys (the VID would derive the rest).
let miss_path = match miss_reason {
Some(n) if keys.is_empty() || n == KeyNode::NoVid => vec![n],
_ => Vec::new(),
};
KeydbResolution {
keys,
matched: true,
shape: Some(shape),
miss_path,
entries_loaded,
}
}
}
// The de-conflated result of a keydb lookup: keys plus enough context that a
// MATCHED-but-underivable disc is reported distinctly from a true miss. See
// [`KeydbSource::resolve_from`] and issue #46.
struct KeydbResolution {
keys: Vec<UnitKey>,
matched: bool,
shape: Option<MatchedEntry>,
miss_path: Vec<KeyNode>,
// Per-disc entries loaded in the keydb — named in a true-miss verdict so a
// reporter can confirm a wrong-pressing (`… not in keydb (N entries loaded)`).
entries_loaded: usize,
}
impl KeydbResolution {
// A true miss: the disc hash was not in the keydb at all.
fn miss(entries_loaded: usize) -> Self {
Self {
keys: Vec::new(),
matched: false,
shape: None,
miss_path: Vec::new(),
entries_loaded,
}
}
}
@@ -465,9 +530,8 @@ fn extract_zip(data: &[u8]) -> Result<String, Error> {
Err(Error::KeydbInvalid)
}
// Write `text` to `path` crash-safely (temp file, fsync, atomic rename) so an
// interrupted update never leaves a half-written keydb. See
// docs/keydb.md#write-atomic for the fsync-before-rename argument.
// Write `text` to `path` crash-safely (temp file, fsync, atomic rename) so an interrupted
// update never leaves a half-written keydb.
fn write_atomic(path: &Path, text: &str) -> Result<(), Error> {
let werr = || Error::KeydbWrite {
path: path.display().to_string(),
@@ -527,9 +591,8 @@ fn write_atomic(path: &Path, text: &str) -> Result<(), Error> {
}
impl KeySource for KeydbSource {
// Resolve this disc's base per-CPS-unit Unit Keys from the keydb. A
// MISSING keydb yields no keys (Ok(empty)); an UNUSABLE one is a source
// failure (Err) — never conflated. See docs/keydb.md#get-unit-keys-trait.
// Resolve this disc's base per-CPS-unit Unit Keys from the keydb. A MISSING keydb yields no
// keys (Ok(empty)); an UNUSABLE one is a source failure (Err) — never conflated.
fn get_unit_keys(&self, ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
match self.cached_db() {
Ok(db) => Ok(Self::unit_keys_from(&db, ctx)),
@@ -542,12 +605,34 @@ impl KeySource for KeydbSource {
}
}
// Expose the keydb's host certs through the trait, wiring the disc's MKB
// generation through for revocation filtering. See
// docs/keydb.md#host-certs-trait.
fn host_certs(&self, mkb: Option<u32>) -> Vec<HostCert> {
// De-conflated resolution (issue #46): besides the keys, report whether the
// disc MATCHED, why nothing derived on a keyless match, and the matched
// entry's shape to log — so a matched-no-key disc is not a mute "no entry".
fn resolve_unit_keys(&self, ctx: &dyn ResolveCtx) -> Result<UnitKeyResolution, Error> {
match self.cached_db() {
Ok(db) => db.host_certs(mkb),
Ok(db) => {
let r = Self::resolve_from(&db, ctx);
Ok(UnitKeyResolution {
keys: r.keys,
matched: r.matched,
miss_path: r.miss_path,
matched_entry: r.shape,
store_entries: Some(r.entries_loaded),
})
}
Err(e) => match self.load_failure(&e) {
// Missing keydb: the documented benign miss (nothing matched).
None => Ok(UnitKeyResolution::default()),
// Corrupt / unreadable keydb: a SOURCE FAILURE, not a miss.
Some(err) => Err(err),
},
}
}
// The keydb's host certs, best first. `mkb` is ignored: cert selection takes no disc input.
fn host_certs(&self, _mkb: Option<u32>) -> Vec<HostCert> {
match self.cached_db() {
Ok(db) => db.host_certs_ranked(),
// Vec-returning trait method: log the failure, return nothing.
Err(e) => {
let _ = self.load_failure(&e);
@@ -556,6 +641,12 @@ impl KeySource for KeydbSource {
}
}
// Keyed by disc hash, not by samples: asked once per resolve, not per
// piece (KU §2.3 step 8, N-KU10; KSK1).
fn answer_depends_on_samples(&self) -> bool {
false
}
fn label(&self) -> &'static str {
"keydb"
}
@@ -570,9 +661,8 @@ mod tests {
#[cfg(unix)]
const PINNED_MTIME: &str = "202601011200.00";
// Force a file's mtime to PINNED_MTIME so two different files can be
// made bit-identical to a (len, mtime) stamp on purpose, rather than
// hoping the filesystem produces that case. See docs/keydb.md#test-pinned-mtime.
// Force a file's mtime to PINNED_MTIME so two different files can be made bit-identical to
// a (len, mtime) stamp on purpose, rather than hoping the filesystem produces that case.
#[cfg(unix)]
fn pin_mtime(p: &std::path::Path) {
let ok = std::process::Command::new("touch")
@@ -661,9 +751,8 @@ mod tests {
}
}
// The committed (cps, key) pairs resolve_and_apply derives from a
// source's Unit Keys (idx -> idx + 1). KATs compare against THIS. See
// docs/keydb.md#test-committed.
// The committed (cps, key) pairs resolve_and_apply derives from a source's Unit Keys (idx
// -> idx + 1). KATs compare against THIS.
fn committed(uks: &[UnitKey]) -> Vec<(u32, [u8; 16])> {
uks.iter()
.map(|u| (u.idx.saturating_add(1), u.key))
@@ -682,9 +771,89 @@ mod tests {
assert_eq!(c.samples(4).unwrap(), Vec::<Vec<u8>>::new());
}
// KAT (a): disc with terminal Unit Keys, no enc_title_keys. Stored
// terminal unit keys are returned with CPS numbering preserved. See
// docs/keydb.md#test-kat-a.
// Issue #46 — de-conflated resolution: a matched entry with a Media Key but
// NO VID on this path (locked VID-per-path rule) is reported `matched` with a
// `NoVid` miss path — the exact case behind the reporter's "no entry".
#[test]
fn resolve_matched_media_key_without_vid_reports_no_vid_not_a_miss() {
let mut e = blank_entry(HASH);
e.media_key = Some([0x33u8; 16]);
let db = db_with(e, Vec::new());
// enc_title_keys present so derivation is attempted; ctx supplies no VID.
let r = KeydbSource::resolve_from(&db, &ctx(HASH, vec![[0x44u8; 16]], None));
assert!(r.keys.is_empty(), "no VID -> nothing derivable");
assert!(r.matched, "the disc WAS found — not a true miss");
assert_eq!(r.miss_path, vec![KeyNode::NoVid]);
let shape = r.shape.expect("a matched entry carries a shape");
assert!(shape.has_media_key);
assert!(!shape.has_vuk);
assert!(!shape.has_unit_keys);
assert_eq!(shape.enc_title_keys_len, 1);
assert!(!shape.vid_available);
}
// KU J23: stored unit keys for SOME CPS units plus a Media Key with no VID. The stored
// keys come back, and the miss path still says `NoVid`: the VID would derive the rest.
#[test]
fn partial_stored_keys_with_a_media_key_and_no_vid_report_no_vid() {
let mut e = blank_entry(HASH);
e.media_key = Some([0x33u8; 16]);
e.unit_keys = vec![(1, [0x55u8; 16])];
let db = db_with(e, Vec::new());
let r = KeydbSource::resolve_from(&db, &ctx(HASH, vec![[0x44u8; 16]; 2], None));
assert_eq!(r.keys.len(), 1, "the stored key");
assert_eq!(r.miss_path, vec![KeyNode::NoVid]);
}
// A hash that is NOT in the keydb is the one true miss: not matched, no
// shape, no miss path (the library renders this as `no entry`).
#[test]
fn resolve_unmatched_hash_is_a_true_miss() {
let db = db_with(blank_entry(HASH), Vec::new());
let r = KeydbSource::resolve_from(&db, &ctx("0xdeadbeef", Vec::new(), None));
assert!(!r.matched);
assert!(r.keys.is_empty());
assert!(r.shape.is_none());
assert!(r.miss_path.is_empty());
// The store size travels with the miss so the verdict can name it.
assert_eq!(r.entries_loaded, 1);
}
// Matched but no derivation material at all (no stored keys, no enc title
// keys): matched with an EMPTY miss path, which the library renders as the
// generic `no derivable key` — still distinct from a true miss.
#[test]
fn resolve_matched_with_no_material_has_empty_miss_path() {
let db = db_with(blank_entry(HASH), Vec::new());
let r = KeydbSource::resolve_from(&db, &ctx(HASH, Vec::new(), None));
assert!(r.matched);
assert!(r.keys.is_empty());
assert!(
r.miss_path.is_empty(),
"no specific reason -> library supplies NoDerivableKey"
);
let shape = r.shape.expect("matched -> shape present");
assert!(!shape.has_vuk && !shape.has_media_key && !shape.has_unit_keys);
}
// Matched entry that DOES resolve: keys present, matched, empty miss path,
// and the shape reflects the stored unit keys.
#[test]
fn resolve_matched_with_stored_unit_keys_reports_keys_and_shape() {
let mut e = blank_entry(HASH);
e.unit_keys = vec![(1, [0xA0u8; 16])];
let db = db_with(e, Vec::new());
let r = KeydbSource::resolve_from(&db, &ctx(HASH, Vec::new(), None));
assert!(!r.keys.is_empty());
assert!(r.matched);
assert!(r.miss_path.is_empty());
let shape = r.shape.expect("matched -> shape");
assert!(shape.has_unit_keys);
assert_eq!(shape.unit_keys_len, 1);
}
// KAT (a): disc with terminal Unit Keys, no enc_title_keys. Stored terminal unit keys are
// returned with CPS numbering preserved.
#[test]
fn kat_a_disc_with_unit_keys_is_terminal_and_preserves_cps_numbering() {
let mut e = blank_entry(HASH);
@@ -717,9 +886,8 @@ mod tests {
assert_eq!(committed(&got), vec![(1u32, [0xA0u8; 16])]);
}
// Orphan-unit completeness (the real keydb bug): a PARTIAL stored list
// plus a VUK that boils BOTH declared units must return BOTH, not shadow
// the VUK with the partial list. See docs/keydb.md#test-orphan-union.
// Orphan-unit completeness (the real keydb bug): a PARTIAL stored list plus a VUK that
// boils BOTH declared units must return BOTH, not shadow the VUK with the partial list.
#[test]
fn union_partial_stored_plus_vuk_yields_all_declared_units() {
let vuk = [0x5Au8; 16];
@@ -773,9 +941,8 @@ mod tests {
);
}
// KAT (c): MK + physical (unlock) VID derives MK -> VUK -> UK. The
// PHYSICAL VID must win over the keydb's stored VID. See
// docs/keydb.md#test-kat-c.
// KAT (c): MK + physical (unlock) VID derives MK -> VUK -> UK. The PHYSICAL VID must win
// over the keydb's stored VID.
#[test]
fn kat_c_disc_with_mk_uses_physical_vid_over_keydb_vid() {
let mk = [0x77u8; 16];
@@ -859,9 +1026,8 @@ mod tests {
rec
}
// KAT (f): disc with NO per-disc MK, resolved via the keydb PK pool
// (PK -> MK -> VUK -> UK) with a known-answer MKB/PK construction. See
// docs/keydb.md#test-kat-f.
// KAT (f): disc with NO per-disc MK, resolved via the keydb PK pool (PK -> MK -> VUK -> UK)
// with a known-answer MKB/PK construction.
#[test]
fn kat_f_disc_with_pk_pool_yields_uks() {
// Planted PK and the MK it resolves to (see libfreemkv boil.rs KAT).
@@ -995,6 +1161,102 @@ mod tests {
assert_eq!(KeydbSource::new("/nonexistent/keydb.cfg").label(), "keydb");
}
// KSK1: keyed by disc hash, so `resolve` asks it once per rip, not once per
// piece (KU §2.3 step 8, N-KU10).
#[test]
fn keydb_source_answer_does_not_depend_on_samples() {
assert!(!KeydbSource::new("/nonexistent/keydb.cfg").answer_depends_on_samples());
// KU J23: a keydb's VID need is per disc (its `NoVid` miss path), not blanket.
assert!(!KeydbSource::new("/nonexistent/keydb.cfg").uses_vid());
}
// A stamp with no mtime (mtime-less FS) or a future mtime (clock skew) must
// not be stuck un-settled forever — it now settles via the inode-identity
// fallback once the observation has aged past the granularity.
#[test]
fn is_settled_falls_back_when_mtime_is_absent_or_in_the_future() {
use std::time::{Duration, SystemTime};
let entry = |modified: Option<SystemTime>, stamped_at: SystemTime| CacheEntry {
stamp: FileStamp {
dev: 1,
ino: 2,
len: 3,
modified,
},
stamped_at,
db: Arc::new(KeyDb::empty()),
stats: crate::keydb_format::ParseStats::default(),
};
let gran = Duration::from_secs(2);
let now = SystemTime::now();
// None mtime, stamped well in the past → settled via the fallback.
assert!(
entry(None, now - Duration::from_secs(3600)).is_settled(gran),
"a mtime-less entry must settle once it has aged past the granularity"
);
// None mtime, just stamped → still inside the settle window, NOT settled.
assert!(
!entry(None, now).is_settled(gran),
"a freshly stamped mtime-less entry is too young to trust yet"
);
// A FUTURE mtime (clock skew) with an old stamp → still settles via the
// fallback rather than being wedged un-settled forever.
assert!(
entry(
Some(now + Duration::from_secs(3600)),
now - Duration::from_secs(3600)
)
.is_settled(gran),
"a future mtime must not wedge the entry un-settled forever"
);
// Sanity: a real past mtime still settles the normal way.
assert!(
entry(Some(now - Duration::from_secs(3600)), now).is_settled(gran),
"a normal past mtime still settles"
);
}
/// Best-first with no disc input: unannotated certs in file order, then revoked
/// ones by latest revocation. The trait ignores its MKB argument.
#[test]
fn host_certs_ranked_orders_unannotated_then_latest_revocation_first() {
let dir = scratch("ranked");
let path = dir.join("keydb.cfg");
let hc = |marker: u8, note: &str| {
format!(
"| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{}{}\n",
"00".repeat(20),
format!("{marker:02x}").repeat(92),
note
)
};
let cfg = [
hc(0x68, " ; Revoked in MKBv68"),
hc(0xA1, ""),
hc(0x72, " ; Revoked in MKBv72"),
hc(0xB2, ""),
]
.concat();
std::fs::write(&path, cfg).unwrap();
let src = KeydbSource::new(&path);
let markers =
|certs: Vec<HostCert>| -> Vec<u8> { certs.iter().map(|c| c.certificate[0]).collect() };
let want = vec![0xA1, 0xB2, 0x72, 0x68];
assert_eq!(
markers(src.host_certs()),
want,
"inherent (production credentials)"
);
assert_eq!(
markers(KeySource::host_certs(&src, Some(99))),
want,
"trait ignores mkb"
);
assert_eq!(markers(KeySource::host_certs(&src, None)), want);
std::fs::remove_dir_all(&dir).ok();
}
/// No keydb → no host credentials, not an error (inherent and trait forms).
#[test]
fn host_certs_empty_when_keydb_missing() {
@@ -1004,8 +1266,8 @@ mod tests {
assert!(KeySource::host_certs(&src, Some(68)).is_empty());
}
/// The TRAIT `host_certs` surfaces a `| HC |` row and now wires the MKB
/// generation through. Placeholder all-zero material (never a real key).
/// The TRAIT `host_certs` surfaces a `| HC |` row for any `mkb` argument.
/// Placeholder all-zero material (never a real key).
#[test]
fn trait_host_certs_returns_keydb_hc_row() {
let dir = std::env::temp_dir().join(format!("fmk_hc_{}", std::process::id()));
@@ -1027,9 +1289,9 @@ mod tests {
std::fs::remove_dir_all(&dir).ok();
}
// Caching: the 62 MiB / 181k-entry keydb is parsed ONCE. A single
// AACS-cert rip drives this source at least three times; the mtime is
// pinned into the PAST first, since an entry is only trusted once settled. See docs/keydb.md#test-repeated-lookups.
// Caching: the 62 MiB / 181k-entry keydb is parsed ONCE. A single AACS-cert rip drives this
// source at least three times; the mtime is pinned into the PAST first, since an entry is
// only trusted once settled.
#[cfg(unix)]
#[test]
fn repeated_lookups_parse_the_keydb_once() {
@@ -1058,9 +1320,8 @@ mod tests {
);
}
// The cache must not go stale: a keydb replaced UNDERNEATH the source has
// a different size/mtime stamp and must be re-read. See
// docs/keydb.md#test-changed-keydb.
// The cache must not go stale: a keydb replaced UNDERNEATH the source has a different
// size/mtime stamp and must be re-read.
#[test]
fn a_changed_keydb_file_is_reparsed() {
let dir = scratch("cache-stamp");
@@ -1087,9 +1348,9 @@ mod tests {
assert_eq!(src.parse_count(), 2, "one parse per distinct file");
}
// Two stale-cache discriminators, tested ONE AT A TIME (each test
// disables the other via `mtime_granularity`). Discriminator 1, the
// inode: an ATOMIC RENAME must be re-read even with identical length and mtime. See docs/keydb.md#test-inode-rename.
// Two stale-cache discriminators, tested ONE AT A TIME (each test disables the other via
// `mtime_granularity`). Discriminator 1, the inode: an ATOMIC RENAME must be re-read even
// with identical length and mtime.
#[cfg(unix)]
#[test]
fn a_same_length_same_mtime_rename_is_still_re_read() {
@@ -1124,9 +1385,8 @@ mod tests {
assert_eq!(src.parse_count(), 2, "the replacement must be re-parsed");
}
// Discriminator 2, the settle window: a keydb rewritten IN PLACE (same
// inode/length, mtime forced back) must still be re-read while too young
// to trust. See docs/keydb.md#test-inplace-rewrite.
// Discriminator 2, the settle window: a keydb rewritten IN PLACE (same inode/length, mtime
// forced back) must still be re-read while too young to trust.
#[cfg(unix)]
#[test]
fn an_in_place_rewrite_under_an_unsettled_stamp_is_still_re_read() {
@@ -1170,9 +1430,8 @@ mod tests {
);
}
// A corrupt-but-parseable keydb must keep saying so on EVERY hit, not
// just the parse that produced the summary (a cache hit skips `parse`).
// See docs/keydb.md#test-corrupt-warns.
// A corrupt-but-parseable keydb must keep saying so on EVERY hit, not just the parse that
// produced the summary (a cache hit skips `parse`).
#[cfg(unix)]
#[test]
fn a_corrupt_keydb_warns_on_every_lookup_not_only_on_the_parse() {
@@ -1210,9 +1469,8 @@ mod tests {
);
}
// A HEALTHY keydb must stay silent — the re-emission above must not turn
// into a per-lookup log line for every operator with an intact file. See
// docs/keydb.md#test-healthy-silent.
// A HEALTHY keydb must stay silent — the re-emission above must not turn into a per-lookup
// log line for every operator with an intact file.
#[cfg(unix)]
#[test]
fn a_healthy_keydb_warns_on_no_lookup() {
@@ -1236,9 +1494,8 @@ mod tests {
);
}
// The cost side of the settle rule, asserted as a decision: a keydb
// written just now is re-read on every lookup until quiet for the
// granularity. See docs/keydb.md#test-fresh-not-trusted.
// The cost side of the settle rule, asserted as a decision: a keydb written just now is
// re-read on every lookup until quiet for the granularity.
#[test]
fn a_freshly_written_keydb_is_not_trusted_from_cache() {
let dir = scratch("cache-fresh");
@@ -1267,9 +1524,8 @@ mod tests {
);
}
// `save()` replaces the very file this source reads, so it drops the
// cached parse ITSELF rather than trusting the (size, mtime) stamp — the
// one change the stamp cannot see. See docs/keydb.md#test-save-invalidates.
// `save()` replaces the very file this source reads, so it drops the cached parse ITSELF
// rather than trusting the (size, mtime) stamp — the one change the stamp cannot see.
#[cfg(unix)]
#[test]
fn save_invalidates_the_cache_even_when_the_file_stamp_is_unchanged() {
@@ -1303,9 +1559,8 @@ mod tests {
);
}
// A corrupt keydb is a source FAILURE, not "this disc has no key": both
// used to collapse into Ok(Vec::new()) with ZERO tracing. Only a MISSING
// file is genuinely benign. See docs/keydb.md#test-corrupt-is-error.
// A corrupt keydb is a source FAILURE, not "this disc has no key": both used to collapse
// into Ok(Vec::new()) with ZERO tracing. Only a MISSING file is genuinely benign.
#[test]
fn a_corrupt_keydb_is_an_error_not_an_empty_answer() {
let dir = scratch("corrupt");
@@ -1431,6 +1686,41 @@ mod tests {
assert!(matches!(src.save(garbage), Err(Error::KeydbInvalid)));
}
/// DK/PK/HC rows with the RIGHT prefix but MALFORMED content (bad hex, a
/// short cert) parse to nothing, so `save` must count zero real entries and
/// reject — a prefix-only count let unparseable content persist past the
/// `entries > 0` guard.
#[test]
fn save_rejects_prefixed_but_unparseable_dk_pk_hc_rows() {
let dir = scratch("save-prefixed-junk");
let src = KeydbSource::new(dir.join("k.cfg"));
// Each line has a valid keydb PREFIX but content no parser accepts: PK not
// 16 bytes of hex, DK's DEVICE_KEY not valid hex, HC cert far shorter than
// the 92-byte minimum.
let junk = b"| PK | 0xnothex\n\
| DK | DEVICE_KEY 0xZZ | DEVICE_NODE 0x0800 | KEY_UV 0x00000400 | KEY_U_MASK_SHIFT 0x17\n\
| HC | HOST_PRIV_KEY 0x00 | HOST_CERT 0x0011\n";
assert!(
matches!(src.save(junk), Err(Error::KeydbInvalid)),
"prefixed-but-unparseable rows must not count as entries"
);
// Contrast: the SAME shapes, but well-formed, ARE counted and saved.
let good = format!(
"| PK | 0x{}\n| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{}\n",
"11".repeat(16),
"00".repeat(20),
"00".repeat(92),
);
let ok = src
.save(good.as_bytes())
.expect("well-formed rows must save");
assert_eq!(
ok.entries, 2,
"one PK + one HC row are the two real entries"
);
}
// A keydb.cfg over the size cap or not valid UTF-8 must surface as a
// SOURCE FAILURE (KeydbInvalid), never a silent empty result.
#[test]
@@ -1642,4 +1932,37 @@ mod tests {
));
assert_eq!(std::fs::read_to_string(&good).unwrap(), "0xGOOD = keep\n");
}
/// Load-failure verdicts by IO kind — a keydb that can't be read must NEVER
/// masquerade as "this disc has no key". Permission-denied and corrupt/oversized
/// files surface as distinct loud errors; only a genuinely-absent file is the
/// documented benign case (online-only is a valid mode).
#[test]
fn load_failure_maps_io_kind_to_the_right_verdict() {
use std::io::{Error as IoErr, ErrorKind};
let src = KeydbSource::new("/nonexistent/keydb.cfg");
// Missing file → benign None (no keydb present; not an error on its own).
assert!(
src.load_failure(&IoErr::from(ErrorKind::NotFound))
.is_none(),
"a missing keydb is the benign online-only case"
);
// Permission denied → LOUD KeydbLoad, never a silent no-key.
assert!(
matches!(
src.load_failure(&IoErr::from(ErrorKind::PermissionDenied)),
Some(Error::KeydbLoad { .. })
),
"no read permission must surface as KeydbLoad, not 'no key'"
);
// Over the size cap / non-UTF8 → LOUD KeydbInvalid (corrupt/truncated).
assert!(
matches!(
src.load_failure(&IoErr::from(ErrorKind::InvalidData)),
Some(Error::KeydbInvalid)
),
"an unusable keydb must surface as KeydbInvalid"
);
}
}
+396 -69
View File
@@ -5,8 +5,6 @@
//! logic is identical; the only deviation is [`KeyDb::load`], which returns a
//! standalone [`std::io::Result`] here instead of `libfreemkv::error::Result`
//! (so the format crate carries no dependency on libfreemkv's error type).
//
// See docs/keydb-format.md — why the full parser API (incl. dead_code items) is retained.
#![allow(dead_code)]
use std::collections::HashMap;
@@ -17,15 +15,12 @@ use libfreemkv::aacs::types::{DeviceKey, HostCert};
/// A keydb per-disc unit key: the CPS-unit number paired with its 16-byte key.
pub type NumberedUnitKey = (u32, [u8; 16]);
// Upper bound on the on-disk keydb.cfg size accepted by `KeyDb::load` (the
// public UHD keydb is ~62 MiB; 128 MiB bounds a hostile/corrupt file).
// Mirror of keydb.rs's MAX_KEYDB_BYTES decompression cap; keep them equal.
const MAX_KEYDB_BYTES: u64 = 128 * 1024 * 1024;
/// Upper bound on parsed disc entries. The real public keydb carries
/// ~170k+ entries, so the cap sits well above that while still bounding
/// memory against a pathological input. Surplus lines are ignored.
const MAX_DISC_ENTRIES: usize = 500_000;
/// Upper bound on the keydb.cfg byte size (the public UHD keydb is ~62 MiB;
/// 128 MiB bounds a hostile/corrupt file). The SINGLE definition for the crate:
/// [`KeyDb::load`] uses it as the on-disk load cap and [`crate::KeydbSource`]
/// re-uses it as the decompression-bomb cap on the save/update path, so the two
/// can never drift out of lockstep.
pub(crate) const MAX_KEYDB_BYTES: u64 = 128 * 1024 * 1024;
/// Parsed AACS key database.
///
@@ -52,9 +47,7 @@ pub struct KeyDb {
pub disc_entries: HashMap<Arc<str>, DiscEntry>,
}
// Key material must never leak via `{:?}`; both types below carry raw AACS
// key bytes. See docs/keydb-format.md for why these hand-write a redacting
// `Debug` instead of deriving it. Pinned by `keydb_debug_is_redacted` / `disc_entry_debug_is_redacted`.
// Key material must never leak via `{:?}`; both types below carry raw AACS key bytes.
impl std::fmt::Debug for KeyDb {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
@@ -176,9 +169,7 @@ pub(crate) fn parse_hex20(s: &str) -> Option<[u8; 20]> {
libfreemkv::hex::parse_hex_fixed::<20>(s)
}
// What `KeyDb::parse` threw away, by reason. See docs/keydb-format.md — why
// this counter exists (silent corruption on a third-party keydb) and why
// it's logged once per parse rather than per-line.
// What `KeyDb::parse` threw away, by reason.
#[derive(Debug, Default, Clone, Copy, PartialEq, Eq)]
pub(crate) struct ParseStats {
/// `| DK` rows that parsed as neither a positioned nor an orphan device key.
@@ -191,8 +182,6 @@ pub(crate) struct ParseStats {
pub hc2_rejected: usize,
/// `0x… = …` disc rows the field parser refused outright.
pub disc_rejected: usize,
/// Disc rows dropped because [`MAX_DISC_ENTRIES`] was already reached.
pub disc_over_cap: usize,
/// Disc rows that REPLACED an earlier row with the same hash (last wins).
/// Not necessarily corruption, but never something to discover silently:
/// a duplicated hash means one of the two rows' keys is now unreachable.
@@ -208,7 +197,6 @@ impl ParseStats {
+ self.hc_rejected
+ self.hc2_rejected
+ self.disc_rejected
+ self.disc_over_cap
}
// Emit the one summary line, if there is anything to say; returns whether
@@ -225,7 +213,6 @@ impl ParseStats {
hc_rejected = self.hc_rejected,
hc2_rejected = self.hc2_rejected,
disc_rejected = self.disc_rejected,
disc_over_cap = self.disc_over_cap,
disc_duplicate = self.disc_duplicate,
"keydb.cfg lines were rejected while parsing; the file may be truncated or corrupt (keys past the damage will not resolve)"
);
@@ -240,6 +227,34 @@ pub(crate) fn is_disc_entry_line(line: &str) -> bool {
(line.starts_with("0x") || line.starts_with("0X")) && line.contains(" = ")
}
/// True when `line` is a row `KeyDb::parse` would actually ACCEPT into the db — a disc row OR a
/// DK/PK/HC/HC2 row that its real parser accepts, not merely one carrying the right `| XX`
/// prefix. `KeydbSource::save`'s "won't persist unparseable content" guard counts entries with
/// THIS, so a syntactically prefixed but malformed DK/PK/HC row (right prefix, bad hex/short
/// cert) can no longer inflate the entry count past the `entries > 0` check. Mirrors the
/// dispatch in [`KeyDb::parse_counted`] exactly.
pub(crate) fn is_parseable_entry_line(line: &str) -> bool {
let line = line.trim();
if line.is_empty() || line.starts_with(';') || line.starts_with('#') {
return false;
}
// Order matches `parse_counted`: HC2 before HC (both share the `| HC`
// prefix); an orphan DK (no position fields) still counts as a DK.
if line.starts_with("| DK") {
return KeyDb::parse_device_key(line).is_some() || KeyDb::parse_orphan_dk(line).is_some();
}
if line.starts_with("| PK") {
return KeyDb::parse_processing_key(line).is_some();
}
if line.starts_with("| HC2") {
return KeyDb::parse_host_cert_v2(line).is_some();
}
if line.starts_with("| HC") {
return KeyDb::parse_host_cert(line).is_some();
}
is_disc_entry_line(line) && KeyDb::parse_disc_entry(line).is_some()
}
impl KeyDb {
/// Construct an empty KeyDb. Used by unit tests; production code
/// reaches a populated KeyDb via [`KeyDb::load`] or [`KeyDb::parse`].
@@ -274,6 +289,11 @@ impl KeyDb {
disc_entries: HashMap::new(),
};
// Strip a leading UTF-8 BOM (U+FEFF): it is NOT trim()-able whitespace, so
// it would cling to line 1's `0x…` hash and silently drop the first disc
// row (a keydb saved by a Windows editor).
let data = data.strip_prefix('\u{feff}').unwrap_or(data);
for line in data.lines() {
let line = line.trim();
@@ -347,15 +367,10 @@ impl KeyDb {
continue;
}
// Disc entry: starts with 0x / 0X (see `is_disc_entry_line`).
if is_disc_entry_line(line) {
// The cap is a memory bound against a pathological file, but
// hitting it means real discs are being dropped — count it, the
// way `load`'s size/UTF-8 caps produce a diagnostic.
if db.disc_entries.len() >= MAX_DISC_ENTRIES {
stats.disc_over_cap += 1;
continue;
}
// Gate on the `0x` PREFIX alone (not `is_disc_entry_line`, which also
// needs " = "): a malformed `0x…` row is COUNTED as rejected, never
// silently dropped. No entry cap — a keydb must hold every row.
if line.starts_with("0x") || line.starts_with("0X") {
match Self::parse_disc_entry(line) {
// The map key IS the entry's own `disc_hash` allocation
// (`Arc` clone, no second copy of the string).
@@ -390,9 +405,7 @@ impl KeyDb {
Ok(db)
}
// `Self::load` from an ALREADY-OPEN file, without the log. See
// docs/keydb-format.md — why the same descriptor must supply both the
// identity stamp and the bytes, and why rejection counts are retained here.
// `Self::load` from an ALREADY-OPEN file, without the log.
pub(crate) fn load_counted(
f: std::fs::File,
path: &std::path::Path,
@@ -464,6 +477,7 @@ impl KeyDb {
/// is `None` the disc's generation is unknown and cannot be filtered, so
/// every cert is returned; certs with no revocation annotation are always
/// returned.
#[deprecated(note = "use host_certs_ranked; host cert selection takes no disc input")]
pub fn host_certs(&self, mkb: Option<u32>) -> Vec<HostCert> {
self.host_certs
.iter()
@@ -476,6 +490,19 @@ impl KeyDb {
.collect()
}
/// Every host cert, best first, with no disc input: certs with no `Revoked in
/// MKBv<N>` note in file order, then revoked ones by latest revocation (a later
/// revocation is the better bet without knowing the disc's generation).
pub fn host_certs_ranked(&self) -> Vec<HostCert> {
let mut ranked: Vec<&KeydbHostCert> = self.host_certs.iter().collect();
// Stable sort: ties (every unannotated cert, equal revocations) keep file order.
ranked.sort_by_key(|hc| {
let r = hc.revoked_at_mkb;
(r.is_some(), std::cmp::Reverse(r.unwrap_or(0)))
});
ranked.into_iter().map(|hc| hc.cert.clone()).collect()
}
/// Standalone keydb accessor: the disc's Volume ID (the keydb `I` token),
/// looked up by the same disc-hash form [`Self::find_disc`] accepts. Pure
/// file lookup; no crypto/derivation.
@@ -507,9 +534,6 @@ impl KeyDb {
/// line per disc entry (sorted by hash). `parse(to_keydb_cfg(kd))` reproduces
/// every field (see `round_trips_through_parse`). Used by the key-import tool
/// to export a complete keydb.cfg (keys + host certs + VIDs).
///
/// See docs/keydb-format.md for why the trailing `; <comment>` is only
/// emitted after a `U` (unit-keys) field.
pub fn to_keydb_cfg(&self) -> String {
fn hx(b: &[u8]) -> String {
use std::fmt::Write;
@@ -762,7 +786,6 @@ impl KeyDb {
let is_uhd = comment.contains("(UHD)");
// Title kept VERBATIM (trimmed), a faithful copy for exact round-trip.
// See docs/keydb-format.md — why display prettification does NOT belong here.
let before_fields = rest.split(" | ").next().unwrap_or("");
// A title-only entry (no key fields) carries its `;` comment on the same
// chunk — strip it so the comment doesn't leak into the title.
@@ -946,9 +969,8 @@ mod tests {
assert_eq!(b.host_certs[0].revoked_at_mkb, Some(72));
}
// REAL-DATA IDEMPOTENCE: parse → serialize (S1) → parse → serialize (S2);
// S1 must equal S2 byte-for-byte. REQUIRES a real keydb (KEYDB_PATH env);
// `#[ignore]` by default. See docs/keydb-format.md.
// REAL-DATA IDEMPOTENCE: parse → serialize (S1) → parse → serialize (S2); S1 must equal S2
// byte-for-byte. REQUIRES a real keydb (KEYDB_PATH env); `#[ignore]` by default.
#[test]
#[ignore = "needs a real keydb.cfg: KEYDB_PATH=<path> cargo test -- --ignored"]
fn to_keydb_cfg_is_idempotent_on_real_keydb() {
@@ -1006,8 +1028,6 @@ mod tests {
}
// The real keydb for the `#[ignore]`d real-scale tests, or a LOUD failure.
// See docs/keydb-format.md — why silent `return` on a missing fixture is
// wrong here.
fn real_keydb_path() -> std::path::PathBuf {
keydb_path().expect(
"KEYDB_PATH must point at a real keydb.cfg; run: \
@@ -1212,9 +1232,8 @@ mod tests {
);
}
// `load_counted` must read the HANDLE it was given, never re-resolve the
// path (identity stamp + bytes must match). See docs/keydb-format.md.
// With the path already replaced, the handle's contents must still win.
// `load_counted` must read the HANDLE it was given, never re-resolve the path (identity
// stamp + bytes must match).
#[cfg(unix)]
#[test]
fn load_counted_reads_the_handle_not_the_path() {
@@ -1345,9 +1364,9 @@ mod tests {
);
}
// The SIBLING type: `KeyDb`'s hand-written `Debug` protects the whole-db
// rendering only; `{:?}` on one `KeydbHostCert` element walks a different
// path carrying the AACS host PRIVATE key. See docs/keydb-format.md.
// The SIBLING type: `KeyDb`'s hand-written `Debug` protects the whole-db rendering only;
// `{:?}` on one `KeydbHostCert` element walks a different path carrying the AACS host
// PRIVATE key.
#[test]
fn host_cert_debug_is_redacted_including_the_sibling_wrapper() {
let hc = KeydbHostCert {
@@ -1412,12 +1431,16 @@ mod tests {
assert_eq!(stats.pk_rejected, 1, "the bad PK row must be counted");
assert_eq!(stats.hc_rejected, 1, "the short HC row must be counted");
assert_eq!(stats.hc2_rejected, 1, "the short HC2 row must be counted");
assert_eq!(stats.rejected(), 4, "four rows rejected in total");
// A `0x…` row missing " = " is malformed (a hash the user sees but that
// never resolves); it is COUNTED now, not silently ignored — exactly the
// "hash in the file but nothing resolves" bug class.
assert_eq!(
stats.disc_rejected, 1,
"a prefixed row missing \" = \" is counted"
);
assert_eq!(stats.rejected(), 5, "five rows rejected in total");
// The one GOOD row still loaded — counting must not change parsing.
assert_eq!(db.processing_keys, vec![[0xABu8; 16]]);
// A `0x` line with no " = " is not a disc row at all, so it is not a
// rejection (it is a comment-shaped line the format allows).
assert_eq!(stats.disc_rejected, 0);
}
/// A duplicated disc hash silently last-wins through `HashMap::insert`:
@@ -1438,26 +1461,132 @@ mod tests {
assert_eq!(db.get_uk("0xaaaa"), vec![(1, [0x02u8; 16])]);
}
/// Hitting `MAX_DISC_ENTRIES` drops real discs, so it must produce a
/// diagnostic like `load`'s size / UTF-8 caps do — not a bare `continue`.
/// Driven through the cap constant itself so it cannot rot if the cap moves.
/// A keydb is a database: EVERY disc row must load, with no count-based cap
/// silently dropping entries. Regression guard for the removed
/// `MAX_DISC_ENTRIES` cap — a disc verifiably in the file must resolve.
#[test]
fn parse_counts_disc_rows_dropped_over_the_entry_cap() {
fn parse_keeps_every_disc_row_no_entry_cap() {
let k = "01".repeat(16);
let mut cfg = String::with_capacity((MAX_DISC_ENTRIES + 2) * 16);
for i in 0..MAX_DISC_ENTRIES + 2 {
let n = 600_000; // comfortably past the old 500k cap
let mut cfg = String::with_capacity(n * 16);
for i in 0..n {
cfg.push_str(&format!("0x{i:x} = T | U | 1-0x{k}\n"));
}
let (db, stats) = KeyDb::parse_counted(&cfg);
assert_eq!(
db.disc_entries.len(),
MAX_DISC_ENTRIES,
"cap still enforced"
assert_eq!(db.disc_entries.len(), n, "every disc row must be kept");
assert_eq!(stats.rejected(), 0, "no row dropped");
}
// ── whole-flow robustness: every way a row a user can SEE in the file could
// silently fail to load. Each is a regression guard for a real gap. ──
/// A minimal, well-formed disc row for `hash` (lowercase hex, no 0x).
fn disc_line(hash: &str) -> String {
let uk = "01".repeat(16);
let vuk = "02".repeat(16);
format!("0x{hash} = TITLE | V | 0x{vuk} | U | 1-0x{uk}")
}
/// A UTF-8 BOM on the first line must not eat the first disc row.
#[test]
fn parse_strips_leading_bom_first_row_survives() {
let h = "aa".repeat(20);
let cfg = format!("\u{feff}{}\n", disc_line(&h));
let (db, stats) = KeyDb::parse_counted(&cfg);
assert_eq!(db.disc_entries.len(), 1, "BOM must not drop the first row");
assert!(db.find_disc(&format!("0x{h}")).is_some());
assert_eq!(stats.rejected(), 0);
}
/// CRLF (`\r\n`) line endings must load — `.lines()` + `trim()` handle the `\r`.
#[test]
fn parse_handles_crlf_line_endings() {
let h = "bb".repeat(20);
let cfg = format!("{}\r\n{}\r\n", disc_line(&h), disc_line(&"cc".repeat(20)));
let (db, _) = KeyDb::parse_counted(&cfg);
assert_eq!(db.disc_entries.len(), 2);
assert!(db.find_disc(&format!("0x{h}")).is_some());
}
/// Leading whitespace on a disc row must not drop it (trim covers it).
#[test]
fn parse_tolerates_leading_whitespace() {
let h = "dd".repeat(20);
let cfg = format!(" \t{}\n", disc_line(&h));
let (db, _) = KeyDb::parse_counted(&cfg);
assert!(db.find_disc(&format!("0x{h}")).is_some());
}
/// An uppercase `0X` prefix AND uppercase hash must load and be findable by
/// any case / prefix form (the lookup normalizes).
#[test]
fn parse_and_lookup_are_case_and_prefix_insensitive() {
let cfg = "0XABCDEF0123456789ABCDEF0123456789ABCDEF01 = T | V | 0x0202020202020202020202020202020202 | U | 1-0x0101010101010101010101010101010101\n"
.replace("0202020202020202020202020202020202", &"02".repeat(16))
.replace("0101010101010101010101010101010101", &"01".repeat(16));
let (db, _) = KeyDb::parse_counted(&cfg);
// uppercase-with-0x, lowercase-with-0x, and no-prefix all resolve.
assert!(
db.find_disc("0xABCDEF0123456789ABCDEF0123456789ABCDEF01")
.is_some()
);
assert!(
db.find_disc("0xabcdef0123456789abcdef0123456789abcdef01")
.is_some()
);
assert!(
db.find_disc("abcdef0123456789abcdef0123456789abcdef01")
.is_some()
);
}
/// A `0x…` row missing the `" = "` separator must be COUNTED as rejected,
/// never silently ignored — otherwise a visibly-present disc vanishes.
#[test]
fn parse_counts_a_prefixed_row_missing_the_separator() {
let h = "ee".repeat(20);
// no " = " (bare "=") — malformed for the grammar.
let cfg = format!(
"0x{h}=TITLE|V|0x{}|U|1-0x{}\n",
"02".repeat(16),
"01".repeat(16)
);
let (db, stats) = KeyDb::parse_counted(&cfg);
assert!(
db.find_disc(&format!("0x{h}")).is_none(),
"malformed row not loaded"
);
assert_eq!(
stats.disc_over_cap, 2,
"the two rows dropped by the cap must be counted"
stats.disc_rejected, 1,
"the drop MUST be counted, not silent"
);
assert!(
stats.log() || stats.rejected() > 0,
"a rejection is diagnosable"
);
}
/// Duplicate hash: last row wins AND the collision is counted (a keydb with
/// the same hash twice means one row's keys become unreachable).
#[test]
fn parse_counts_duplicate_hash_last_wins() {
let h = "ff".repeat(20);
let cfg = format!("{}\n{}\n", disc_line(&h), disc_line(&h));
let (db, stats) = KeyDb::parse_counted(&cfg);
assert_eq!(db.disc_entries.len(), 1);
assert_eq!(stats.disc_duplicate, 1, "the overwrite must be counted");
}
/// A row carrying ONLY terminal unit keys (no V field) still loads and is
/// findable — VUK-less discs must resolve from their stored unit keys.
#[test]
fn parse_keeps_unit_key_only_rows() {
let h = "12".repeat(20);
let cfg = format!("0x{h} = TITLE | U | 1-0x{}\n", "01".repeat(16));
let (db, _) = KeyDb::parse_counted(&cfg);
let e = db.find_disc(&format!("0x{h}")).expect("row loads");
assert!(e.vuk.is_none());
assert_eq!(e.unit_keys.len(), 1, "the stored unit key is kept");
}
/// A disc row rejected by the FIELD parser is counted separately from one
@@ -1798,6 +1927,7 @@ mod tests {
// ── Host-cert revocation: parse + host_certs(mkb) filter ────────────────
#[test]
#[allow(deprecated)]
fn host_cert_revoked_parses_and_filters_by_mkb() {
let revoked_line = format!(
"| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{} ; Revoked in MKBv72",
@@ -1827,6 +1957,7 @@ mod tests {
}
#[test]
#[allow(deprecated)]
fn host_cert_without_revocation_included_for_all_mkb() {
let line = format!(
"| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{}",
@@ -1843,6 +1974,7 @@ mod tests {
}
#[test]
#[allow(deprecated)]
fn hc2_revocation_propagates_when_hc_has_none() {
// The HC line carries no annotation; the revocation lives on the HC2
// line. The combined cert must still be filtered by that generation
@@ -2008,8 +2140,205 @@ mod tests {
assert_eq!(eb2.volume_size, None);
}
// KEYDB-parser integration tests relocated from libfreemkv; exercise the
// parser end-to-end against a real keydb.cfg. See docs/keydb-format.md.
// ── HC line: full parse into (priv[20], cert[92], revoked) ──────────────
/// A real-world-shaped `| HC |` line — a 20-byte private key, a 92-byte
/// cert that begins with the AACS host-cert header `02 01 00 5c ff ff 80 00
/// 02 10 …`, and a `; Revoked in MKBv72` note — must parse to exactly those
/// three: 20-byte key, 92-byte cert, and `revoked_at_mkb = Some(72)`.
#[test]
fn hc_line_parses_into_key_cert_and_revocation_generation() {
// Cert = 4-byte header + 6-byte host id + filler to the 92-byte length.
// Synthetic bytes only; no real key material.
let mut cert = vec![0x02u8, 0x01, 0x00, 0x5c, 0xff, 0xff, 0x80, 0x00, 0x02, 0x10];
cert.resize(92, 0xcc);
let cert_hex: String = cert.iter().map(|b| format!("{b:02x}")).collect();
let line = format!(
"| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{cert_hex} ; Revoked in MKBv72",
"88".repeat(20),
);
let hc = KeyDb::parse_host_cert(&line).expect("a well-formed HC line must parse");
assert_eq!(hc.cert.private_key, [0x88u8; 20], "20-byte private key");
assert_eq!(hc.cert.certificate.len(), 92, "AACS 1.0 cert is 92 bytes");
assert_eq!(
&hc.cert.certificate[..4],
&[0x02, 0x01, 0x00, 0x5c],
"header preserved"
);
assert_eq!(
hc.revoked_at_mkb,
Some(72),
"the ; Revoked in MKBvN note captures N"
);
// A 1.0 HC line carries no AACS 2.0 credentials.
assert!(hc.cert.private_key_v2.is_none());
assert!(hc.cert.certificate_v2.is_none());
}
/// A `| HC |` line with NO `; Revoked …` note is live: `revoked_at_mkb`
/// is `None`, so `host_certs` returns it at every generation.
#[test]
fn hc_line_without_note_is_live() {
let line = format!(
"| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{}",
"00".repeat(20),
"ab".repeat(92),
);
let hc = KeyDb::parse_host_cert(&line).unwrap();
assert_eq!(hc.revoked_at_mkb, None, "no note ⇒ never revoked");
}
// ── host_certs(mkb): the exact per-generation revocation filter ─────────
/// The hardware-critical filter: several certs revoked at DIFFERENT
/// generations plus one live cert. Each cert carries a distinct marker byte
/// so the returned SET is identified exactly, not merely counted. A cert
/// revoked at generation R is usable iff the disc's generation is `< R`.
#[test]
#[allow(deprecated)]
fn host_certs_filters_several_certs_across_mkb_generations() {
let hc = |marker: u8, note: &str| {
format!(
"| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{}{}\n",
"00".repeat(20),
format!("{marker:02x}").repeat(92),
note
)
};
let cfg = format!(
"{}{}{}{}{}",
hc(0x53, " ; Revoked in MKBv53"),
hc(0x58, " ; Revoked in MKBv58"),
hc(0x70, " ; Revoked in MKBv70"),
hc(0x72, " ; Revoked in MKBv72"),
hc(0x11, ""), // live: no revocation note
);
let db = KeyDb::parse(&cfg);
assert_eq!(db.host_certs.len(), 5);
// The first cert byte of each cert host_certs(mkb) hands back, sorted.
let markers = |mkb: Option<u32>| {
let mut m: Vec<u8> = db
.host_certs(mkb)
.iter()
.map(|c| c.certificate[0])
.collect();
m.sort_unstable();
m
};
// gen 77: every revocation has fired; only the live cert survives.
assert_eq!(markers(Some(77)), vec![0x11]);
// gen 50: below every revocation ⇒ all five are usable.
assert_eq!(markers(Some(50)), vec![0x11, 0x53, 0x58, 0x70, 0x72]);
// gen 58: revoked-at-53 and revoked-at-58 are out at the boundary
// (disc_gen < revoked is false); 70, 72 and the live cert remain.
assert_eq!(markers(Some(58)), vec![0x11, 0x70, 0x72]);
// gen 70: 70 itself is out at its own generation; only 72 + live remain.
assert_eq!(markers(Some(70)), vec![0x11, 0x72]);
// unknown disc MKB ⇒ cannot filter ⇒ every cert is returned.
assert_eq!(markers(None), vec![0x11, 0x53, 0x58, 0x70, 0x72]);
}
// ── Cert selection by the embedded 6-byte Host ID ──────────────────────
/// The AACS host ID lives INSIDE the 92-byte cert (offset 4, 6 bytes): a
/// cert beginning `02 01 00 5c ff ff 80 00 02 10 …` carries host id
/// `ff ff 80 00 02 10`. The parser keeps the cert bytes verbatim, so a
/// caller can select the cert for a specific host by that embedded id.
#[test]
fn host_cert_selected_by_embedded_6_byte_host_id() {
let cert_hex = |host_id: [u8; 6]| {
let mut bytes = vec![0x02u8, 0x01, 0x00, 0x5c]; // 4-byte header
bytes.extend_from_slice(&host_id); // host id at offset 4
bytes.resize(92, 0xcc); // pad to the 92-byte AACS 1.0 cert length
bytes.iter().map(|b| format!("{b:02x}")).collect::<String>()
};
let id_a = [0xff, 0xff, 0x80, 0x00, 0x02, 0x10];
let id_b = [0xff, 0xff, 0x80, 0x00, 0x02, 0x11];
let cfg = format!(
"| HC | HOST_PRIV_KEY 0x{p} | HOST_CERT 0x{a}\n\
| HC | HOST_PRIV_KEY 0x{p} | HOST_CERT 0x{b}\n",
p = "00".repeat(20),
a = cert_hex(id_a),
b = cert_hex(id_b),
);
let db = KeyDb::parse(&cfg);
assert_eq!(db.host_certs.len(), 2);
let find = |want: [u8; 6]| {
db.host_certs
.iter()
.find(|hc| hc.cert.certificate.get(4..10) == Some(&want[..]))
};
let hit = find(id_b).expect("the cert carrying host id …0211 must be found");
assert_eq!(&hit.cert.certificate[4..10], &id_b);
assert_eq!(hit.cert.certificate.len(), 92);
// A host id no cert carries selects nothing — and the byte-range slice
// must not panic when it does not match.
assert!(find([0x00, 0x00, 0x00, 0x00, 0x00, 0x00]).is_none());
}
/// A 1.0-only keydb (only `| HC |` rows, no `| HC2 |`) must leave every
/// cert's AACS 2.0 fields empty — no phantom v2 credentials appear.
#[test]
fn keydb_with_only_hc_rows_yields_no_v2_certs() {
let cfg = format!(
"| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{}\n\
| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{}\n",
"00".repeat(20),
"ab".repeat(92),
"01".repeat(20),
"cd".repeat(92),
);
let db = KeyDb::parse(&cfg);
assert_eq!(db.host_certs.len(), 2);
assert!(
db.host_certs
.iter()
.all(|hc| hc.cert.private_key_v2.is_none()),
"no HC2 rows ⇒ no v2 private keys"
);
assert!(
db.host_certs
.iter()
.all(|hc| hc.cert.certificate_v2.is_none()),
"no HC2 rows ⇒ no v2 certs"
);
}
/// The HC2 (AACS 2.0, type 0x11) slot: a full HC + HC2 pair parses into the
/// 1.0 fields AND the v2 fields, with the v2 private key exactly 32 bytes
/// and the v2 cert 132 bytes. Complements the field-ordering HC2 tests by
/// asserting the concrete v2 byte lengths a valid slot must yield.
#[test]
fn hc2_slot_parses_into_v2_key_and_cert_with_correct_lengths() {
let cfg = format!(
"| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{}\n\
| HC2 | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{}\n",
"00".repeat(20),
"00".repeat(92),
"5b".repeat(32),
"6c".repeat(132),
);
let db = KeyDb::parse(&cfg);
assert_eq!(db.host_certs.len(), 1, "HC2 augments the preceding HC");
let c = &db.host_certs[0].cert;
assert_eq!(c.certificate.len(), 92, "1.0 cert retained");
assert_eq!(
c.private_key_v2,
Some([0x5bu8; 32]),
"32-byte v2 private key"
);
assert_eq!(
c.certificate_v2.as_ref().map(|v| v.len()),
Some(132),
"132-byte v2 cert"
);
}
// KEYDB-parser integration tests relocated from libfreemkv; exercise the parser end-to-end
// against a real keydb.cfg.
/// REQUIRES a real keydb; `#[ignore]` by default. Run with:
/// `KEYDB_PATH=/path/to/keydb.cfg cargo test --release -- --ignored`.
@@ -2094,7 +2423,7 @@ mod tests {
for entry in &candidate_entries {
for (_, key) in &entry.unit_keys {
let mut unit = original.clone();
libfreemkv::aacs::content::decrypt_unit(&mut unit, key);
libfreemkv::test_util::decrypt_unit(&mut unit, key);
attempts += 1;
any_key_changed_the_unit |= unit != original;
if libfreemkv::aacs::content::is_clean(&unit, libfreemkv::disc::ContentFormat::BdTs)
@@ -2109,8 +2438,6 @@ mod tests {
}
// Reaching here is the EXPECTED outcome for this AACS 2.0 (BEE) sample.
// See docs/keydb-format.md for why "no key worked" alone would be an
// unusable assertion, and what is asserted instead.
assert!(attempts > 0, "no unit key was tried at all");
assert!(
any_key_changed_the_unit,
+252 -20
View File
@@ -1,11 +1,10 @@
//! Pluggable AACS key sources for libfreemkv.
//!
//! libfreemkv owns the AACS crypto; this crate provides [`KeySource`] impls
//! that look a disc up and drive the boil-down primitives to terminal Unit
//! Keys: [`KeydbSource`] (local `keydb.cfg`) and [`OnlineSource`] (remote key
//! service). Applications choose and order sources, then resolve and hand
//! the key to `Disc::decrypt_with`; compose several with [`MultiSource`].
//! See docs/lib-scope.md for the mechanism-vs-policy split with libfreemkv.
//! libfreemkv owns the AACS crypto; this crate provides [`KeySource`] impls that look a disc up
//! and drive the boil-down primitives to terminal Unit Keys: [`KeydbSource`] (local
//! `keydb.cfg`) and [`OnlineSource`] (remote key service). Applications choose and order
//! sources, then resolve and hand the key to `Disc::decrypt_with`; compose several with
//! [`MultiSource`].
mod keydb;
/// The `keydb.cfg` parser (`KeyDb`, `DiscEntry`, …). Public: parsing the keydb
@@ -17,7 +16,13 @@ mod paths;
pub use keydb::{KeydbSource, UpdateResult};
pub use keydb_format::{DiscEntry, KeyDb};
pub use online::{MIN_SAMPLE_UNITS, OnlineSource, validate_keyserver_url};
#[cfg(feature = "test-hooks")]
pub use online::set_last_decode_reachability;
pub use online::{
DecodeReachability, KeyserverUrlFault, KeyserverUrlRejection, MIN_SAMPLE_UNITS, OnlineSource,
check_keyserver_url, check_keyserver_url_static, take_last_decode_reachability,
validate_keyserver_url,
};
pub use paths::{default_keydb_path, existing_keydb_path, keydb_search_paths};
// Re-exported for downstream convenience so apps need only depend on this crate
@@ -39,13 +44,11 @@ pub(crate) fn uks_from_vuk(vuk: &[u8; 16], enc_title_keys: &[[u8; 16]]) -> Vec<U
/// An ordered composition of key sources, driven as one.
///
/// [`MultiSource::get_unit_keys`] tries each inner source in order and
/// returns the first non-empty Unit Key set (and
/// [`MultiSource::get_fmts_indexes`] does the same for the forensic set). The
/// caller supplies the list AND the order — local-first `[Keydb, Online]`,
/// online-first `[Online, Keydb]`, etc. `MultiSource` is itself a
/// [`KeySource`], so it nests and composes. See docs/lib-scope.md for the
/// `Err`-vs-`Ok(empty)` failure contract.
/// [`MultiSource::get_unit_keys`] tries each inner source in order and returns the first
/// non-empty Unit Key set (and [`MultiSource::get_fmts_indexes`] does the same for the forensic
/// set). The caller supplies the list AND the order — local-first `[Keydb, Online]`,
/// online-first `[Online, Keydb]`, etc. `MultiSource` is itself a [`KeySource`], so it nests
/// and composes.
pub struct MultiSource {
sources: Vec<Box<dyn KeySource>>,
}
@@ -57,9 +60,9 @@ impl MultiSource {
}
}
// Drive `sources` in order, returning the first non-empty result and
// preserving Ok/Err when nothing resolves. `get` selects the trait method so
// the base and forensic paths share one implementation. See docs/lib-scope.md.
// Drive `sources` in order, returning the first non-empty result and preserving Ok/Err when
// nothing resolves. `get` selects the trait method so the base and forensic paths share one
// implementation.
fn first_non_empty(
sources: &[Box<dyn KeySource>],
get: impl Fn(&dyn KeySource, &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error>,
@@ -94,6 +97,39 @@ impl KeySource for MultiSource {
first_non_empty(&self.sources, |s, c| s.get_unit_keys(c), ctx)
}
// De-conflated counterpart: first inner source with keys wins; else the
// richest miss survives composition (a MATCHED inner source beats a plain
// miss). Same Err/Ok failure contract as `get_unit_keys`.
fn resolve_unit_keys(
&self,
ctx: &dyn ResolveCtx,
) -> Result<libfreemkv::keysource::UnitKeyResolution, libfreemkv::Error> {
let mut first_failure: Option<libfreemkv::Error> = None;
let mut matched: Option<libfreemkv::keysource::UnitKeyResolution> = None;
for s in &self.sources {
match s.resolve_unit_keys(ctx) {
Ok(r) if !r.keys.is_empty() => return Ok(r),
Ok(r) => {
if r.matched && matched.is_none() {
matched = Some(r);
}
}
Err(e) => {
if first_failure.is_none() {
first_failure = Some(e);
}
}
}
}
if let Some(r) = matched {
return Ok(r);
}
match first_failure {
Some(e) => Err(e),
None => Ok(libfreemkv::keysource::UnitKeyResolution::default()),
}
}
// Forensic-index counterpart to `get_unit_keys`, same failure-preserving
// rule. A source with no forensic material (keydb, via the trait default)
// contributes empty and is skipped; today the online source answers.
@@ -101,9 +137,9 @@ impl KeySource for MultiSource {
first_non_empty(&self.sources, |s, c| s.get_fmts_indexes(c), ctx)
}
/// UNION every inner source's host certs (filtered at the given MKB
/// generation). Without this a composed source would hide an inner source's
/// cert from the OEM cert-auth route — the gap this fixes.
/// UNION every inner source's host certs, in source order (`mkb` is passed
/// through unchanged). Without this a composed source would hide an inner
/// source's cert from the OEM cert-auth route — the gap this fixes.
fn host_certs(&self, mkb: Option<u32>) -> Vec<libfreemkv::aacs::types::HostCert> {
self.sources
.iter()
@@ -114,4 +150,200 @@ impl KeySource for MultiSource {
fn label(&self) -> &'static str {
"multi"
}
// KU3-6: depends on samples if ANY inner source does, so a nested
// sample-dependent source keeps getting the per-piece ask.
fn answer_depends_on_samples(&self) -> bool {
self.sources.iter().any(|s| s.answer_depends_on_samples())
}
// Same `any(inner)` rule (KU-K1 review): `resolve` retries the composition
// whenever any inner source's last failure was transport-class.
fn last_failure_was_transport(&self) -> bool {
self.sources.iter().any(|s| s.last_failure_was_transport())
}
// KU J23: any inner source that consumes the VID makes the composition one.
fn uses_vid(&self) -> bool {
self.sources.iter().any(|s| s.uses_vid())
}
}
#[cfg(test)]
mod tests {
use super::*;
use libfreemkv::aacs::types::HostCert;
// A minimal KeySource that only supplies host certs — the sole MultiSource
// behaviour under test. It never resolves a unit key.
struct CertOnly {
certs: Vec<HostCert>,
}
impl KeySource for CertOnly {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
Ok(Vec::new())
}
fn host_certs(&self, _mkb: Option<u32>) -> Vec<HostCert> {
self.certs.clone()
}
}
// A host cert tagged with a distinct marker byte so a returned set can be
// identified exactly. Synthetic bytes only; no real key material.
fn cert(marker: u8) -> HostCert {
HostCert {
private_key: [marker; 20],
certificate: vec![marker; 92],
private_key_v2: None,
certificate_v2: None,
}
}
fn boxed(certs: Vec<HostCert>) -> Box<dyn KeySource> {
Box::new(CertOnly { certs }) as Box<dyn KeySource>
}
/// `MultiSource::host_certs` must UNION every inner source's certs. Without
/// this a composed source hides an inner source's cert from the OEM
/// cert-auth route — the gap the union closes.
#[test]
fn multi_source_host_certs_unions_every_inner_source() {
let multi = MultiSource::new(vec![
boxed(vec![cert(0x01)]),
boxed(vec![cert(0x02), cert(0x03)]),
]);
let mut markers: Vec<u8> = multi
.host_certs(Some(50))
.iter()
.map(|c| c.certificate[0])
.collect();
markers.sort_unstable();
assert_eq!(
markers,
vec![0x01, 0x02, 0x03],
"certs from every inner source must be unioned"
);
}
/// With no inner source holding a cert, the union is empty (not an error).
#[test]
fn multi_source_host_certs_empty_when_no_source_has_one() {
let multi = MultiSource::new(vec![boxed(vec![]), boxed(vec![])]);
assert!(multi.host_certs(None).is_empty());
}
// A source whose `answer_depends_on_samples` is fixed to the given value —
// the only behaviour KSK2 exercises.
struct FixedDependency(bool);
impl KeySource for FixedDependency {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
Ok(Vec::new())
}
fn answer_depends_on_samples(&self) -> bool {
self.0
}
}
// KSK2: `MultiSource::answer_depends_on_samples` is `any(inner)`, so nesting
// a sample-dependent online source under a sample-independent keydb still
// gets the per-piece ask (`resolve` must not treat the pair as keydb-only).
#[test]
fn multi_source_answer_depends_on_samples_is_any_inner() {
let all_independent = MultiSource::new(vec![
Box::new(FixedDependency(false)) as Box<dyn KeySource>,
Box::new(FixedDependency(false)) as Box<dyn KeySource>,
]);
assert!(!all_independent.answer_depends_on_samples());
let keydb_then_online = MultiSource::new(vec![
Box::new(FixedDependency(false)) as Box<dyn KeySource>,
Box::new(FixedDependency(true)) as Box<dyn KeySource>,
]);
assert!(keydb_then_online.answer_depends_on_samples());
}
// A source whose `uses_vid` is fixed to the given value.
struct FixedVid(bool);
impl KeySource for FixedVid {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
Ok(Vec::new())
}
fn uses_vid(&self) -> bool {
self.0
}
}
// KU J23: like `answer_depends_on_samples`, a composed source uses the VID if any
// inner one does, so a nested online source still makes a Missing piece VID-derivable.
#[test]
fn multi_source_uses_vid_is_any_inner() {
let neither = MultiSource::new(vec![
Box::new(FixedVid(false)) as Box<dyn KeySource>,
Box::new(FixedVid(false)) as Box<dyn KeySource>,
]);
assert!(!neither.uses_vid());
let keydb_then_online = MultiSource::new(vec![
Box::new(FixedVid(false)) as Box<dyn KeySource>,
Box::new(FixedVid(true)) as Box<dyn KeySource>,
]);
assert!(keydb_then_online.uses_vid());
}
// A source whose `last_failure_was_transport` is fixed to the given value.
struct FixedTransport(bool);
impl KeySource for FixedTransport {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
Ok(Vec::new())
}
fn last_failure_was_transport(&self) -> bool {
self.0
}
}
// For consistency with `answer_depends_on_samples` (KU3-6): a composed
// source's last-failure verdict is `any(inner)` too, so `resolve` retries
// the composition whenever ANY inner source's last failure was transport.
#[test]
fn multi_source_last_failure_was_transport_is_any_inner() {
let none_transport = MultiSource::new(vec![
Box::new(FixedTransport(false)) as Box<dyn KeySource>,
Box::new(FixedTransport(false)) as Box<dyn KeySource>,
]);
assert!(!none_transport.last_failure_was_transport());
let one_transport = MultiSource::new(vec![
Box::new(FixedTransport(false)) as Box<dyn KeySource>,
Box::new(FixedTransport(true)) as Box<dyn KeySource>,
]);
assert!(one_transport.last_failure_was_transport());
}
/// The `mkb` generation must reach each inner source so its OWN revocation
/// filter can act; a source that drops everything at a given generation is
/// honoured by the union.
#[test]
fn multi_source_host_certs_passes_mkb_through_to_inner_sources() {
struct GenAware;
impl KeySource for GenAware {
fn get_unit_keys(
&self,
_c: &dyn ResolveCtx,
) -> Result<Vec<UnitKey>, libfreemkv::Error> {
Ok(Vec::new())
}
fn host_certs(&self, mkb: Option<u32>) -> Vec<HostCert> {
match mkb {
// Modelled on a cert revoked in MKBv72.
Some(g) if g >= 72 => Vec::new(),
_ => vec![cert(0xaa)],
}
}
}
let multi = MultiSource::new(vec![Box::new(GenAware) as Box<dyn KeySource>]);
assert_eq!(multi.host_certs(Some(50)).len(), 1, "usable below gen 72");
assert!(
multi.host_certs(Some(72)).is_empty(),
"the gen must reach the inner source's filter"
);
}
}
+1845 -304
View File
File diff suppressed because it is too large Load Diff
+2 -3
View File
@@ -4,9 +4,8 @@
//! just reads a handed-in path. This module gives the *search* list (first
//! existing wins) and the single *default* write location.
//!
//! freemkv is portable: `keydb.cfg` lives next to the exe —
//! `<dir of current exe>/keydb.cfg` — with no OS config-dir fallback.
//! `--keydb PATH` bypasses this module. See docs/paths.md for rationale.
//! freemkv is portable: `keydb.cfg` lives next to the exe — `<dir of current exe>/keydb.cfg` —
//! with no OS config-dir fallback. `--keydb PATH` bypasses this module.
use std::path::PathBuf;
+10 -14
View File
@@ -138,7 +138,7 @@ fn keydb_source_label_is_keydb() {
assert_eq!(KeydbSource::new("/nonexistent/keydb.cfg").label(), "keydb");
}
// ── KeydbSource: MKB-aware host-cert serving from a file ────────────────────
// ── KeydbSource: host-cert serving from a file ──────────────────────────────
#[test]
fn keydb_source_serves_host_cert_from_hc_row() {
@@ -152,14 +152,13 @@ fn keydb_source_serves_host_cert_from_hc_row() {
let path = s.write("keydb.cfg", &line);
let src = KeydbSource::new(&path);
// Inherent (no-MKB, scan-options) form.
// Inherent (scan-options) form.
assert_eq!(
src.host_certs().len(),
1,
"inherent host_certs sees the HC row"
);
// Trait form now wires the MKB generation through (no revocation annotation
// → always returned).
// Trait form: `mkb` is ignored; the ranked list is returned.
let via_trait = KeySource::host_certs(&src, Some(70));
assert_eq!(via_trait.len(), 1, "trait host_certs sees the HC row");
assert_eq!(via_trait[0].certificate.len(), 92);
@@ -229,13 +228,14 @@ fn online_source_metadata() {
}
#[test]
fn validate_keyserver_url_gates_scheme_and_ssrf() {
fn validate_keyserver_url_gates_scheme_and_allows_lan() {
assert!(validate_keyserver_url("https://8.8.8.8/keys").is_ok());
// Home app: LAN/loopback/link-local hosts pass; cleartext http and invalid addresses do not.
assert!(validate_keyserver_url("https://127.0.0.1/keys").is_ok());
assert!(validate_keyserver_url("https://169.254.1.5/keys").is_ok());
assert!(validate_keyserver_url("https://[::1]:9000/keys").is_ok());
assert!(validate_keyserver_url("http://127.0.0.1/keys").is_err());
// SSRF blocking is IP-based, not scheme-gated: https://<private-IP> is rejected too.
assert!(validate_keyserver_url("https://127.0.0.1/keys").is_err());
assert!(validate_keyserver_url("http://169.254.169.254/latest/meta-data/").is_err());
assert!(validate_keyserver_url("http://[::1]:9000/keys").is_err());
assert!(validate_keyserver_url("https://0.0.0.0/keys").is_err());
assert!(validate_keyserver_url("ftp://example.com/keys").is_err());
assert!(validate_keyserver_url("").is_err());
}
@@ -368,9 +368,7 @@ fn multi_source_real_keydb_resolves_through_chain() {
);
}
// ── MultiSource: a source that could not ANSWER is not a source that said no ─
// See docs/multi-source-failure-collapse.md — the Ok/Err collapse incident
// this section's tests exist to catch.
// ── MultiSource: a source that could not ANSWER is not a source that said no ─.
/// A source that always fails, with a chosen error — the key-service outage.
struct FailingSource {
@@ -398,8 +396,6 @@ fn unauthorized() -> libfreemkv::Error {
libfreemkv::Error::KeyServiceUnauthorized
}
// See docs/multi-source-failure-collapse.md — must report FAILURE, not a
// clean "no key", when no source holds a key and one could not answer.
#[test]
fn multi_source_reports_a_source_failure_instead_of_a_clean_no_key() {
let multi = MultiSource::new(vec![