Compare commits

...
179 Commits
Author SHA1 Message Date
Matthew Jackson 36e9d93525 Move tests to side files 2026-10-05 20:22:40 -07:00
Matthew Jackson d2f6bbd272 v1.8.0: bump version (unified release) 2026-10-05 02:17:02 +00:00
Matthew Jackson 8c2d9be19a changelog: 1.8.0 release notes 2026-10-04 18:28:24 -07:00
Matthew Jackson 473f629d13 ci: monthly grouped dependabot; actions/checkout 7
Dependabot runs monthly with grouped version updates.
actions/checkout 5 -> 7 in ci, qa and release (#13).
2026-10-03 11:16:01 -07:00
Matthew Jackson 1662345b7e online: halt and liveness wiring, forensic fallback, LAN key-service hosts
The query worker honours ctx.halt() and reports liveness (following
libfreemkv's Progress -> Liveness rename). A definitive key-service miss
allows the forensic phase fallback. LAN, loopback and link-local hosts
are allowed, vetted with libfreemkv's is_blocked_ip.
CI path-patches freemkv-unlock and runs cargo-deny from the repo dir;
Stop tests use idle bounds instead of tight wall-clock limits.
2026-10-01 18:06:06 -07:00
Matthew Jackson 0ec14fc338 keysources: sources report VID use, sample dependence and transport failures
keydb and MultiSource say whether an answer depends on samples; online
failures are classed as transport or not without erasing the
reachability slot; OnlineSource and MultiSource report uses_vid, and
partial stored keys plus a Km still report NoVid. Factory builds get a
DNS-free key-service URL check. Tests decrypt through libfreemkv's
test_util.
2026-09-28 07:18:27 -07:00
Matthew Jackson 520df9ebe3 online: stall-only key-service timeouts; a stopped query sends nothing
Key-service timeouts measure stalls rather than totals, and a Stop ends
a query mid-flight. A query stopped before it starts sends nothing,
builds no body and reads no disc material.
2026-09-27 23:50:33 -07:00
Matthew Jackson 5bf4afce0b online, keydb: no env proxy; typed keyserver URL verdict; host certs best first
The key service never uses an environment proxy: it was resolved to the
service's pinned address, so every lookup failed. Keyserver URLs get a
typed verdict, and a test-hooks feature lets consumers' tests set the
last reachability. keydb hands out host certs best first, with no disc
input.
2026-09-27 17:24:04 -07:00
Matthew Jackson d0a3ab852b v1.7.7: bump version (unified release) 2026-09-26 21:16:59 +00:00
Matthew Jackson 91b906fa61 Prepare 1.7.7 with concise comments and shared CI policy 2026-09-26 11:46:35 -07:00
Matthew Jackson e987506d96 v1.7.6: bump version (unified release) 2026-09-26 04:05:43 +00:00
Matthew Jackson 60cff0ead6 chore: absorb qa into dev for lockstep promote [skip-media] 2026-09-25 12:25:47 -07:00
Matthew Jackson c333c6000b v1.7.6: bump version (unified release) 2026-09-25 05:32:01 +00:00
Matthew Jackson adf2401c06 chore(qa): gate 1.7.6 promote — media matrix not needed [skip-media]
The 1.7.6 unified release ships no rip-path change (Linux desktop shell,
notifications, menu_layout refactor, XDG paths). Real-media QA on EC2
adds ~2h + real money to the release cycle to prove code that never
enters the rip pipeline. This empty commit puts `[skip-media]` in qa's
HEAD so the media matrix in qa.yml auto-skips per the documented bypass;
the full-platform builds/tests still run.
2026-09-24 21:37:29 -07:00
Matthew Jackson b89b7c9660 chore(qa-sync): absorb qa's post-1.7.5 release commits into dev
Release-orchestrate committed to qa (v1.7.5 version bump, etc.) but did
not push those commits back to dev, so promote's fast-forward gate
refused (qa was diverged from dev). This is an ours-merge: the commit
records qa as an ancestor of dev without applying its content, since
dev has already moved past 1.7.5 to 1.7.6 with the equivalent
information in place.
2026-09-24 21:21:11 -07:00
Matthew Jackson 1121522534 chore(release): bump to 1.7.6, follow the CI-required release-notes shape
Bumps Cargo.toml + Cargo.lock to 1.7.6. Also fixes a CI-shape defect the
previous release left behind: dev's CHANGELOG still said
`## [1.7.5] — UNRELEASED` after 1.7.5 shipped (release-orchestrate dated
main but did not push a fresh UNRELEASED heading back to dev). Corrects
that here so the changelog job accepts 1.7.6 on the way to qa:

  ## [1.7.6] — UNRELEASED
    - Version aligned; unified release driven by freemkv 1.7.6.
  ## [1.7.5] — 2026-09-23
    - Version aligned (or actual content, per the shipped main entry).

The driver for 1.7.6 is the freemkv Linux desktop shell (GTK4 +
libadwaita) + rip-finished desktop notification (issue #56).
2026-09-24 20:23:08 -07:00
Matthew Jackson 146f53873d v1.7.5: bump version (unified release) 2026-09-23 04:40:56 +00:00
Matthew Jackson b925a4fa50 docs: 1.7.5 changelog entry for the unified release [skip-media]
Version aligned to 1.7.5. No functional change to this crate — the release
is driven by freemkv-unlock mirroring the freemkv-firmware 0.9.0 ABI, where
the drive's separate Ake (0x06) and Bus (0x07) levers are retired into a
single Encryption (0x06) lever.
2026-09-22 21:05:01 -07:00
Matthew Jackson 4f169a4881 v1.7.4: bump version (unified release) 2026-09-21 23:03:38 +00:00
Matthew Jackson e251e0bf6f docs: mark 1.7.4 changelog heading UNRELEASED (release workflow dates it) 2026-09-21 15:07:11 -07:00
Matthew Jackson 6704c1e05b docs: 1.7.4 changelog + consumer release notes 2026-09-21 11:47:20 -07:00
Matthew Jackson a34980970b docs/ci: drop stale local-precommit/release.sh references — CI is the only gate 2026-09-21 09:11:35 -07:00
Matthew Jackson 948511853e test(online): trim interpret_reply reachability aside to 3 lines for comment-guard 2026-09-21 08:51:04 -07:00
Matthew Jackson 373cdc4cf2 ci: adopt central freemkv/.github guards + checkout-siblings composite
Thin-caller comment/leak/identity guards → freemkv/.github reusable
workflows; drop local ci/comment-guard.py and ci/leak-guard.sh. Replace
per-job sibling checkout + .cargo/config heredoc with the shared
checkout-siblings@v1 composite (checkout libfreemkv+freemkv-unlock,
patch libfreemkv). Add cancel-in-progress concurrency to ci.yml/qa.yml;
standardize actions/checkout to @v5.
2026-09-21 08:48:58 -07:00
Matthew Jackson a60b021c35 Merge: ureq 3.4.2 + bounded reply body (supersedes Dependabot #12) 2026-09-20 20:15:19 -07:00
Matthew Jackson fc8b6e3a05 bump ureq 3.4.2 + preserve rolling-idle body timeout
ureq 3.4.1 (#1194, "Fix timeout budgets restarting and applying to later
phases") made timeout_recv_response bound only the response HEADERS and
turned timeout_recv_body into an ABSOLUTE total-body deadline. The online
key-service POST set recv_response only, so under 3.4.2 the reply body read
had NO deadline at all — a server that sent headers then stalled mid-reply
would hang the key request forever.

The /decode reply is a small JSON key answer that arrives in one read (a
few hundred bytes; the 1 MiB cap is a defensive anti-OOM ceiling, not an
expected size), so this is a fine total-cap case, not a large-body slow-link
download: no rolling-idle connector is warranted. Set timeout_recv_body to
the same TIMEOUT_SECS total ceiling so the POST can never hang forever.

Regression guard the_reply_body_read_is_bounded_not_only_the_headers asserts
recv_body is now set. Full suite + precommit (fmt/clippy/deny/tests) green.
2026-09-20 20:03:54 -07:00
Matthew Jackson bdbc805ba5 keydb: report match state + entry shape to de-conflate "no entry" (#46)
Implement KeySource::resolve_unit_keys for KeydbSource by splitting
unit_keys_from into resolve_from, which reports: whether find_disc
matched; the specific miss reason (NoVid when a Media Key has no VID,
else NoDerivableKey); the matched entry's shape (vuk / unit_keys count /
media_key / keydb_vid / enc_title_keys len / vid_available); and the
loaded entry count for a self-diagnosing true-miss verdict.

MultiSource forwards resolve_unit_keys so a matched inner source's richer
verdict survives composition. No derivation-logic change: the locked
VID-per-path rule is intentional and the investigation found no
correctness bug there — only the conflated verdict. Tests cover
matched-no-VID, matched-no-material, true miss, and the success path.
2026-09-20 19:16:06 -07:00
Matthew Jackson fa75d28441 freemkv-keysources 1.7.4: keydb loads every entry — drop entry cap, strip BOM, count malformed rows
- Remove the MAX_DISC_ENTRIES cap: a keydb is a database and must hold every
  disc row it contains (a count-based drop silently lost real discs).
- Strip a leading UTF-8 BOM before parsing so a Windows-saved keydb's FIRST
  disc row survives (U+FEFF is not trim()-able whitespace).
- Gate the disc branch on the 0x/0X prefix alone and COUNT a malformed 0x row
  (missing " = ") as rejected instead of silently ignoring it — surfaces the
  "hash is in the file but nothing resolves" class of bug.
- Tests: BOM, CRLF, leading whitespace, case/prefix-insensitive lookup,
  duplicate-hash last-wins, unit-key-only rows, 600k-entry no-drop, and the
  load_failure verdicts (permission / invalid / missing).

[skip-media]
2026-09-20 09:43:41 -07:00
Matthew Jackson 5d2e11a4dd freemkv-keysources 1.7.4: version bump (unified 1.7.4) 2026-09-19 14:03:36 -07:00
Matthew Jackson 50bd0e4be2 v1.7.3: bump version (unified release) 2026-09-19 05:31:15 +00:00
Matthew Jackson 148419c2bb changelog: 1.7.3 unified-release entry (freemkv --share + libfreemkv multi-angle UHD fix) 2026-09-18 22:18:41 -07:00
Matthew Jackson 1b97e6186d merge: record v1.7.2 release into dev (-s ours; keep path deps + dev's next-cycle work) 2026-09-18 21:39:15 -07:00
Matthew Jackson 23988922eb v1.7.2: bump version (unified release) 2026-09-18 18:37:25 +00:00
Matthew Jackson e4ecc3cf83 merge: record v1.7.1 main into dev (dev already at 1.7.1; keep path deps) 2026-09-17 23:08:12 -07:00
Matthew Jackson 55ef44e74e comment-guard: trim over-cap comment blocks to caps (comments only)
Restructure 10 over-cap comment blocks in keydb.rs and online.rs to fit
comment-guard's caps (inline // asides <=3 lines, /// item-doc prose <=8)
by tightening wording. No code changes; behavior unchanged.
2026-09-17 22:37:38 -07:00
Matthew Jackson 173a532a6e deps: revert ureq to 3.4.0 (3.4.2 body-deadline regression; keep the rest of this repo's dep changes) 2026-09-17 22:29:32 -07:00
Matthew Jackson 6879df55eb deps: rustls 0.23.43→0.23.45 (RUSTSEC TLS-1.3 advisory) + ureq 3.4.2 2026-09-17 22:18:07 -07:00
Matthew Jackson 0a735498fb online: block NAT64 well-known prefix in SSRF guard; fix https-only doc 2026-09-15 14:06:04 -07:00
Matthew Jackson 531e3a40bf fix(keysources): resolve own-repo audit findings (v1.6.0->dev)
online.rs:
- validate_keyserver_url/resolve_and_guard now reject any non-https:// scheme
  as a Config fault, so a cleartext misconfig is caught ONCE at config time
  instead of failing every rip via query()'s runtime https check.
- is_blocked_ip: add 198.18.0.0/15 (RFC 2544 benchmarking) and 192.0.0.0/24
  (RFC 6890 IETF protocol assignments, incl. 192.0.0.170/171) to the SSRF guard.
- DNS in-flight cap is now PER HOST, not process-global, so black-holed lookups
  to one dead keyserver can't starve resolution of a healthy different keyserver.
- reject a non-empty, non-":port" tail after a bracketed IPv6 authority
  (e.g. https://[::1]extra/) instead of silently dropping it.

keydb.rs / keydb_format.rs:
- MAX_KEYDB_BYTES defined once in keydb_format and re-used by keydb.rs.
- CacheEntry::is_settled falls back to inode identity + stamped_at age when the
  mtime is absent (mtime-less FS) or in the future (clock skew), so the ~62 MiB
  keydb no longer re-parses on every lookup.
- save()'s entry-count guard counts DK/PK/HC/HC2 rows via the real parsers
  (is_parseable_entry_line), so a prefixed-but-malformed row can't pass entries>0.

Tests added for each fix.
2026-09-15 10:30:50 -07:00
Matthew Jackson ae8e5ade3c fix(online): record decode POST reachability for the caller to classify
OnlineSource now records the HTTP outcome of its actual /decode POST — a
Status(code) for any answer (200/404/422/429/5xx) or Transport for a
connect/timeout/DNS/TLS failure — in a per-thread slot, readable via the new
take_last_decode_reachability() / DecodeReachability. This lets a caller
(autorip) classify a no-key result from the REAL decode instead of firing a
second, redundant probe against the POST-only endpoint. No change to key
resolution itself. Targets 1.7.2.
2026-09-14 14:55:12 -07:00
Matthew Jackson 8b914e7121 v1.7.1: bump version (unified release) 2026-09-14 03:34:52 +00:00
Matthew Jackson 62ebd61ff1 docs(changelog): 1.7.1 release notes (LibreDrive unlock fix) 2026-09-13 20:01:03 -07:00
Matthew Jackson 30c16bc415 ci: bump actions/checkout v4->v7; deps: ureq 3.4.0->3.4.1 (dependabot) 2026-09-13 18:30:58 -07:00
Matthew Jackson 8c3afbee67 chore: bump version to 1.7.1 (unified release lockstep) 2026-09-13 18:21:52 -07:00
Matthew Jackson 6e4f05dec4 test: assert full key bytes; correct keydb-format docs
Several tests asserted only key[0] (the first byte), which a byte-transposition bug could pass — assert the full 16-byte key everywhere, including a KAT that recomputes the expected VUK-derived keys directly. test_parse_disc_entry used identical all-zero placeholders for M/I/V/U, which couldn't catch a field mix-up; give each field a distinct byte pattern and assert every field.

Docs: keydb-format.md claimed DiscEntry::title was genuinely unused dead code, but it's read by to_keydb_cfg and the redacting Debug impl — correct the comment to only list empty/find_vuk as truly unused. Also cross-reference the two independent MAX_KEYDB_BYTES constants (keydb.rs's decompression cap and keydb_format.rs's on-disk load cap) so they're kept in sync, and bump Cargo.lock's version metadata.
2026-09-05 12:38:16 -07:00
Matthew Jackson 5bf6c4875b log disc-side read failures; skip a bogus CPS unit 0
Two disc-side reads were silently swallowed with unwrap_or(&[]): a failure reading the disc's encrypted title keys or its MKB looked identical to a disc that genuinely had none, hiding real I/O/parse errors from the operator. Log a warning (mirroring online.rs's existing pattern) before falling back to empty in both cases.

Also, a stored unit key numbered 0 (an invalid CPS unit number; valid numbering is >= 1) was mapped via num.saturating_sub(1) onto idx 0, silently colliding with and shadowing a legitimate unit 1. Skip unit number 0 instead of mis-mapping it, and add a regression test.
2026-09-05 12:38:06 -07:00
Matthew Jackson 1c29377780 parse the keydb outside the cache lock
cached_db held self.cache locked across the full KeyDb::load_counted parse of the ~62 MiB keydb, so a reparse (triggered by a changed mtime) stalled every other worker waiting on the cache. Take a brief lock for the fast-path settled-cache-hit check, release it, parse outside the lock, then re-acquire and double-check: if a peer installed the same settled stamp while we parsed, adopt their entry and drop our redundant parse rather than racing a lost update. Stamp and bytes still come from the one already-open file handle, preserving the one-open-one-identity invariant.
2026-09-05 12:37:59 -07:00
Matthew Jackson 7c544e3780 cap outstanding DNS-resolver threads
Each DNS lookup for the SSRF guard spawns a thread that is never joined and can hang for the full OS resolver timeout, so a black-holed keyserver leaks one thread+stack per attempt. Track outstanding resolver threads with an atomic counter; over a cap of 4, report the host as unreachable instead of spawning another. Decrement only when the (possibly long-hung) lookup actually returns, so the cap reflects threads truly in flight.
2026-09-05 12:37:53 -07:00
Matthew Jackson 4a241e04bc block 6to4/Teredo tunnels in the SSRF guard
The IPv6 SSRF guard already re-checked IPv4-mapped/-compatible addresses, but 6to4 (2002::/16) and Teredo (2001:0000::/32) also tunnel an embedded IPv4 and were passed through untouched, letting an attacker wrap 127.0.0.1 or a cloud metadata address in either encapsulation to bypass the block list. Decode both forms and re-check the embedded IPv4 through is_blocked_ip. Add a test covering 6to4/Teredo-wrapped loopback and metadata addresses, plus a 6to4-wrapped public IP that must still be allowed.
2026-09-05 12:37:48 -07:00
Matthew Jackson d19481ae59 persist keydb.cfg at mode 0600 (it holds key material)
keydb.cfg carries AACS key material plus the host private key/cert. write_atomic renamed a File::create()'d temp file onto it, so the final mode depended on umask and could leave the keys world-readable. On Unix, open the temp file with mode 0600 explicitly via OpenOptionsExt; non-Unix keeps the plain create().
2026-09-05 12:37:43 -07:00
Matthew Jackson 707c8bdc57 ci: identity-guard — block unrelated-corp / non-canonical committer emails 2026-09-02 14:46:19 -07:00
Matthew Jackson 599422ef22 release: 1.7.0
CI/lint hardening since v1.6.14: clippy on stable with the MSRV single-sourced
from Cargo.toml, cargo-deny dependency-audit gate with first-party git sources
allowed (#15), and audience-based comment-guard (#17). No library code changes.
2026-09-02 12:27:48 -07:00
Matthew Jackson ed5066af48 v1.6.14: bump version (unified release) 2026-08-31 23:04:30 +00:00
Matthew Jackson 9b28a7c89d chore: reconcile v1.6.13 release commit from main into dev 2026-08-31 15:48:53 -07:00
Matthew Jackson 8a8397965b docs(changelog): 1.6.14 entry 2026-08-31 15:32:40 -07:00
Matthew Jackson a52698b6bf chore(msrv): floor to 1.88, pin CI on the floor, allow uninlined_format_args
Declared rust-version and the ci.yml/qa.yml toolchain pins both sit at 1.88 (the
edition-2024 floor once zip 8 and let-chains are accounted for), so CI builds on
exactly the MSRV: an accidental bump to a newer toolchain fails CI by design and
moving the floor takes a deliberate pin edit. The uninlined_format_args allow
reconciles 1.88's clippy with the allow-by-default it became in 1.97+.
2026-08-31 11:06:35 -07:00
dependabot[bot] 89e58d716b build(deps): bump codecov/codecov-action from 5 to 7
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5 to 7.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v5...v7)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-30 21:52:43 -07:00
Matthew Jackson bf18ec0cec v1.6.13: bump version (unified release) 2026-08-28 15:49:04 +00:00
Matthew Jackson 7e9849ccbb Release 1.6.13: version align 2026-08-27 21:22:11 -07:00
Matthew Jackson 94f082a43b v1.6.12: bump version (unified release) 2026-08-27 22:57:36 +00:00
Matthew Jackson 1fbf3f6c63 changelog: 1.6.12 (unreleased) 2026-08-27 12:18:41 -07:00
Matthew Jackson b75599a5cb v1.6.11: bump version (unified release) 2026-08-25 23:17:15 -07:00
Matthew Jackson e83b7a3489 test: raise coverage across keydb, keydb_format, online, lib sources
Adds targeted unit tests for previously-uncovered branches: keydb load
failure classification (invalid-UTF-8 / unreadable path), zip extraction
success/no-cfg-member paths, write_atomic create/rename failure modes,
keydb_format hex-digit-parsing edge cases, iter_disc_entries, to_keydb_cfg
HC2 emission and empty-title/comment-field combinations, MultiSource::label,
and the online key-service client's SSRF-guard malformed-authority and
unresolvable-host paths, UK-as-string and VUK-derivation reply handling,
and get_fmts_indexes. Local line coverage moves from ~90.3% to ~92.1%.
2026-08-25 22:11:01 -07:00
Matthew Jackson 4ef4d5dfd6 docs: 1.6.11 changelog 2026-08-25 22:08:48 -07:00
Matthew Jackson febf4d0504 docs: standardize README badges (License, CI, codecov) 2026-08-25 20:32:34 -07:00
Matthew Jackson 5e96e3e9d8 ci(coverage): authenticate Codecov upload with org CODECOV_TOKEN 2026-08-25 19:32:39 -07:00
Matthew Jackson 2415a58d5d ci(coverage): add --ignore-run-fail so instrumentation flake never blocks upload 2026-08-25 19:04:51 -07:00
Matthew Jackson edb14d3aea ci: add non-required Codecov coverage job, config, and README badge
Add an additive coverage job to ci.yml that mirrors the test job's
sibling-checkout layout and pinned 1.97.0 toolchain, runs cargo llvm-cov,
and uploads to Codecov tokenlessly (fail_ci_if_error:false). Nothing in
the release cascade depends on it. Also add codecov.yml (informational,
no PR comment) and a README with the coverage badge.
2026-08-25 18:55:26 -07:00
dependabot[bot] 29d67e803d build(deps): bump actions/checkout from 5 to 7 (#8)
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-25 10:04:24 -07:00
Matthew Jackson ce96676389 CHANGELOG: 1.6.10 (unified release; driven by libfreemkv TrueHD discontinuity fix) 2026-08-23 16:30:29 -07:00
Matthew Jackson df92b830bd v1.6.9: bump version (unified release) 2026-08-22 08:34:34 -07:00
Matthew Jackson b31cdc2519 changelog: 1.6.9 unified-release entry 2026-08-22 08:25:03 -07:00
Matthew Jackson 397a8a2960 v1.6.8: bump version (unified release) 2026-08-21 12:54:57 -07:00
Matthew Jackson 1306901662 docs: 1.6.8 changelog (unified release) 2026-08-21 12:47:34 -07:00
Matthew Jackson d41e200e56 ci: require a CHANGELOG entry for the crate version on main
Adds a 'changelog' job that fails when Cargo.toml's version has no matching
'## [X.Y.Z]' heading in CHANGELOG.md, hard on main and PRs into main, warning
on dev/qa — so the unified release can't ship a crate with a blank changelog.
2026-08-21 11:23:46 -07:00
Matthew Jackson 44f963cf1b v1.6.7: bump version (unified release) 2026-08-21 09:31:44 -07:00
Matthew Jackson acbf5929fc docs: 1.6.7 changelog (unified release, autorip-driven) 2026-08-21 09:23:55 -07:00
Matthew Jackson 38ccf5eaba v1.6.6: bump version (unified release) 2026-08-20 20:03:52 -07:00
Matthew Jackson 73b1de1de0 docs: 1.6.6 changelog 2026-08-20 19:56:40 -07:00
Matthew Jackson eefd40ad12 v1.6.5: bump version (unified release) 2026-08-20 06:35:07 +00:00
Matthew Jackson df73285813 docs: 1.6.5 changelog 2026-08-19 21:20:51 -07:00
Matthew Jackson cf72927527 Strip a trailing ; comment on M/I/V keydb fields, not just U
parse_disc_entry's U arm stripped a `; MKBv…` comment before hex-parsing the
value, but the M/I/V arms did not. When a real keydb.cfg disc row's LAST field
is M/I/V (no U after it), the comment glues onto that value and parse_hex16 of
"0x… ; MKBv64" fails — silently dropping a media/id/VUK key the file actually
holds, so the disc reports no key. Strip the comment in M/I/V too (a hex16 never
contains ';', so it is safe), with a red-before-green regression test.
2026-08-19 10:10:07 -07:00
Matthew Jackson 484f024801 test: use .test not .invalid for the unresolvable key-service hosts
The round-2 guard tests used `keyserver.invalid` — RFC 6761 reserved and
guaranteed non-resolvable, which is the correct property. But the shared
leak-guard net lists `.invalid` among the private-infra TLD suffixes
(.internal/.local/.lan/.corp), so every such host tripped it and turned the
repo red on push — the exact cost the precommit script's own comment records
from 1.6.3. `.test` is equally RFC-6761-reserved and equally non-resolvable,
carries the same test guarantee, and is not in the infra net. No behaviour
change; the tests assert the identical never-resolves path.
2026-08-18 23:43:43 -07:00
Matthew Jackson e08a4e5fd3 Give the keydb cache a real file identity, and keep it talking
Round 1's parse cache keyed on `(len, modified)`, which is not a file
identity. mtime is stored at whole-second resolution on HFS+, on many
container overlays and on most network filesystems, so an external
writer — the daily-refresh job, a second freemkv process, an editor, a
sync tool — replacing keydb.cfg within one second with a file of the
same byte length left the stamp bit-identical and the source served the
superseded parse for the rest of the process's life. Retired or
since-corrected keys, reported as success. The doc claiming "an
externally replaced keydb is still picked up on the next call" was false
in exactly that case.

Two discriminators, chosen so neither costs a 62 MiB content hash (the
141 ms x 3+ loads per rip the cache removed):

  1. dev + ino. Every atomic publisher (temp + fsync + rename, including
     this crate's save()) creates a new inode, so the stamp differs
     unconditionally, whatever the clock did.
  2. A settle window. An entry is trusted only once the file has been
     quiet for the mtime granularity (2 s); the proof that this closes
     the residual in-place case, and the network-clock caveats, are in
     CacheEntry::is_settled. Cost: re-parsing for up to 2 s after each
     write of a file that is rewritten daily.

What it still does NOT catch, now stated in the doc instead of implied
away: a writer that forges the timestamp back to the cached value
(touch -t, rsync --times, a backup restore) into the same inode at the
same length. save() therefore keeps invalidating directly.

Also fixed, all in the same cache:

  - The corruption warning was swallowed. ParseStats::log runs inside
    KeyDb::parse and a cache hit skips parse, so a damaged keydb warned
    once per process and was then served in silence to every later rip.
    It is re-emitted on every hit that serves the file; an intact keydb
    still says nothing.
  - TOCTOU. cached_db stat'd the path and then separately loaded the
    path — two resolutions of one name with no atomicity, so a rename in
    the gap filed one file's stamp beside another file's keys. The file
    is opened once; the stamp is an fstat on that handle and the bytes
    come from it.
  - Memory residency is recorded rather than quietly widened: a hit
    holds every disc's key material plus host-cert private keys for the
    process's life, unzeroized. Nothing in this crate zeroizes on drop
    and that predates the cache; closing it means zeroize-on-drop across
    KeyDb, DiscEntry and libfreemkv's aacs::types, which is cross-crate
    work and not smuggled in behind a cache fix.

On KeydbHostCert's derived Debug: audited, NOT a leak. libfreemkv's
HostCert hand-writes a redacting Debug (private_key: "<redacted>",
certificate_len), so `{:?}` on a host cert prints no key bytes. But that
is an external guarantee this crate inherits silently, and the existing
redaction test only covered the outer types. It is now pinned here, for
host certs and device keys, verified red against a libfreemkv patched to
derive Debug instead.

Two KEYDB_PATH-gated tests that could not fail are fixed too:
test_decrypt_real_unit fell off the end after trying every key (a no-op
decrypt_unit passed it), and test_resolve_keys_vuk_path returned early
with zero assertions when the fixture had no qualifying entry.

CHANGELOG: entries for this and for dd9a5dc, including the previously
unrecorded Arc<str> break to KeyDb::disc_entries / DiscEntry::disc_hash.
2026-08-18 21:48:47 -07:00
Matthew Jackson 3cdf0201ae Stop making a misconfigured key service quieter than an outage
Round 1 separated a config fault from a transient one and then routed
them to OPPOSITE severities. `GuardFail::Config` returned
`Ok(Vec::new())` from `query` while `GuardFail::Unreachable` returned
`Err`, so a mistyped port — `https://host:notaport/` — produced one
`warn!` and a benign-looking empty, which `MultiSource::first_non_empty`
cannot distinguish from "no source has a row for this disc". The disc
was reported as E7022 "no key for this disc". That is the seven-hour-502
collapse with the polarity reversed, and worse: an outage self-heals, a
typo in a config file does not.

Every way of failing to ASK is now `Err(KeyServiceUnavailable)`, logged
at error level with text that names it as a URL to fix rather than an
outage to wait out. That covers the address-guard rejection (bad port,
malformed URL, a host resolving to a non-public address) and the
cleartext `http://` refusal, which had the same silent shape.

The two input-shaped guards — an over-cap MKB, too few content samples —
deliberately stay `Ok(empty)`: they are properties of this disc, not
faults, and calling them an outage would mislabel them in the other
direction. The line between the three cases is now written down at the
guards themselves.

Not a rip-breaking change: an `Err` never blocks the chain, because
`first_non_empty` still returns any later source's key set outright and
surfaces a failure only when nothing anywhere resolved.

Also: the agent-reuse cache key is now sorted and deduped. It was an
ordered `Vec<SocketAddr>` compare, so a round-robin keyserver — the one
deployment the connection reuse exists for — reordered its A/AAAA
records and missed the cache on every query, silently paying a fresh TLS
handshake each time. Set equality is also the honest predicate for
"the same validated addresses".

Note for libfreemkv: there is no `KeyServiceMisconfigured` code, so a
permanent fault borrows the transient E7028. E7028's "the source never
answered" is exactly right here; only its "retry later" hint is wrong.
2026-08-18 21:48:23 -07:00
Matthew Jackson dd9a5dc79b Stop leaking key material through Debug, and stop reporting failures as "no key"
Two defect classes dominate this change, and both are the same shape: something
that should have been loud was silent.

KEY MATERIAL IN A PUBLIC `Debug`. `KeyDb` and `DiscEntry` both carried a derived
`Debug` over raw key bytes — every processing key, and every disc's Media Key,
VID, VUK and Unit Keys. Both are public API, so a single `{:?}` in a caller's
log line, a `tracing` field, or an `assert_eq!` panic message published the
whole keydb: on the real file that is 184,860 discs' key material plus the
processing-key pool. libfreemkv's equivalent types (`aacs::types`) all
hand-write a redacting `Debug` for exactly this reason; this crate's copy of the
parser dropped the convention when it was relocated. Both impls are restored,
field-for-field in the same style, and pinned by tests that fail against the
derive.

A SOURCE THAT COULD NOT ANSWER IS NOT A SOURCE THAT SAID "NO KEY". `OnlineSource`
was engineered at length to separate `Err` (the service could not answer) from
`Ok(empty)` (it answered and holds nothing) after a seven-hour run of HTTP 502s
was reported to operators as E7022 "no key for this disc". Three places
re-collapsed that distinction:

  * `MultiSource` discarded every inner `Err` (`if let Ok(uks) = ..`), so a
    composition whose sources had all failed returned a clean `Ok(empty)` — the
    incident reproduced one layer up, in both the base and the forensic path. It
    now returns the FIRST source failure when nothing resolved, matching
    libfreemkv's own resolver, while a later success still wins outright.
  * `KeydbSource` turned every `KeyDb::load` failure into `Ok(empty)` with no
    log at all. A keydb over the 128 MiB cap or one that is not valid UTF-8 is a
    corrupt, truncated or half-written file, not a disc without a key; those are
    now logged and returned as errors. A MISSING keydb stays the documented
    benign empty.
  * A mistyped port in the key-service URL (`https://host:notaport/`) reused the
    whole authority as the hostname, so it failed DNS and surfaced as
    `Unreachable` — a standing misconfiguration reported as a transient outage,
    the one collapse `GuardFail`'s doc says must never happen because the two
    halves demand opposite operator actions. It is now a `Config` rejection, the
    same verdict the bracketed-IPv6 branch already gave.

The keydb parser was equally silent: every malformed `| DK` / `| PK` / `| HC` /
`| HC2` / `0x…` row was dropped with no counter and no log, and so was a row
lost to the entry cap or overwritten by a duplicate hash. A byte-shifted mirror
download therefore passed `save()`'s validation, was atomically committed as the
new keydb, and resolved nothing for every disc past the damage — indistinguishably
from a healthy file. Rejections are now counted by reason and summarised in one
log line per parse. The real published keydb parses with zero rejections and zero
duplicates, so the summary stays quiet on healthy input.

Also fixed, with tests:

  * The 62 MiB / 184,860-entry keydb was re-read and re-parsed on EVERY call —
    ~140 ms each in release, three or more times per AACS-cert rip. `KeydbSource`
    now caches the parse behind the file's (size, mtime) stamp (~1 µs on a hit),
    re-reads a file replaced underneath it, and drops its own cache on `save()`
    so a same-size rewrite inside the timestamp resolution cannot be missed.
    Peak RSS is unchanged; the parse simply is not repeated.
  * A disc row prefixed `0X` was dropped silently, though `parse_hex` in the same
    file deliberately accepts both cases. `save()`'s entry count now CALLS the
    parser's own predicate instead of restating it, so the two cannot drift again.
  * Each disc hash was stored twice — once as the map key, once in the entry —
    ~13 MB of duplicate text at real scale, now one shared allocation.
  * `OnlineSource` built a fresh agent per query, paying a second TLS handshake
    on every FMTS disc. The agent is reused only while the freshly re-resolved
    and re-guarded address set is identical, so the anti-rebinding guarantee is
    untouched.

Test coverage for defences that were never exercised: the cleartext-`http://`
refusal, the over-cap MKB skip and the too-few-samples skip (a reordering that
let a plaintext POST through — carrying the bearer token and base64 key material
— would have shipped green), and the 1 MiB reply cap. The real-scale tests used
to `return` early when `KEYDB_PATH` was unset, so CI ran them, asserted nothing
about 181k-entry behaviour and reported success; they are `#[ignore]`d with a
documented run command and now fail loudly if run without their fixture.
2026-08-18 14:34:44 -07:00
Matthew Jackson c7484d5b95 ci: stop Dependabot proposing Rust versions that do not exist
`dtolnay/rust-toolchain` is not a dependency in the sense Dependabot
means: it is versioned by the Rust release it installs, and the tag we
pin is the toolchain CI is pinned to on purpose — precommit.sh runs the
same one locally, so a lint that passes on a developer's newer default
cannot pass CI by accident.

Read as semver, those tags produced a 1.97.0 -> 1.100.0 proposal, a Rust
version that does not exist. Every such PR 404s on toolchain download
across all eight repos and regenerates weekly: eight permanently-red PRs
that promote.yml then has to special-case when it decides whether dev is
green.

freemkv already carries this ignore; this is the same block in the other
seven, so the fleet stays uniform. Bumping the toolchain stays a
deliberate, all-eight-repos change made by hand together with
precommit.sh.
2026-08-18 12:14:58 -07:00
Matthew Jackson 5f751e4a45 v1.6.4: bump version (unified release) 2026-08-15 09:13:34 -07:00
Matthew Jackson ebe39c0b74 Version sync to 1.6.4 2026-08-15 08:14:31 -07:00
Matthew Jackson 864e572c4f v1.6.3: bump version (unified release) 2026-08-10 09:04:44 -07:00
Matthew Jackson a6cd50bc3f Carry the same licence and community files as the other repos
These are eight public repos that ship one product on one version, and
they had drifted: freemkv-engine carried no LICENSE at all (GitHub
reported its licence as none), two crates had no code of conduct, four
had no contributing guide, and none had a security policy — so there was
no private route to report a vulnerability in a disc-decryption tool.

SECURITY.md names GitHub Security Advisories on each repo rather than an
address, so there is nothing to keep in sync and no inbox to go stale.
2026-08-09 22:08:22 -07:00
Matthew Jackson f3560032d7 Keep the cross-platform jobs for the release candidate
dev is where work lands and is meant to be pushed to often, so what runs
there should be the cheap answer to "did I break it": lint, tests and the
Linux build. The macOS and Windows jobs now run on qa and main instead of
on every push to dev.

Nothing is deleted and no platform goes unchecked before a release. qa.yml
already covers macOS and Windows independently, and the jobs that live only
here -- the Intel macOS build, the Windows release build -- still run, on
the branches where a cross-platform break is worth blocking on.

They are SKIPPED on dev via `if`, not left unscheduled. A queued job would
be worse than a slow one: release.sh's CI gate refuses while any run for
the commit is still in progress, so a job that never gets a runner blocks
releases silently, with no error anywhere. That is the failure the
real-media note in qa.yml describes, and it is why this is an `if` on the
job rather than a narrower set of trigger branches.
2026-08-09 20:24:09 -07:00
Matthew Jackson fb202a06e5 Use a .test hostname in the pinned-resolver tests
leak-guard's internal-infra net lists .invalid alongside .internal,
.local, .lan and .corp, so the hostname these tests use to prove the
pinned resolver is consulted tripped it. The point is only that the name
never resolves, and .test is reserved for exactly that by RFC 6761 -- it
is also what the rest of this suite already uses (example.test).

No change to what the tests assert.
2026-08-09 20:07:26 -07:00
Matthew Jackson c820b5ab3f Release 1.6.3
Version bump and changelog for the 1.6.3 sync. All eight crates ship the
same version, so the crates with no functional change this cycle say so
rather than carrying an empty section.
2026-08-09 20:01:33 -07:00
Matthew Jackson f61a3c3372 keysources: move to ureq 3
The three crates move together because they must: freemkv resolves its
siblings from their git TAGS in its own CI, where keysources still wanted
ureq 2, so cargo saw no conflict and the earlier split bump went green on
a dependency graph the release never uses. The qa and hash-matrix
workflows resolve those siblings from their dev PATHS, cargo unifies to a
single ureq, and the mismatch stops compiling. Verified here the way
those workflows see it: freemkv built against the sibling paths now
resolves one ureq 3.4.0 and no ureq 2 anywhere.

Notable API changes:
  AgentBuilder::new()..build()   -> Config::builder()..build() + Agent
  .redirects(0)                  -> .max_redirects(0)
  .timeout_read(d)               -> .timeout_recv_response(Some(d))
  .resolver(closure)             -> the Resolver trait
  .set(k, v)                     -> .header(k, v)
  ureq::Response                 -> http::Response<ureq::Body>
  Error::Status(code, _)         -> Error::StatusCode(code)
  Error::Transport(_)            -> fans out; matched with a catch-all

Two hazards the new resolver API introduces, both handled:

Agent::new_with_config compiles fine and then silently ignores a custom
resolver, so the pinned addresses would be dropped and every request
would go back to live DNS -- reopening the DNS-rebinding window the
guards exist to close, with no symptom. Only Agent::with_parts takes a
resolver, and that is what every pinned agent here uses.

ResolvedSocketAddrs is a fixed 16-slot array whose push writes straight
into it, so a host publishing more than 16 addresses would panic the
calling thread. Truncated to the first 16.

The four reply stubs used ureq::Response::new, which v3 removes; they are
rebuilt with http::Response::builder over a ureq::Body, so they stay
server-free unit tests.
2026-08-09 19:45:27 -07:00
Matthew Jackson caf5d2a3b0 keysources: prove the hardened agent honours its pinned addresses
resolve_and_guard validates the keyserver's addresses and hardened_agent
pins them, so a hostname cannot re-resolve to an internal address between
the guard and the POST. The validation half has seven tests; the pin had
none, and none of the existing tests makes a connection, so all 87 of them
pass even with the resolver wiring removed entirely.

That is a bad failure mode to leave uncovered, because a mis-wired
resolver fails open: connections fall back to live DNS, the rebinding
window reopens, and nothing looks wrong.

Pin the agent at a loopback listener the test owns, then ask it for a host
that cannot resolve (.invalid, reserved by RFC 2606). Only a consulted
resolver can turn that name into a connection. Asserts the connection
arrives, the round-trip completes, and the original host still travels in
the request.

Verified by removing the resolver wiring: this test fails on the timeout
while the other 87 stay green.
2026-08-09 19:29:26 -07:00
dependabot[bot] 8ccdd9988f build(deps): bump softprops/action-gh-release from 2 to 3 (#2)
Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 2 to 3.
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](https://github.com/softprops/action-gh-release/compare/v2...v3)

---
updated-dependencies:
- dependency-name: softprops/action-gh-release
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-09 17:14:34 -07:00
dependabot[bot] 0a66749f60 build(deps): update zip requirement from 2 to 8 (#6)
Updates the requirements on [zip](https://github.com/zip-rs/zip2) to permit the latest version.
- [Release notes](https://github.com/zip-rs/zip2/releases)
- [Changelog](https://github.com/zip-rs/zip2/blob/master/CHANGELOG.md)
- [Commits](https://github.com/zip-rs/zip2/compare/v2.0.0...v8.6.0)

---
updated-dependencies:
- dependency-name: zip
  dependency-version: 8.6.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-09 16:59:05 -07:00
dependabot[bot] a00035b06d build(deps): update base64 requirement from 0.22 to 0.23 (#5)
Updates the requirements on [base64](https://github.com/marshallpierce/rust-base64) to permit the latest version.
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.22.0...v0.23.1)

---
updated-dependencies:
- dependency-name: base64
  dependency-version: 0.23.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-09 16:59:02 -07:00
dependabot[bot] 98a51184c9 build(deps): bump actions/checkout from 5 to 7 (#3)
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-09 16:58:30 -07:00
Matthew Jackson 3c9963426b Have Dependabot watch the dependencies, and land its PRs on dev
Version updates were never configured here -- only security alerts, which
report but never open a pull request. So the absence of Dependabot PRs was
not "nothing to update", it was "nothing was checking".

Updates target `dev`, never `main`. main is a release pointer that
release.sh moves to each tag, so a bot commit on it would put work there
that no tag contains -- the exact state that aborted the 1.6.2 cascade at
its last step, and pointing a scheduled bot at main would recreate it
weekly.

Minor and patch bumps are grouped into one PR rather than one per crate:
eight repos times a handful of dependencies is a volume nobody reads, and
an unread PR queue is indistinguishable from no updates at all.

The freemkv crates are excluded. They depend on each other by git tag,
re-pinned by release.sh inside the release commit, and Dependabot cannot
see that cascade -- a PR bumping one could pin a version whose tag does
not exist yet.
2026-08-09 16:48:33 -07:00
Matthew Jackson 93bbe789fa v1.6.2: bump version (unified release) 2026-08-08 22:18:38 -07:00
Matthew Jackson a5077167ec changelog: 1.6.2 2026-08-08 08:53:15 -07:00
Matthew Jackson 9f99745912 v1.6.1: bump version (unified release) 2026-08-07 19:08:49 -07:00
Matthew Jackson e543bc4513 changelog: mark 1.6.1 UNRELEASED so release.sh can date it
release.sh dates the heading when it cuts the tag, and refuses a bare
'## [1.6.1]' because it cannot tell an undated heading from one it has
already stamped. Shipping a tag whose public changelog says UNRELEASED is
permanent and cannot be corrected inside that tag, so the check is right
to stop rather than guess.
2026-08-07 17:57:30 -07:00
Matthew Jackson f13f0a58fe Scope the qa matrix to the platforms each crate actually supports
The first cut ran release-profile tests on all three platforms everywhere,
which invented coverage no crate had ever claimed. bdemu is Linux-only — its
ci.yml has no macOS or Windows job at all, and its tests call
ExitStatus::signal(), which does not exist on Windows, so they do not compile
there let alone run. The rest follow the policy ci.yml already set: tests
execute on Linux and macOS, Windows compiles them so the cfg(windows) halves
still get codegen before release time.

That policy is now explicit in qa.yml as a windows-build job rather than
implied by which jobs happen to exist.

Recorded, not papered over: running the suites on Windows DOES find real
defects — autorip has five Windows path-handling failures in mover.rs and
freemkv-engine has three tests hardcoding /dev/null. freemkv ships a Windows
GUI, so they matter. They are pre-existing rather than 1.6.1 regressions, and
fixing them is its own piece of work.

Also build the binary before cli-parity.sh. cli-integration.sh builds its own;
cli-parity.sh only checks for one and exits 2 with "build it first", so both
legs died before running a single case.
2026-08-05 21:22:30 -07:00
Matthew Jackson 8945934ca9 Only patch the siblings the qa job actually checks out
cargo loads every [patch.crates-io] path whether or not the patch is used, so
naming a crate that was not cloned fails resolution with "failed to load source
for dependency freemkv-i18n" — which reads like a missing dependency rather
than the workflow bug it is.

The generated stanza listed all four libraries for every crate that needs
patching, while each job only checks out the siblings it actually depends on.
freemkv-keysources, freemkv-engine and bdemu therefore could not resolve at
all; freemkv and autorip happened to be fine because they check out all four.

Caught by the qa gate on its first run, which is the gate working.
2026-08-05 21:10:38 -07:00
Matthew Jackson 99df6f7cb0 Stamp every qa push as a release candidate
Every push to qa now tags v<version>-rc<N>, N incrementing, before the gates
run. That answers "which build is on qa, and is it the one I tested?" without
anyone having to remember it.

The tag lands whether the run goes green or red, deliberately. A red candidate
needs a name more than a green one does: "rc3 failed release-tests on windows"
is a sentence you can act on, "qa is red" is not. Red on qa is the gate doing
its job — the branch saying this is not production worth yet.

release.yml now excludes v*-rc*. Its trigger was v*, which matches the
candidate tags, so without this every push to qa would have built and PUBLISHED
a GitHub release — including for the candidates that failed.
2026-08-05 21:03:01 -07:00
Matthew Jackson 6ffc839397 Add the qa gate: dev for speed, qa for proof
dev -> qa -> main, across every public repo.

dev is where work lands and is meant to be pushed to often, so ci.yml stays
the fast answer: fmt, clippy, unit tests, leak-guard. qa is the release
candidate, and qa.yml is the claim that a commit is production worth —
release-profile tests on all three platforms, and the Linux cross-target
clippy that the local precommit gate has always run but CI never did.

Release profile matters as its own gate: overflow checks are off, debug_assert
is compiled out, and inlining changes what the optimiser can prove. A test
that only passes in debug never guarded the binary anyone ships.

qa.yml checks siblings out at qa rather than dev. This is not a monorepo, and
a qa run resolving its dependencies from unreleased dev tips would go green on
a combination that is not the one shipping — the exact mismatch the branch
model exists to prevent. ci.yml now tracks whichever branch triggered it for
the same reason, since it also fires on qa.

The consequence is that repos move to qa together, in dependency order. qa is
backfilled from main so an unchanged crate still presents working code to the
crates built on it.
2026-08-05 20:59:34 -07:00
Matthew Jackson 28a07754e7 changelog: 1.6.1 section 2026-08-05 13:59:41 -07:00
Matthew Jackson d6d701e199 v1.6.0: bump version (unified release) 2026-08-03 05:57:34 -07:00
Matthew Jackson aeec3d58d7 v1.6.0: bump version (unified release) 2026-08-02 22:35:52 -07:00
Matthew Jackson b5a96aa0f7 Record the 1.6.0 fixes in the changelog
The fvi provenance fix, the key-service outage codes, the settings BOM
data-loss fix and the per-track-kind picker rows all landed on dev without a
changelog line. A release that ships undocumented fixes is one nobody can
audit later.
2026-08-02 15:08:27 -07:00
Matthew Jackson f76356c7ab Merge branch 'fix/key-service-outage-not-missing-key' into dev 2026-08-02 14:53:34 -07:00
Matthew Jackson c4259249e1 Return a failure from the online source instead of an empty result
`query` returned `Vec::new()` for every outcome: a 502, a refused token, a DNS
failure, an unreadable body, and a service that answered 200 holding no key for
this disc. Only the last one means what the caller then reported -- E7022, "no
key source has a decryption key for this disc". The module already knew the
difference and logged it; the distinction died at the `return`, and the tracing
line that carried it is not on by default (RUST_LOG=info did not surface it
through the CLI in practice).

`query` now returns `Result<Vec<UnitKey>, Error>`:

  Ok(non-empty)  the service answered with a key
  Ok(empty)      the service ANSWERED and holds nothing, or there was nothing
                 to ask with (no service configured, blocked URL, over-cap MKB,
                 too few samples) -- a genuine miss, E7022 is correct
  Err(..)        the service could not answer -- nothing is known

`classify_http_status` is the table the fix exists for: 401/403 ->
KeyServiceUnauthorized, 429 -> KeyServiceRateLimited, 5xx -> unavailable, and
anything else non-2xx -> unavailable too. Deliberately: the genuine miss is a
**200 with no key in the body**, so a status code is never evidence about this
disc, and guessing "no key" from one is the original bug. An unreadable,
over-cap or unparseable body is the service failing mid-answer, so those are
failures as well.

`resolve_and_guard` now reports WHY it rejected a URL. A host that did not
resolve (DNS failure or DNS timeout) is the service being unreachable, and was
being folded in with "this URL is not allowed" -- one is transient, the other
never fixes itself. `GuardFail::Config` still falls through to the next source
as before.

The reply-to-verdict mapping moved into `interpret_reply` so it is testable
without a network. A stub HTTP server is not an option here: the SSRF guard
blocks loopback by design, so 127.0.0.1 can never be reached through `query`.

Nothing new is logged. The request body carries base64 key material and the
reply carries keys; the added fields are the error code and the status already
being recorded.

Red before green: http_5xx_is_a_source_failure_not_a_missing_key,
transport_failure_is_a_source_failure and unparseable_reply_is_a_source_failure
all fail against the old collapse-to-empty behaviour and pass after.
service_with_a_key_still_resolves_it_in_order is a regression pin -- it passes
either way, and is here because the happy path went through a signature change.
2026-08-02 11:53:39 -07:00
Matthew Jackson ca60c8fe1c ci: build against libfreemkv's dev tip, not its last release
Two things were wrong here.

`cargo test --locked` could never pass. Cargo.lock is gitignored for
this crate (library convention), so there is no committed lockfile to
lock against and cargo refuses to create one — every run failed before
compiling a line. That is the immediate red.

The deeper problem is what the workflow was building against. The
committed Cargo.toml resolves libfreemkv from a git tag, so CI compiled
this branch against the LAST RELEASE of libfreemkv and never against
libfreemkv's dev tip. A change spanning the two repos could not be
verified until release.sh cut the tags — at which point a break is
discovered in the middle of shipping. Green meant "compiles against last
month's library".

Now each repo is checked out into a subdirectory of the workspace,
reproducing the local ~/Developer/freemkv layout, and a generated
.cargo/config.toml redirects the git-tag patch at those paths. That is
the same config-level [patch.crates-io] mechanism local development
already uses, and it wins over the manifest patch. Subdirectories
because actions/checkout refuses a path: outside $GITHUB_WORKSPACE.
Nothing in the manifests or the release cascade changes.

Adds macOS and Windows jobs: this crate ships wherever freemkv and
autorip ship and drags in all of libfreemkv's platform code, and neither
OS was compiling it outside a release build.

clippy gains --all-targets, which is what turned up the manual
str::repeat in keydb_format's round-trip test — test code had never been
linted.
2026-07-31 13:47:38 -07:00
Matthew Jackson 2131c2c4e4 ci: build dev as well as main
Work now lands on dev and CI must be green there; main only moves at
release time, to the tagged commit, so a push to main is the release
validation run rather than day-to-day feedback.

leak-guard already runs on every branch (on: [push, pull_request]) and
release.yml stays tag-triggered, so neither needed a change.
2026-07-31 13:16:56 -07:00
Matthew Jackson 796da099a8 Sweep the pinned toolchain to Rust 1.97
The Windows UI needs current winsafe, whose real minimum is 1.89 (its manifest
under-declares 1.87 while it uses NonNull::from_ref). Rather than stop at the
minimum, this goes to current stable and fixes what that costs.

The counter-intuitive result: 1.97 is CHEAPER than 1.89. libfreemkv had 54
clippy errors at 1.89 and 6 at 1.97, because clippy tightened the noisy
collapsible_if lint in between. Stopping at the minimum would have been the most
expensive choice available.

Roughly 47 lints across the eight repos, the large majority auto-fixed:
libfreemkv 6, freemkv-engine 14, bdemu 8, freemkv-keysources 7, autorip 6,
freemkv-unlock 3, freemkv-i18n 3. The hand-fixed ones are a descending sort to
sort_by_key(Reverse), four manual checked-division sites, a loop counter replaced
by enumerate, and a loop whose first let-else became a while-let.

Worth recording for whoever bumps next: clippy is MSRV-AWARE. Those 54 lints only
appear once the crate DECLARES 1.89 or later, because let-chains become
available. A bare `cargo +1.89 clippy` against a manifest still pinned at 1.87
reports clean and is meaningless — gate with the real precommit script, which is
also the only thing that covers build scripts.

The pin still sits below the Mac default, so it keeps doing its job: catching
lint drift locally before CI sees it.
2026-07-29 21:00:55 -07:00
Matthew Jackson 8efc287eb3 Pin the toolchain to Rust 1.87
The Windows UI needs winsafe, whose current release requires rustc 1.87. The
alternative was pinning winsafe back to an older release, which would bake a
stale API surface into a brand-new UI permanently to dodge one minor version.

The pin's purpose is to sit BELOW the Mac default so clippy drift is caught
locally before CI, not to stay on 1.86 specifically, so 1.87 preserves the
discipline exactly.

Verified before moving anything, not after: `cargo +1.87 clippy -- -D warnings`
and `cargo +1.87 fmt --check` are clean across all eight repos, and the full
precommit gate (fmt + clippy + tests) passes on libfreemkv, autorip, bdemu,
freemkv-engine and freemkv-keysources. Zero new lints, zero formatting drift.
2026-07-29 20:02:02 -07:00
Matthew Jackson e963104dcf Require https for the online key service
Every request to the key service carries base64 AACS key material in its body —
the unit-key file, the MKB, content samples, the volume ID — and a bearer token
in a header when one is configured. Over http:// all of that goes to any on-path
observer, and the token is replayable.

There is no plaintext deployment to preserve. The address guard already rejects
loopback, RFC1918, ULA and link-local, so an http:// URL can only ever reach the
public internet, which is the worst case for cleartext; a self-hosted keyserver
on a LAN is impossible under that guard whatever its scheme. A non-https URL now
skips the online source with a warning rather than transmitting, so a
misconfiguration falls through to the local keydb and says why instead of
leaking quietly.
2026-07-29 16:36:43 -07:00
Matthew Jackson 7408a916bf Report why an online key lookup failed, and cap the keydb load properly
A key-service failure was indistinguishable from "this disc has no key".
Five of the seven exits in the online query returned an empty result with no
log at all, and the one that did log threw away the HTTP status the service
had already handed us — so a 502, an expired token, a rate limit and a
genuine miss all produced the same single line, and the disc was reported as
having no key. Each exit now says which one it was, and the genuine miss is
logged distinctly from every failure.

Nothing new is disclosed: the address that failed the guard, the response
body and the bearer token are all still kept out of the log. The HTTP status
is the service's own response code, and the size cap is a constant.

A malformed key in a returned UK set no longer skips that one element. The
mux trusts any non-empty result as the complete forensic set, so a short set
silently omits an index; the whole reply is rejected instead.

Also make the on-disk keydb cap real. It was a metadata pre-check followed by
an unbounded read, which is not a cap: it is skipped when the stat fails, and
a FIFO or character device reports zero length, so --keydb /dev/zero passed
the guard and then read until the process was OOM-killed (NUL is valid UTF-8,
so it never even failed the decode). Read through the same take() cap the
download path already used, and add the boundary test the load path never had.
2026-07-29 16:06:02 -07:00
Matthew Jackson efaaab6ab8 docs: drop a source name from the two keydb size-cap comments
Both MAX_KEYDB_BYTES comments justified the 128 MiB ceiling by naming
the public keydb they were sized against. The number and the reasoning
are the useful part; the name is not, and it trips the pre-commit
scan. Keep the justification, drop the name.
2026-07-29 15:29:48 -07:00
Matthew Jackson ca7a0e41c2 Raise keydb size cap 64 MiB -> 128 MiB
The public UHD keydb is ~62 MiB and growing, leaving only ~2 MiB
under the old 64 MiB ceiling — updates would start failing as it grows.
128 MiB restores years of headroom while still bounding a decompression
bomb. Both cap sites (keydb.rs download/save, keydb_format.rs on-disk load)
bumped; the unrelated MKB cap in online.rs is untouched.
2026-07-29 07:03:19 -07:00
Matthew Jackson 102895330d 1.6.0: version sync + doc fixes
Bump to 1.6.0 (workspace sync, no source change). Fix stale get_uk ->
get_unit_keys references in the crate description + comments, correct the
libfreemkv version note, and drop a phantom Mapfile mention from the tests.
2026-07-28 15:35:19 -07:00
Matthew Jackson acc0144870 changelog: date the released 1.5.x sections (were stale UNRELEASED) 2026-07-28 13:55:23 -07:00
Matthew Jackson d5261cc097 v1.5.2: bump version (unified release) 2026-07-22 22:19:45 -07:00
Matthew Jackson 311aef939c Version sync 1.5.2 (inherits libfreemkv CSS DVD descramble fix) 2026-07-22 11:00:39 -07:00
Matthew Jackson ad16d745a4 v1.5.1: bump version (unified release) 2026-07-20 17:02:17 -07:00
Matthew Jackson 1a586239bc v1.5.0: bump version (unified release) 2026-07-19 15:37:30 -07:00
Matthew Jackson a89e688866 v1.4.5: bump version (unified release) 2026-07-18 16:08:06 -07:00
Matthew Jackson 03e42c1f5a changelog: backfill 1.4.3/1.4.4 and add 1.4.5 2026-07-18 13:31:59 -07:00
Matthew Jackson 3175b71fce keydb: fix device keys silently dropped on uppercase 0X hex prefix
parse_device_key parsed node/uv/shift via from_str_radix(trim_start_matches("0x")),
a CASE-SENSITIVE strip: an uppercase 0X prefix was not removed, from_str_radix
failed, and the WHOLE device key was dropped. Now uses the canonical
libfreemkv::hex::parse_hex_u16/u32/u8. Red->green regression test added.
2026-07-17 21:25:59 -07:00
Matthew Jackson ee6e5efa4e keysources: split KeySource into get_unit_keys + get_fmts_indexes
Base per-CPS-unit keys and the AACS 2.1 forensic index set are now two explicit
trait operations instead of one overloaded get_uk whose return count was implicit.
keydb serves unit keys (forensic opts out via the default); online implements both;
MultiSource drives both in order. Teed up for 1.4.5. Local WIP baseline.
2026-07-17 20:51:24 -07:00
Matthew Jackson 65e08c6dc2 v1.4.4: bump version (unified release) 2026-07-16 21:44:10 -07:00
Matthew Jackson 10c6838506 online: build the /decode request from a DecodeSampleSet (proven sufficient by type, not a runtime len check) 2026-07-16 21:43:03 -07:00
Matthew Jackson 1911e362a8 v1.4.3: bump version (unified release) 2026-07-16 21:06:07 -07:00
Matthew Jackson 74160f00e9 online: re-export MIN_SAMPLE_UNITS from libfreemkv 2026-07-16 21:01:16 -07:00
Matthew Jackson 22c37f1b50 online: parse UK as an array (1 key plain, 32 keys forensic)
The key service now always returns {"UK":[...]} - an array of one for a
plain movie sample, or all 32 index-ordered variant keys for a forensic
sample. OnlineSource::query accepts both the legacy string form and the
array form, emitting one UnitKey per element (index = array position).
Re-export MIN_SAMPLE_UNITS so callers size their samples correctly.
2026-07-16 19:41:44 -07:00
Matthew Jackson 00dd77d49b v1.4.2: bump version (unified release) 2026-07-15 19:37:56 -07:00
Matthew Jackson fe78d7445f Adapt keydb test to segregated decrypt primitives; 1.4.2 changelog
The KEYDB-gated test that feeds real key material into libfreemkv's AACS crypto
now composes decrypt_unit + is_clean (the removed decrypt_unit_try_keys wrapper's
behaviour), unchanged semantics.
2026-07-15 19:35:25 -07:00
Matthew Jackson 36fb6f0c61 v1.4.1: bump version (unified release) 2026-07-14 14:45:36 -07:00
Matthew Jackson 1a9a3befe3 Changelog for 1.4.1 2026-07-14 14:43:14 -07:00
Matthew Jackson d7152e0f73 v1.4.0: bump version (unified release) 2026-07-13 19:28:58 -07:00
Matthew Jackson 6ceb92ade9 Changelog: 1.4.0 (Blu-ray 3D / MVC) 2026-07-13 12:47:03 -07:00
Matthew Jackson 32b24c0ba7 v1.3.2: bump version (unified release) 2026-07-10 14:26:14 -07:00
Matthew Jackson 974c1dc664 1.3.2: carry UnitKey.variant_number (all sources emit 0)
Construct unit keys via UnitKey::new; ordinary content is variant 0. Inherits libfreemkv 1.3.2. No behaviour change.
2026-07-10 14:00:28 -07:00
Matthew Jackson 7766527e49 v1.3.1: bump version (unified release) 2026-07-10 12:35:16 -07:00
Matthew Jackson a0e5d0e405 1.3.1: relicense to MIT
Relicensed from AGPL-3.0 to MIT, effective 1.3.1 (<=1.3.0 remain AGPL).
Version sync to 1.3.1.
2026-07-10 12:31:19 -07:00
Matthew Jackson 26cdab9080 test: use example.test placeholder host, not .invalid
leak-guard bans the .invalid TLD (grouped with private .internal/.local/etc);
example.test is the RFC 6761 reserved test TLD — equally unreachable, not on
the ban list — so the leak-guard CI check passes.
2026-07-10 08:52:57 -07:00
Matthew Jackson 05420176d1 v1.3.0: bump version (unified release) 2026-07-10 08:43:50 -07:00
Matthew Jackson b90d64fe31 docs: 1.3.0 changelog 2026-07-10 08:19:22 -07:00
Matthew Jackson f6a1e08593 release: bump to 1.3.0 2026-07-08 14:48:50 -07:00
Matthew Jackson f208d6cf70 keydb: round-trip AACS 2.0 host certs; tighten entry validation
- to_keydb_cfg emits the sibling `| HC2 |` line (AACS 2.0 host cert),
  the inverse of parse_host_cert_v2, so a round-trip never silently
  drops v2 host certs.
- save-validation mirrors the parser's real disc-entry rule exactly: a
  `0x` line counts as an entry only if it also contains ` = `, so
  validating + persisting content that parses to zero usable entries
  (e.g. a stray `0xDEADBEEF` line) can no longer succeed.
- disc-entry title is kept verbatim (parens and all) on parse, matching
  the emit path so it round-trips.
2026-07-08 14:45:25 -07:00
Matthew Jackson b577f7fa17 Bump to 1.2.3 2026-07-07 16:36:25 -07:00
Matthew Jackson 87e18c64cd keydb/online: resolve off aacs::derive + types, drop boil wrappers
libfreemkv deleted the aacs::boil veneer, so switch the resolve path to the
raw primitives: derive_media_key_from_{pk,dk}, derive_vuk, decrypt_unit_key
from aacs::derive, and the newtypes from aacs::types. Add a local uks_from_vuk
helper composing decrypt_unit_key. No behaviour change; fmt/clippy/test green
on Rust 1.86.
2026-07-05 12:09:50 -07:00
Matthew Jackson c7db7c7f9c v1.2.2: bump version (unified release) 2026-07-03 20:50:20 -07:00
Matthew Jackson f18a5b3edf keydb: adopt corrected mk_from_dk (2-arg), reviving the DK->MK fallback
libfreemkv's AACS work changes mk_from_dk from (dk, mkb, vid) to (dk, mkb) —
the VID enters at the VUK step, not the MK step. Adopt the 2-arg call so the DK
pool is tried unconditionally (no longer gated on a VID being present), matching
libfreemkv. Without this keysources fails to compile against the new libfreemkv.
(Equivalent to the stranded b80d4cb; applied directly since that branch had
diverged surrounding context.)
2026-07-03 20:36:50 -07:00
Matthew Jackson 3f162511d3 v1.2.1: bump version (unified release) 2026-07-01 19:08:59 -07:00
Matthew Jackson 9b7a707947 v1.2.1: bump version 2026-07-01 17:30:25 -07:00
Matthew Jackson a6f73b624b Cargo: publish = false + git-pin libfreemkv (off crates.io)
keysources deps libfreemkv, now git-only, so keysources follows it off
crates.io. A committed [patch.crates-io] redirects the bare libfreemkv version
req to the git tag (local dev overrides to a path via the gitignored config).
2026-06-29 21:04:05 -07:00
Matthew Jackson cdadd3dc70 docs: 1.2.0 changelog date → 2026-06-29 (release prep) 2026-06-29 09:57:29 -07:00
Matthew Jackson 177ee8c08e docs: 1.2.0 changelog — drop pre-1.1.0 carryover entries
The 'KeydbSource owns save+update' (Added) and 'Processing-Key
decryption restored' (Fixed) entries were stale carryover from the
never-released 1.1.0-beta.1; that code shipped in v1.1.0 (zero diff in
v1.1.0..HEAD). The 1.2.0 entry now lists only the real 1.2.0 changes
(hex unification, DiscInputs version stride, MKB cap).
2026-06-29 01:21:46 -07:00
Matthew Jackson c81fb8bbd4 v1.2.0: bump version + reconcile CHANGELOG to [1.2.0] 2026-06-28 23:34:19 -07:00
Matthew Jackson ec86a22e7c 1.2.0: route online/keydb hex parsing through libfreemkv::hex (one parser) 2026-06-28 22:12:07 -07:00
Matthew Jackson 5f550be7cf 1.1.1: DiscInputs gains version; tests pull stride from inputs.version 2026-06-28 21:45:58 -07:00
Matthew Jackson 5ce5e424d3 1.1.1: align online MKB cap with libfreemkv read cap (64 MiB) + log over-cap
The online source dropped any MKB over 10 MiB while libfreemkv's reader
captures up to 64 MiB — an MKB in that band was silently un-forwardable
(no key, no surfaced cause). Match the cap and log when it is exceeded.
2026-06-28 21:12:57 -07:00
Matthew Jackson 086e763ab7 v1.1.0: bump version (unified release) 2026-06-28 18:19:57 -07:00
Matthew Jackson 049a11c6d4 keydb: public-crate hygiene — scrub internal tool name from comments, synthetic test disc hash 2026-06-28 16:49:16 -07:00
Matthew Jackson 15e41136cf keydb: union stored UKs + VUK/MK-derived unit keys (dedup by value)
unit_keys_from now loads all stored unit keys AND the VUK/MK-derived ones, deduping by key value instead of first-hit, so a disc with partial stored UKs plus a VUK yields every declared CPS unit key. Plus keydb.cfg format round-trip (to_keydb_cfg inverse of parse).
2026-06-28 15:06:24 -07:00
Matthew Jackson 14f0f8c675 keydb_format: faithful verbatim titles + tag-letter field fix + real-data idempotence test
Found via a parse->serialize->parse->serialize byte-identical check over the full
182,535-disc OEM keydb:
- Title was reduced to a parenthesised substring, truncating real titles
  ("Lawrence of Arabia (Restored Version) - Disc 2 - 4K Ultra HD" -> "Restored
  Version"). Keep the title VERBATIM (faithful copy); display prettification is a
  separate layer.
- A disc titled exactly "M" (`= M | M | 0x...`) had its title eaten as the M
  field tag, dropping the real media key. Field scan now skips parts[0] (always
  the title), so a title that is a tag letter (M/I/V/U/D) can't shadow a field.
- Add to_keydb_cfg_is_idempotent_on_real_keydb (KEYDB_PATH-gated): proves parse
  is lossless on its own output and the serializer is deterministic.
2026-06-27 15:58:30 -07:00
Matthew Jackson 56fb0d53c9 keydb_format: add to_keydb_cfg serializer (inverse of parse); rename DiscEntry.disc_id -> vid
Central round-trippable codec: parse + to_keydb_cfg in one place.
Emits HC, DK, PK, then disc entries sorted by hash. Round-trip test
(parse -> serialize -> re-parse) covers HC/DK/PK/disc. disc_id -> vid
names the field for what it is (the I-field volume ID).
2026-06-27 14:42:58 -07:00
Matthew Jackson 7b6531069b keysources: expose the keydb.cfg parser (pub keydb_format / KeyDb, DiscEntry)
Make the parser public — parsing a keydb is not secret (freemkv uses it, and so
do tools that build a per-disc registry from it, e.g. a Volume-ID index). Purely
additive: a private module in this public crate becomes nameable; no behaviour
change, nothing removed.
2026-06-26 21:07:56 -07:00
Matthew Jackson 7d935f2002 KeydbSource owns keydb save + update
Move the keydb save/validation/atomic-write path out of libfreemkv onto
KeydbSource. New KeydbSource::save(bytes) validates + decompresses (zip /
gz / plain, decompressed-size capped) and crash-safely writes to the
source's OWN path (sibling-temp + fsync + rename + parent-dir fsync) —
not a hardcoded default, so the caller chooses the destination.
KeydbSource::update(fetch, url) calls an INJECTED fetch closure then
save, keeping this crate transport-agnostic on the update path (the app
supplies its own TLS / SSRF-guarded transport). UpdateResult moves here
and is re-exported. Add flate2 + zip (already in the resolved graph via
libfreemkv) for decompression; no new HTTP stack.
2026-06-26 17:34:03 -07:00
Matthew Jackson a546b050ec Restore AACS PK key-processing path in keysources
unit_keys_from now resolves the Media Key in order: stored per-disc MK
-> keydb Processing Key pool (mk_from_pk vs this disc's own MKB) ->
device-key pool (mk_from_dk), then MK+VID -> VUK -> UK. MK/VUK entries
still honored directly; cross-disc MK-pool brute stays retired. Fixes the
factually-wrong justifying comment + adds PK-pool KATs.
2026-06-26 17:03:58 -07:00
Matthew Jackson 9726ba5980 AACS: own keydb parser + 100% parse + get_uk derivation
- Relocate keydb.cfg parser into keydb_format.rs (libfreemkv no longer knows
  keydb); add 100% parse (mkb_version/volume_size/is_uhd, revoked_at_mkb) +
  helper API (get_uk/get_uks/get_vid/host_certs(mkb)).
- KeydbSource/OnlineSource/MultiSource -> get_uk(ctx); MultiSource host_certs
  union; KAT-proven derivation parity. NumberedUnitKey alias. clippy clean.
2026-06-26 12:19:24 -07:00
Matthew Jackson dbe7bd257a ci: add CI workflow (fmt + clippy -D warnings + test on Rust 1.86)
The crate previously had only leak-guard.yml and release.yml, so its ~42
tests never ran on push/PR — only on tag. Mirror the sibling crates'
ci.yml (lint + test jobs, dtolnay/rust-toolchain@1.86.0, rust-cache).
No --locked: this library does not track Cargo.lock (release.yml notes
--locked would fail on a fresh runner).
2026-06-25 17:58:30 -07:00
Matthew Jackson 251c08e6e3 freemkv-keysources: fixture-based integration tests for key sources
Add tests/key_sources.rs exercising the published KeySource impls
end-to-end over real fixture files and the libfreemkv parsers:

- KeydbSource: disc-hash lookup from a real keydb.cfg (VUK hit ranks
  ahead of the universal DK pool), hash miss yields only the pool,
  missing file is silent/not errored, label + needs_samples, and
  host-cert serving from a | HC | row (inherent and trait paths).
- paths: exe-local search list, default == search head, existing path
  reflects on-disk state (local-only, no OS fallback).
- MapfileSource: persisted # freemkv-uk: keys read back as a terminal
  Key::Unit, one-shot exhaustion, missing/keyless mapfile offers nothing.
- OnlineSource: unconfigured no-op (no network), one-shot latch,
  metadata, and validate_keyserver_url scheme/SSRF gating.
- MultiSource: caller-supplied order/precedence (and its reverse),
  empty-source skip, needs_samples/errored OR-aggregation, nesting, and
  a real keydb-then-mapfile precedence chain over fixtures.
2026-06-24 23:33:00 -07:00
Matthew Jackson 02931638d8 v1.0.0-rc.5.3: bump version (unified release) 2026-06-24 21:55:08 -07:00
Matthew Jackson 8d4d8cb576 freemkv-keysources: keydb search is local to the executable
keydb_search_paths() returns exactly [<exe dir>/keydb.cfg] (or empty if
current_exe is unavailable) — no %APPDATA%/.config/XDG lookup. Matches
libfreemkv::keydb::default_path.
2026-06-24 20:46:31 -07:00
31 changed files with 10631 additions and 1126 deletions
+33
View File
@@ -0,0 +1,33 @@
# Version updates are monthly and grouped into one PR per ecosystem.
# Security updates are separate and run whenever an advisory is published.
version: 2
updates:
- package-ecosystem: cargo
directory: /
target-branch: dev
schedule:
interval: monthly
open-pull-requests-limit: 5
groups:
all:
patterns:
- '*'
ignore:
- dependency-name: freemkv-unlock
- dependency-name: libfreemkv
- dependency-name: freemkv-keysources
- dependency-name: freemkv-i18n
- dependency-name: freemkv-engine
- package-ecosystem: github-actions
directory: /
target-branch: dev
schedule:
interval: monthly
open-pull-requests-limit: 5
groups:
all:
patterns:
- '*'
ignore:
- dependency-name: dtolnay/rust-toolchain
- dependency-name: freemkv/.github*
+262
View File
@@ -0,0 +1,262 @@
name: CI
on:
push:
# dev -> qa -> main. `dev` is where work lands and is meant to be pushed
# to often: these are the FAST checks, so a mistake surfaces in minutes.
# `qa` is the release candidate — it runs these too, plus the expensive
# suite in qa.yml. `main` only ever moves at release time, to a tagged
# commit that was already green on qa.
branches: [main, dev, qa]
pull_request:
# Rapid dev pushes supersede their own in-progress runs instead of stacking.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
# WHY THE SIBLING CHECKOUTS — this is NOT a monorepo.
#
# The committed Cargo.toml resolves libfreemkv from a git TAG (it is off
# crates.io). That means CI was building this branch against the LAST RELEASE
# of libfreemkv, never against libfreemkv's dev tip — so a coordinated change
# across the two repos could not be verified until release-orchestrate.yml cut the tags.
# Green here meant "compiles against last month's library".
#
# The fix mirrors the developer's local layout via the shared composite
# freemkv/.github//actions/checkout-siblings: every sibling repo is checked out
# into a subdirectory of $GITHUB_WORKSPACE, reproducing
# ~/Developer/freemkv/{freemkv-keysources,libfreemkv,freemkv-unlock}, and a
# generated .cargo/config.toml redirects the git-tag patch to those paths.
# Config-level [patch.crates-io] wins over the manifest one, which is the same
# mechanism local development already uses (see the note in Cargo.toml).
#
# Subdirectories rather than `../libfreemkv` because actions/checkout refuses
# any `path:` outside $GITHUB_WORKSPACE, and the workspace IS the repo dir.
# Repos stay separate, tags stay separate, the release cascade is untouched.
#
# libfreemkv path-deps ../freemkv-unlock, which resolves to the workspace root
# under this layout — hence the third checkout.
#
# `--locked` is deliberately absent: Cargo.lock is now committed (issue #15) and
# the `deny` job audits the resolved graph, but CI's sibling path-patch layout
# resolves the freemkv crates as path packages, which `--locked` can't satisfy —
# the same reason the binary crates omit it. The lock that has to hold is the one
# release-orchestrate.yml regenerates in each consuming binary.
jobs:
# Single source of truth for the MSRV — read rust-version from Cargo.toml once
# and export it; MSRV-bound jobs consume needs.resolve-msrv.outputs.v. clippy
# runs on stable instead (see CI.md's Toolchain policy).
resolve-msrv:
runs-on: ubuntu-latest
outputs:
v: ${{ steps.msrv.outputs.v }}
steps:
- uses: actions/checkout@v7
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
# CI owns the changelog: main may never carry a released version without a
# CHANGELOG.md entry for it. The release orchestrator dates an existing
# `## [X.Y.Z]` heading — it does NOT invent one — so a version bump that
# forgot its changelog entry must fail here rather than ship a blank release.
# Hard-fails on main (and PRs targeting main); warns elsewhere so an
# in-progress dev/qa branch whose entry isn't written yet is not blocked.
changelog:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Require a CHANGELOG entry for the crate version on main
shell: bash
run: |
set -euo pipefail
version=$(grep -m1 '^version = ' Cargo.toml | sed -E 's/.*"([^"]+)".*/\1/')
if grep -qE "^## \[${version}\]" CHANGELOG.md; then
echo "CHANGELOG.md has an entry for ${version}."
exit 0
fi
msg="CHANGELOG.md has no '## [${version}]' entry"
if [ "${GITHUB_REF_NAME:-}" = "main" ] || [ "${GITHUB_BASE_REF:-}" = "main" ]; then
echo "::error::${msg} — main requires a changelog entry for the released version."
exit 1
fi
echo "::warning::${msg} (allowed on ${GITHUB_REF_NAME:-unknown}; required before it can reach main)."
lint:
needs: resolve-msrv
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: freemkv-keysources
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
# fmt runs on the MSRV pin; clippy runs on stable so new-lint debt is a
# blocking gate the pinned toolchain would hide. Stable is installed first
# for clippy; the MSRV pin goes second so it stays fmt's default toolchain.
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
components: rustfmt
- uses: Swatinem/rust-cache@v2
with:
workspaces: freemkv-keysources
- run: cargo fmt --check
working-directory: freemkv-keysources
# --all-targets so the test code is linted too, not just the library.
- run: cargo +stable clippy --all-targets -- -D warnings
working-directory: freemkv-keysources
test:
needs: resolve-msrv
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: freemkv-keysources
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: freemkv-keysources
- run: cargo test --tests
working-directory: freemkv-keysources
# Dependency-audit gate. Mirrors the `test` job's sibling-checkout layout and
# generated .cargo/config.toml so `cargo metadata` resolves the git-tag patch
# to the local sibling paths; cargo-deny then audits advisories, bans,
# licenses, and sources against the committed deny.toml.
deny:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: freemkv-keysources
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@stable
- uses: taiki-e/install-action@cargo-deny
# Run from the repo dir so the generated .cargo/config.toml path patches apply.
- run: cargo deny --all-features check advisories bans licenses sources
working-directory: freemkv-keysources
# This crate ships wherever freemkv and autorip ship, and it drags in all of
# libfreemkv's platform code (SCSI transports included). Build the tests, not
# `cargo check` — check skips codegen and never compiles test-only code.
check-macos:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: a never-scheduled job leaves the run pending forever, so
# the branch's required checks never report complete.
needs: resolve-msrv
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: macos-latest
steps:
- uses: actions/checkout@v7
with:
path: freemkv-keysources
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: freemkv-keysources
- run: cargo build --tests
working-directory: freemkv-keysources
# Additive, NON-REQUIRED coverage. Mirrors the `test` job's sibling-checkout
# layout and pinned toolchain (nothing in the release cascade depends on it,
# and codecov upload is tokenless with fail_ci_if_error:false, so a flaky
# upload never reddens the run). `--locked` is deliberately absent here for
# the same reason it is in `test`: CI's sibling path-patch layout resolves the
# freemkv crates as path packages, which `--locked` can't satisfy.
coverage:
needs: resolve-msrv
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: freemkv-keysources
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
components: llvm-tools-preview
- uses: Swatinem/rust-cache@v2
with:
workspaces: freemkv-keysources
- uses: taiki-e/install-action@cargo-llvm-cov
- run: cargo llvm-cov --workspace --ignore-run-fail --lcov --output-path lcov.info
working-directory: freemkv-keysources
- uses: codecov/codecov-action@v7
with:
files: freemkv-keysources/lcov.info
slug: freemkv/freemkv-keysources
fail_ci_if_error: false
token: ${{ secrets.CODECOV_TOKEN }}
check-windows:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: a never-scheduled job leaves the run pending forever, so
# the branch's required checks never report complete.
needs: resolve-msrv
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: windows-latest
steps:
- uses: actions/checkout@v7
with:
path: freemkv-keysources
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: freemkv-keysources
- run: cargo build --tests
working-directory: freemkv-keysources
+12
View File
@@ -0,0 +1,12 @@
name: comment-guard
# Thin caller → the canonical reusable guard in freemkv/.github (one definition
# for every repo). Fork PRs still run it; same-repo pushes aren't double-run.
on: [push, pull_request]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
comment-guard:
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork
uses: freemkv/.github/.github/workflows/guard-comment.yml@v2
+14
View File
@@ -0,0 +1,14 @@
name: identity-guard
# Thin caller → the canonical reusable guard in freemkv/.github.
on:
push:
branches: [dev, qa, main]
pull_request:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
identity:
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork
uses: freemkv/.github/.github/workflows/guard-identity.yml@v1
+6 -30
View File
@@ -1,35 +1,11 @@
name: leak-guard
# Self-contained public-repo leak gate. Public CI cannot reach the private
# tooling, so this encodes only the generic net: internal-infra references,
# tracked CLAUDE.md/.claude paths, and AI-attribution in commit messages.
# No project-specific reverse-engineering vocabulary lives here.
# Thin caller → the canonical reusable guard in freemkv/.github.
on: [push, pull_request]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
leak-guard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Compute commit range
id: range
run: |
if [ "${{ github.event_name }}" = "pull_request" ]; then
base="${{ github.event.pull_request.base.sha }}"
head="${{ github.event.pull_request.head.sha }}"
echo "range=$base..$head" >> "$GITHUB_OUTPUT"
else
before="${{ github.event.before }}"
after="${{ github.sha }}"
# New branch / first push: github.event.before is all-zeros.
if [ -z "$before" ] || [ "$before" = "0000000000000000000000000000000000000000" ]; then
echo "range=$after" >> "$GITHUB_OUTPUT"
else
echo "range=$before..$after" >> "$GITHUB_OUTPUT"
fi
fi
- name: Run leak-guard
run: bash ci/leak-guard.sh "${{ steps.range.outputs.range }}"
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork
uses: freemkv/.github/.github/workflows/guard-leak.yml@v1
+157
View File
@@ -0,0 +1,157 @@
name: qa
# ── The qa gate: "is this production worth?" ────────────────────────────────
#
# dev -> qa -> main.
#
# `dev` is for committing often. ci.yml answers "is it green" in minutes with
# fmt, clippy and the unit suite, so a mistake surfaces while it is still cheap
# to fix. `qa` is the release-candidate branch, and THIS workflow is the claim
# that a commit is production worth: everything expensive that can run without
# physical media. `main` only ever receives a qa that went green here.
#
# Sibling repos are checked out at `qa`, NOT `dev`. A qa run that resolved its
# dependencies from dev tips would be validating a combination that is not the
# one being released, which is the exact failure this branch exists to prevent.
# The shared checkout-siblings composite resolves qa->qa from the caller branch.
#
# What this gate CANNOT cover: `disc://` and real `iso://` need physical media,
# and no hosted runner has an optical drive or the image hoard. That coverage
# lives in freemkv's qa.yml (the real-media EC2 matrix), not in this file.
on:
push:
branches: [qa]
workflow_dispatch:
# Supersede an in-progress qa run when a newer candidate lands on the branch.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
# Single source of truth for the MSRV — see ci.yml. Jobs that build/test AT
# the MSRV read needs.resolve-msrv.outputs.v from Cargo.toml's rust-version;
# clippy (cross-lint) runs on stable instead.
resolve-msrv:
runs-on: ubuntu-latest
outputs:
v: ${{ steps.msrv.outputs.v }}
steps:
- uses: actions/checkout@v7
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
# ── Name the candidate ────────────────────────────────────────────────
#
# Every push to `qa` is a release candidate, so every push gets a tag:
# v<version>-rc<N>, N incrementing. That is the answer to "which build is on
# qa right now, and is it the one I tested?" — a question that otherwise gets
# answered from memory.
#
# This runs FIRST and does not depend on the gates, deliberately. A red
# candidate needs a name more than a green one does: "rc3 failed
# release-tests on windows" is a sentence you can act on; "qa is red" is not.
# Red on qa is a working gate, not an incident — it is the branch saying this
# is not production worth yet. Fix on dev, get dev green, push qa again.
#
# release.yml excludes v*-rc* so a candidate never publishes a release.
rc-tag:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Stamp the next rc
shell: bash
run: |
set -euo pipefail
v=$(sed -n 's/^version = "\(.*\)"/\1/p' Cargo.toml | head -1)
[ -n "$v" ] || { echo "no version in Cargo.toml" >&2; exit 1; }
# Numeric sort on the rc ordinal: -rc10 must beat -rc9, and a plain
# lexical sort gets that backwards from the tenth candidate on.
n=$(git tag -l "v$v-rc*" | sed "s|^v$v-rc||" | sort -n | tail -1)
tag="v$v-rc$(( ${n:-0} + 1 ))"
git tag "$tag"
git push origin "$tag"
echo "### Candidate \`$tag\`" >> "$GITHUB_STEP_SUMMARY"
# The debug suite runs on every dev push. Release is a DIFFERENT build:
# overflow checks are off, debug_assert! is compiled out, and inlining
# changes what the optimiser can prove. A test that only passes in debug is
# a test that never guarded the binary anyone actually ships.
release-tests:
needs: resolve-msrv
strategy:
fail-fast: false
matrix:
os: ['ubuntu-latest', 'macos-latest']
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v7
with:
path: freemkv-keysources
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: freemkv-keysources
- run: cargo test --release --tests
working-directory: freemkv-keysources
# clippy's output is target-dependent: cfg-gated code only gets linted on
# the target it compiles for. Linting solely on the dev machine's host
# target is how a lint that CI rejects reaches a push.
cross-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: freemkv-keysources
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- uses: Swatinem/rust-cache@v2
with:
workspaces: freemkv-keysources
- run: rustup target add x86_64-unknown-linux-gnu
- run: cargo clippy --all-targets --target x86_64-unknown-linux-gnu -- -D warnings
working-directory: freemkv-keysources
# Windows compiles the tests but does not run them, matching the policy
# ci.yml already set. The value here is codegen: the #[cfg(windows)] halves
# of the SCSI transport and platform layers compile on no other runner, so
# without this they are first built at release time.
windows-build:
needs: resolve-msrv
runs-on: windows-latest
steps:
- uses: actions/checkout@v7
with:
path: freemkv-keysources
- uses: freemkv/.github/.github/actions/checkout-siblings@v1
with:
checkout: libfreemkv freemkv-unlock
patch: libfreemkv freemkv-unlock
self: freemkv-keysources
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: freemkv-keysources
- run: cargo build --release --tests
working-directory: freemkv-keysources
+24 -15
View File
@@ -4,6 +4,11 @@ on:
push:
tags:
- 'v*'
# NOT the release-candidate tags. Every push to `qa` stamps a
# v<version>-rc<N> so a run can be named, and 'v*' matches those too —
# which would have this workflow build and PUBLISH a GitHub release for
# every candidate, including the red ones.
- '!v*-rc*'
permissions:
contents: write
@@ -12,7 +17,7 @@ jobs:
verify:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Verify Cargo.toml version matches tag
run: |
CARGO_VER="v$(grep '^version' Cargo.toml | head -1 | sed 's/.*"\(.*\)"/\1/')"
@@ -24,28 +29,32 @@ jobs:
# Tests run as a PARALLEL TRIPWIRE: they fail the run if they fail, but the
# release job does NOT `needs:` this job. The tag decision was already gated by
# the local precommit (same Rust 1.86, same commit).
# CI (ci.yml on dev, then a green qa.yml on the same commit).
#
# freemkv-keysources is a library — Cargo.lock isn't tracked, so --locked
# would always fail (no lockfile to lock against on a fresh runner). Its
# Cargo.lock is now committed (issue #15), but --locked is still omitted: on
# the runner the libfreemkv dep resolves from crates.io, not the local path
# patch, which --locked can't reconcile. Its
# libfreemkv dep resolves from crates.io (the dev .cargo/config.toml patch is
# gitignored and absent on the runner), so libfreemkv must already be published
# at this version — the release script enforces that ordering before tagging.
# at this version — release-orchestrate.yml enforces that ordering before tagging.
test:
needs: verify
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: actions/checkout@v7
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ steps.msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
- run: cargo test
# NOTE: the crates.io publish is driven by the maintainer's release tooling
# from local credentials, not from CI — this repo has no CARGO_REGISTRY_TOKEN
# secret. release.sh runs `cargo publish --no-verify` locally. CI here only
# verifies the version/tag match, runs tests (tripwire), and cuts the GitHub
# Release. The binaries git-tag-pin keysources, so they don't wait on the
# crates.io publish — it serves external consumers in parallel.
# NOTE: there is no crates.io publish — keysources is `publish = false` and
# git-tag-only. Consumers git-tag-pin it via a committed [patch.crates-io], so
# the git tag itself is the release artifact. CI here only verifies the
# version/tag match, runs tests (tripwire), and cuts the GitHub Release.
release:
# Only needs `verify`; the GitHub Release is cut as soon as the version
@@ -53,8 +62,8 @@ jobs:
needs: verify
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
uses: softprops/action-gh-release@v3
with:
generate_release_notes: true
-1
View File
@@ -1,5 +1,4 @@
/target
Cargo.lock
# session scratch — never track (may contain RE breadcrumbs)
scratch/
+368
View File
@@ -0,0 +1,368 @@
# Changelog
## [1.8.0] — 2026-10-05
### Fixed
- The online key service ignores `HTTP_PROXY`/`HTTPS_PROXY`/`ALL_PROXY`: with one set, every key lookup failed (the pinned resolver also answered the proxy's address, so the proxy request went to the key service).
### Changed
- `KeydbSource` hands out host certs best first, with no disc input: certs with no `Revoked in MKBv<N>` note in keydb order, then revoked ones by latest revocation. The inherent `host_certs()` and the `KeySource::host_certs` trait method both return this order; the trait ignores its `mkb` argument.
### Deprecated
- `KeyDb::host_certs(mkb)`: use `KeyDb::host_certs_ranked`.
### Added
- `KeyDb::host_certs_ranked`.
- `check_keyserver_url` with `KeyserverUrlRejection`/`KeyserverUrlFault`: a typed Permanent vs Temporary verdict, so callers need not match `validate_keyserver_url`'s text.
- `test-hooks` feature exposing `set_last_decode_reachability`, to test decode-verdict handling without a network.
## [1.7.7] — 2026-09-26
### Maintenance
- Replace comment-overflow documentation with concise source contracts and README instructions; enforce the shared comment policy in CI.
## [1.7.6] — 2026-09-26
### Changed
- Version aligned to 1.7.6 for the unified release. No functional changes to this crate; the release is driven by the freemkv 1.7.6 Linux desktop shell (GTK4 + libadwaita) and the rip-finished desktop notification (issue #56).
## [1.7.5] — 2026-09-23
### Changed
- Version aligned to 1.7.5 for the unified release. No functional changes to this crate; the release is driven by freemkv-unlock mirroring the freemkv-firmware 0.9.0 ABI (the drive's `Ake` and `Bus` levers retired into a single `Encryption` lever).
## [1.7.4] — 2026-09-21
### Fixed
- keydb lookups now report match state and entry shape, so a "no entry" result tells you whether the disc was truly absent or matched-but-underivable (no usable VID) (#46).
- ureq 3.4.2 with a bounded, rolling-idle reply body, so a slow-but-progressing download is no longer killed by an absolute deadline.
### Maintenance
- CI moved to the central reusable workflows.
## [1.7.3] — 2026-09-19
### Changed
- Unified release with freemkv 1.7.3 (`info --share` disc-structure capture for bug reports + the multi-angle UHD main-title fix in libfreemkv; see the libfreemkv/freemkv 1.7.3 notes). No functional changes to this crate.
## [1.7.2] — UNRELEASED
### Added
- `OnlineSource` now records the reachability outcome of its actual `/decode` POST (an HTTP `Status(code)` for any answer, including 200/404/422/429/5xx, or `Transport` for a connect/timeout/DNS/TLS failure) in a per-thread slot, readable via the new `take_last_decode_reachability()` / `DecodeReachability`. This lets a caller (autorip) classify a no-key result from the REAL decode's outcome instead of firing a second, redundant probe against the POST-only endpoint. No behavioural change to key resolution itself.
## [1.7.1] — UNRELEASED
### Changed
- Version aligned to 1.7.1 for the unified release (freemkv-unlock 1.7.1 LibreDrive unlock fix); no functional changes.
## [1.7.0] — 2026-09-02
### Changed
- Version aligned to 1.7.0 for the unified release. Internal CI/lint hardening only (stable-clippy MSRV split, cargo-deny dependency-audit gate, audience-based comment-guard); no functional changes.
## [1.6.14] — 2026-08-31
### Changed
- Version aligned to 1.6.14 for the unified release.
## [1.6.13] — 2026-08-28
### Changed
- Version aligned to 1.6.13 for the unified release.
## [1.6.12] — 2026-08-27
### Changed
- Version aligned to 1.6.12 for the unified release; no functional changes.
## [1.6.11] — 2026-08-26
### Changed
- Version aligned to 1.6.11 for the unified release. No functional changes to
this crate; the release was driven by the libfreemkv main-feature selection
improvements and the autorip mux-quarantine fix (see the libfreemkv and
autorip 1.6.11 notes).
### Added
- Codecov coverage reporting and badge.
- Substantially expanded unit-test coverage.
## [1.6.10] — 2026-08-23
### Changed
- Version aligned to 1.6.10 for the unified release. No functional changes to
this crate; the release was driven by libfreemkv (TrueHD/MLP audio now resyncs
to the next major-sync access unit after a source transport-stream
discontinuity, instead of splicing post-gap audio mid-stream — fixing
decoder-choking seams on discs whose stream carries a continuity-counter gap;
see the libfreemkv 1.6.10 notes).
## [1.6.9] — 2026-08-22
### Changed
- Version aligned to 1.6.9 for the unified release. No functional changes to
this crate; the release was driven by autorip (automatic per-episode TV
ripping — each episode named `S{NN}E{MM}`, with TMDB runtime-aligned episode
numbering across multi-disc seasons — a Manual Rename option, and a unified
per-disc staging state file — see the autorip 1.6.9 notes).
## [1.6.8] — 2026-08-21
### Changed
- Version aligned to 1.6.8 for the unified release. No functional changes to
this crate; the release was driven by autorip (webhooks now fire per pipeline
stage — Rip / Mux / Move — with the Rip hook firing the moment the drive is
free again, plus a Ripper-tab activity-banner fix so it also shows during
moves — see the autorip 1.6.8 notes).
## [1.6.7] — 2026-08-21
### Changed
- Version aligned to 1.6.7 for the unified release. No functional changes to
this crate; the release was driven by autorip (per-webhook event selection,
a progress bar per moved artifact, and move-queue / webhook-error fixes —
see the autorip 1.6.7 notes).
## [1.6.6] — 2026-08-20
### Changed
- Version aligned to 1.6.6 for the unified release. No functional changes
to this crate; the release was driven by autorip (webhooks may now target
private/LAN addresses — see the autorip 1.6.6 notes).
## [1.6.5] — 2026-08-20
### Fixed
- **A media, disc-ID, or VUK key at the end of a keydb row is no longer
lost to the comment beside it.** Many `keydb.cfg` rows end with a
`; MKBv…` note. When that note sat directly after the row's final
field and that field was the media key (`M`), disc ID (`I`), or VUK
(`V`) — rather than a unit key (`U`) — the note was read as part of the
value, the hex parse failed, and the key was silently dropped, so the
disc reported no key even though the file held one. The note is now
stripped from those three fields exactly as it already was from the
unit-key field (a hex value never contains `;`, so the split is safe).
A disc that stopped resolving for this reason resolves again.
- **A misconfigured key service is no longer quieter than an outage.** A
mistyped port, an `http://` URL, or a key-service address the SSRF guard
refuses all mean the same thing: the service was never asked, so nothing is
known about the disc. They were reported as an empty result — the same value a
service returns when it genuinely holds no key — so a standing
misconfiguration surfaced to the operator as "no key for this disc" and never
self-corrected, while a transient DNS failure was correctly reported as a
failure. All three are now failures, logged at error level and separated in
the log text from an outage: one says fix the URL, the other says wait. A
later key source's real key still wins, so nothing that used to rip stops
ripping.
- **A replaced `keydb.cfg` can no longer be missed.** The parsed keydb is cached
behind the file's identity, and that identity was size plus modification time
— which are byte-identical when a file is replaced within the same second by
one of the same length, the common shape of a daily refresh on a filesystem
that records whole seconds. The old, superseded keys were then served for the
rest of the process's life while every lookup reported success. The identity
now includes the file's inode, and a cache entry is not trusted until the file
has been unmodified long enough for its timestamp to be meaningful.
- **A corrupt keydb keeps saying so.** The summary warning that counts rejected
and duplicated rows is produced while parsing, and the cache skips parsing —
so a damaged keydb warned once, at whatever moment it was first read, and was
then served in silence to every later disc. The summary is now re-emitted on
every lookup that serves that file. An intact keydb stays silent.
- **A keydb's identity and its contents are read through one file handle.** The
cache used to stat the path and then separately open it, so a replacement
landing between the two stored one file's identity beside another file's keys.
### Security
- **`{:?}` on a keydb no longer spells out its keys.** `KeyDb` and
`DiscEntry` derived their `Debug`, so a single `{:?}` in a log line, a
tracing field, or a panic message wrote out every processing key and
every disc's media key, disc ID, VUK, and unit keys in full — on the
published file, all 184,860 discs' worth. Both are public API, so any
caller could trip it. Both types now format through a redacting `Debug`
that names each field without its bytes, matching the AACS types this
parser was split from, and a test fails if the plain derive ever returns.
- **The host certificate's redaction is now pinned by a test.** `{:?}` on a
keydb host certificate does not print the AACS host private key, and never
did — but that depended entirely on libfreemkv's own redacting formatter,
with nothing in this crate to notice if it changed. The whole-database
formatter's test did not cover it. It does now, along with device keys.
### Changed
- **Breaking (API):** `KeyDb::disc_entries`' key and `DiscEntry::disc_hash` are
`Arc<str>` rather than `String` (a ~13 MB saving on the published keydb, which
is now held in memory). Code that requires an owned `String` from either needs
`.to_string()`. Unreleased in 1.6.4; recorded here because it shipped to git
consumers without a note.
## [1.6.4] — 2026-08-15
### Changed
- **No functional change.** This crate ships alongside the rest of freemkv at a
matching version; its behaviour is untouched.
## [1.6.3] — 2026-08-10
### Changed
- **The key service is reached over a newer HTTP client, with the same
protections.** Before contacting a key service this crate resolves the host,
rejects any private or link-local address, and then pins the connection to the
addresses it just checked, so a hostname cannot be re-pointed at an internal
machine in the moment between the check and the request. That behaviour is
unchanged; only the client underneath it moved.
### Security
- **The address pinning is now proven by a test rather than assumed.** The checks
that reject a bad address were well covered, but nothing verified that the
approved addresses were the ones actually connected to — and every existing
test still passed with that wiring removed entirely, because none of them
opened a connection. A connection made to an unreachable name can only
succeed if the pinned address is honoured, and that is now asserted. No
defect is being fixed here: the pinning was correct. It simply had no way of
telling anyone if it stopped being correct.
## [1.6.2] — 2026-08-08
Version sync with the workspace. No functional change in this crate.
## [1.6.1] — 2026-08-07
Version sync with the workspace. No functional change in this crate.
## [1.6.0] — 2026-08-03
### Fixed
- **Key-service failures are no longer indistinguishable from "no key for this
disc".** `query` returned an empty vector for a 502, a 401, a 429, a
transport error and an unparseable body alike — the same value a successful
lookup with no match returns. It now returns a typed error, and the HTTP
status is classified into the operator action it implies: 401/403 the token,
429 back off, any other non-2xx the service. A genuine miss is a 200 with no
key, so every non-2xx is a failure by definition. A DNS failure or timeout
now counts as unreachable rather than as a bad URL.
Version sync with the workspace (freemkv-engine split release). No source change
in this crate; it remains a pluggable AACS key-source provider consumed by the
`freemkv` CLI and libfreemkv's key resolver.
## [1.5.2] — 2026-07-22
Version sync with the workspace; inherits libfreemkv 1.5.2 (CSS DVD descramble
fix). No source change in this crate.
## [1.4.5] — 2026-07-18
Version sync; inherits libfreemkv 1.4.5. `KeySource` split into `get_unit_keys` +
`get_fmts_indexes`, and a keydb device-key parse bug on an uppercase `0X` hex
prefix was fixed (case-insensitive hex parsing across the toolchain).
## [1.4.4] — 2026-07-17
Version sync; inherits libfreemkv 1.4.4. The online `/decode` request is built from
a `DecodeSampleSet` proven sufficient by type rather than a runtime length check.
## [1.4.3] — 2026-07-17
Version sync; inherits libfreemkv 1.4.3. The online unit-key reply is parsed as a
list (one key for an ordinary disc, the ordered set for a forensic-variant disc),
and `MIN_SAMPLE_UNITS` is re-exported from libfreemkv.
## [1.4.2] — 2026-07-15
Version sync with the workspace; inherits libfreemkv 1.4.2. The keydb test that
feeds real key material into the AACS crypto was adapted to the segregated
`decrypt_unit` + `is_clean` primitives (behaviour unchanged).
## [1.4.1] — 2026-07-14
Version sync with the workspace; inherits libfreemkv 1.4.1.
## [1.4.0] — 2026-07-13
Version sync with the workspace; inherits libfreemkv 1.4.0.
## [1.3.2] — 2026-07-10
### Changed
- Unit keys carry libfreemkv's new `UnitKey.variant_number`; every source
(keydb, online, VUK-derived) emits `0` — ordinary, non-forensic content —
via the `UnitKey::new` constructor. No behaviour change. Inherits
**libfreemkv 1.3.2**.
## [1.3.1] — 2026-07-10
### Licensing
- **Relicensed to the MIT License, from 1.3.1 onwards** (releases up to and
including 1.3.0 remain under AGPL-3.0).
Version sync with the workspace; inherits libfreemkv 1.3.1.
## [1.3.0] — 2026-07-08
### Added
- **AACS 2.0 host certs round-trip through `keydb.cfg`.** `to_keydb_cfg` now
emits the sibling `| HC2 |` line — the inverse of the v2 host-cert parser — so
writing a keydb back out no longer silently drops AACS 2.0 host certs.
### Changed
- **Resolve runs directly on `libfreemkv::aacs` primitives.** After libfreemkv
dropped its `aacs::boil` veneer, the resolve path now calls
`derive_media_key_from_{pk,dk}`, `derive_vuk`, and `decrypt_unit_key` from
`aacs::derive` with the `aacs::types` newtypes. No behaviour change.
- Inherits **libfreemkv 1.3.0**.
### Fixed
- **keydb save-validation matches the parser exactly.** A `0x` line counts as a
disc entry only when it also contains ` = `, so validating and persisting
content that parses to zero usable entries (e.g. a stray `0xDEADBEEF` line) can
no longer succeed.
- **Disc-entry titles round-trip verbatim** (parentheses and all) — the parse
path now keeps the title exactly as the emit path writes it.
## [1.2.0] — 2026-06-29
### Changed
- **One hex parser across the toolchain.** Online and keydb hex inputs now parse
through `libfreemkv::hex`, the same parser the library uses — no separate
decoder with its own length/nibble rules.
- **`DiscInputs` carries the disc's AACS version**, and the tests derive the
`Unit_Key_RO` stride from `inputs.version` instead of hardcoding it, so an
AACS-1.0 (V10, 48-byte) and AACS-2.x (V20/V21, 64-byte) disc are each handled
at their own stride.
- **Online MKB read cap aligned with libfreemkv (64 MiB)**, and an over-cap MKB
is logged rather than silently truncated.
+1 -1
View File
@@ -36,7 +36,7 @@ This Code of Conduct applies within all community spaces, and also applies when
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at owner@example.invalid. All complaints will be reviewed and investigated promptly and fairly.
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported privately to the project maintainer via GitHub at https://github.com/MattJackson. All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of any incident.
+18
View File
@@ -0,0 +1,18 @@
# Contributing
Thanks for your interest!
- **Report a bug** — open an issue
- **Fix a bug** — fork, branch, PR
- **Suggest a feature** — open an issue first
## Development
```bash
cargo build
cargo test
```
## License
MIT
Generated
+1311
View File
File diff suppressed because it is too large Load Diff
+38 -7
View File
@@ -1,25 +1,56 @@
[package]
name = "freemkv-keysources"
version = "1.0.0-rc.5.2"
version = "1.8.0"
edition = "2024"
rust-version = "1.86"
license = "AGPL-3.0-only"
description = "Pluggable AACS key sources (keydb, online key service, mapfile) for libfreemkv. Each source looks a disc up and hands libfreemkv a Key; the library does all derivation."
rust-version = "1.88"
license = "MIT"
description = "Pluggable AACS key sources (keydb, online key service) for libfreemkv. Each source looks a disc up and hands libfreemkv its terminal Unit Keys via get_unit_keys; the library does all derivation."
repository = "https://github.com/freemkv/freemkv-keysources"
keywords = ["aacs", "blu-ray", "uhd", "decryption", "keydb"]
categories = ["multimedia"]
# OFF crates.io: depends on libfreemkv, which is now git-only (it git-deps the
# firmware crate). A crates.io crate can't carry a git dep, so keysources is
# consumed by git tag too. Clients git-tag-pin it.
publish = false
[dependencies]
# The crate provides the `KeySource` trait + `Key`/`DiscInputs` types these impls fill.
libfreemkv = "1.0.0-rc.5.2"
# libfreemkv is git-only now; the committed [patch.crates-io] below redirects this
# bare version req to the libfreemkv git tag (local dev overrides it to a path).
libfreemkv = "1.8"
# OnlineSource: POST disc inputs + samples to a key service over HTTP.
ureq = { version = "2", features = ["json"] }
ureq = { version = "3", features = ["json"] }
serde_json = "1"
base64 = "0.22"
base64 = "0.23"
# KeydbSource::save: decompress a downloaded keydb (.zip / .gz). Same versions
# already in the resolved graph via libfreemkv — no new tree, no new HTTP stack.
flate2 = "1"
zip = { version = "8", default-features = false, features = ["deflate"] }
# Structural begin/end logging around the keyserver round-trip (never logs
# key material).
tracing = "0.1"
[features]
# `set_last_decode_reachability`, for consumers' tests only (enable in dev-dependencies).
test-hooks = []
[dev-dependencies]
# `test_util` fixtures (`decrypt_unit`, …) for this crate's tests (keys-upfront
# KU-P1). CI resolves it through the sibling path patch.
libfreemkv = { version = "1.8", features = ["test-util"] }
[profile.release]
lto = "thin"
codegen-units = 1
# libfreemkv is git-only (off crates.io). Redirect the bare `libfreemkv = "1.6"`
# req above to the git tag — committed and CI-visible. The release script
# rewrites this tag to the new version before regenerating Cargo.lock. Local dev
# overrides it with a path patch via the gitignored .cargo/config.toml (a
# config-level [patch.crates-io] wins over this manifest one for the same crate).
[patch.crates-io]
libfreemkv = { git = "https://github.com/freemkv/libfreemkv", tag = "v1.8.0" }
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.8.0" }
[lints.clippy]
uninlined_format_args = "allow"
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Matthew Jackson & Contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+21
View File
@@ -0,0 +1,21 @@
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
[![CI](https://github.com/freemkv/freemkv-keysources/actions/workflows/ci.yml/badge.svg)](https://github.com/freemkv/freemkv-keysources/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/freemkv/freemkv-keysources/branch/dev/graph/badge.svg)](https://codecov.io/gh/freemkv/freemkv-keysources)
# freemkv-keysources
Pluggable AACS key sources (keydb, online key service) for
[libfreemkv](https://github.com/freemkv/libfreemkv). Each source looks a disc up
and hands libfreemkv its terminal Unit Keys via `get_unit_keys`; the library
does all derivation.
## Development
Build API documentation with `cargo doc --no-deps --open` for source construction,
keydb paths and online-query contracts. Run `cargo test --tests` to validate
parsing, caching and request guards. Tests use synthetic keys and local fixtures;
never commit real key material or credentials.
## License
MIT — see [LICENSE](LICENSE).
+22
View File
@@ -0,0 +1,22 @@
# Security Policy
## Supported versions
| Version | Supported |
| ------- | --------- |
| 1.6.x | Yes |
| < 1.6 | No |
Only the current 1.6.x line receives security fixes.
## Reporting a vulnerability
Report vulnerabilities privately through GitHub Security Advisories:
https://github.com/freemkv/freemkv-keysources/security/advisories/new
Do not open a public issue for a security report. Include the affected
version, steps to reproduce, and the impact you believe the issue has.
## Response time
You will get an initial response within 7 days.
-120
View File
@@ -1,120 +0,0 @@
#!/usr/bin/env bash
#
# leak-guard.sh — self-contained public-repo leak gate.
#
# This is the LAST line of defense in CI. It is intentionally self-contained:
# public CI cannot reach the private tooling, so this script encodes ONLY the
# generic net — internal infrastructure references, agent-context files, and
# AI-attribution in commit messages. It deliberately contains NO project-
# specific reverse-engineering vocabulary (those words would themselves be a
# leak). The richer private scanner stays private.
#
# Fails (exit 1) if any of the following appear in the repo:
# 1. a tracked CLAUDE.md or .claude/ path (agent context — never public),
# 2. tracked file content matching the internal-infra net,
# 3. a commit message (in the given range) with AI attribution.
#
# Usage:
# leak-guard.sh [<commit-range>]
# <commit-range> optional git rev-list range to scan commit messages
# (e.g. "abc..def"). If omitted, commit-message scan is
# skipped (path + content checks always run).
set -euo pipefail
# Absolute path to this script, resolved before any cd, so we can exclude it
# from the content scan (it necessarily contains the detection patterns).
SELF_ABS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/$(basename "${BASH_SOURCE[0]}")"
REPO="$(git rev-parse --show-toplevel)"
cd "$REPO"
fail=0
note() { printf ' ✗ %s\n' "$1"; fail=1; }
# Internal-infra net — GENERIC ONLY. This script ships in the public repo, so
# the patterns themselves must not name any org-specific identifier (doing so
# would itself leak the infra they guard). We catch the leak *class*:
# - RFC1918 private IPv4 ranges (10/8, 172.16/12, 192.168/16),
# - private/internal/non-routable TLDs (.internal/.local/.lan/.corp/.invalid),
# - docker.internal.
# The full org-specific net (literal hostnames, service names, repo paths,
# vendor tooling, …) lives ONLY in the private scanner and never ships here.
INFRA_RE='\b10\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}|\b172\.(1[6-9]|2[0-9]|3[01])\.[0-9]{1,3}\.[0-9]{1,3}|\b192\.168\.[0-9]{1,3}\.[0-9]{1,3}|\.internal\b|\.local\b|\.lan\b|\.corp\b|\.invalid\b|docker\.internal'
# Home-path net — GENERIC ONLY. Catches an absolute developer home path
# committed into a tracked file (a macOS /Users/<user>/… or Linux /home/<user>/…
# path). This names NO specific user — it matches the leak *class* (any home
# path), so the pattern itself reveals nothing org- or person-specific. A real
# leak (e.g. /Users/alice/Developer/x slipping into a public RELEASE.md) trips
# this regardless of whose machine it came from. The username segment is a
# literal-username class ([A-Za-z0-9._-]) so dynamic/templated paths that build
# the user at runtime — shell `/home/$USER/`, doc `/home/<rip>/`, Rust
# `/home/{user}/` — do NOT false-positive; only a baked-in literal home leaks.
HOMEPATH_RE='/Users/[A-Za-z0-9._-]+/|/home/[A-Za-z0-9._-]+/'
# AI-attribution net (case-insensitive). "claude" matches only as a standalone
# word — NOT preceded by a dot/slash/alnum and NOT followed by .md — so legit
# mentions of CLAUDE.md / .claude/ in a commit message don't false-positive.
ATTR_RE='co-authored-by|generated with|🤖|(?<![.\/A-Za-z0-9])claude(?!\.md)'
echo "── leak-guard: tracked agent-context paths ──"
while IFS= read -r f; do
case "$f" in
CLAUDE.md|*/CLAUDE.md|.claude|.claude/*|*/.claude|*/.claude/*)
note "tracked agent-context file: $f (CLAUDE.md/.claude must never be tracked in a public repo)" ;;
esac
done < <(git ls-files)
# Match a PCRE against a file, emitting "LINE: MATCH". The pattern is passed as
# an argument (not interpolated into a //) so metacharacters like the "/" in a
# path-style token can't break the regex. Reads raw bytes so non-UTF-8 blobs
# don't abort the scan.
pcre_matches() {
perl -e '
my ($file, $re) = @ARGV;
open(my $fh, "<:raw", $file) or exit 0;
my $rx; eval { $rx = qr/$re/i }; exit 0 if $@;
while (my $l = <$fh>) { if ($l =~ /$rx/) { print "$.: $&\n"; } }
' "$1" "$2" 2>/dev/null
}
# This script's own source necessarily contains the detection patterns (e.g.
# the regex tokens in INFRA_RE), so scanning it would always self-flag. Skip it.
SELF="$(git ls-files --full-name -- "$SELF_ABS" 2>/dev/null | head -1)"
echo "── leak-guard: internal-infra references in tracked files ──"
while IFS= read -r f; do
case "$f" in *.png|*.jpg|*.jpeg|*.ico|*.gif|*.bin|*.crate|*.gz|*.zip|*.pdf) continue ;; esac
[ -n "$SELF" ] && [ "$f" = "$SELF" ] && continue
[ -f "$f" ] || continue
while IFS= read -r hit; do
[ -z "$hit" ] && continue
note "internal-infra reference: $f:$hit"
done < <(pcre_matches "$f" "$INFRA_RE")
while IFS= read -r hit; do
[ -z "$hit" ] && continue
note "[HOME-PATH] absolute home path: $f:$hit (no local home path may be committed to a public repo)"
done < <(pcre_matches "$f" "$HOMEPATH_RE")
done < <(git ls-files)
RANGE="${1:-}"
if [ -n "$RANGE" ]; then
echo "── leak-guard: AI-attribution in commit messages ($RANGE) ──"
while IFS= read -r sha; do
[ -z "$sha" ] && continue
msg="$(git log -1 --format='%B' "$sha" 2>/dev/null || true)"
# Pass the pattern as an argument (not interpolated into a //) so the
# lookbehind char class and "/" don't break the regex.
hit="$(printf '%s' "$msg" | perl -e '
my $re = $ARGV[0]; my $rx = qr/$re/i;
while (my $l = <STDIN>) { if ($l =~ /($rx)/) { print "$1\n"; last; } }
' "$ATTR_RE" | head -1 || true)"
[ -n "$hit" ] && note "commit ${sha:0:12}: message contains \"$hit\" (owner rule: zero AI attribution, ever)"
done < <(git rev-list "$RANGE" 2>/dev/null || true)
fi
echo
if [ "$fail" -ne 0 ]; then
echo "✗ leak-guard: blocking finding(s) above — DO NOT MERGE/PUBLISH"
exit 1
fi
echo "✓ leak-guard: clean"
+19
View File
@@ -0,0 +1,19 @@
# Coverage is informational only: it never blocks a merge or a release. The
# release cascade is gated by ci.yml / qa.yml and the local precommit, not by
# a coverage threshold.
coverage:
status:
project:
default:
informational: true
patch:
default:
informational: true
# No PR comment noise; the badge and the Codecov UI are enough.
comment: false
# Exclude non-product code from the coverage denominator.
ignore:
- "tests/**"
- "ci/**"
+42
View File
@@ -0,0 +1,42 @@
# cargo-deny — gates the RESOLVED dependency graph (issue #15).
# Runs in CI against the committed Cargo.lock: RUSTSEC advisories, a permissive
# licence allow-list, and source/ban hygiene. Update the lock (cargo update)
# to clear an advisory rather than adding an `ignore` entry without cause.
[advisories]
yanked = "deny"
# ignore = ["RUSTSEC-0000-0000"] # only with a written justification
[licenses]
# The whole tree is permissive. r-efi's LGPL arm is satisfied by Apache-2.0
# (its expression is "Apache-2.0 OR LGPL-2.1-or-later"), so no copyleft is
# allowed here. Add a licence only when a real dependency introduces it.
allow = [
"MIT",
"Apache-2.0",
"Apache-2.0 WITH LLVM-exception",
"0BSD",
"Unicode-3.0",
"Unlicense",
"Zlib",
"BSD-2-Clause",
"BSD-3-Clause",
"ISC",
"CDLA-Permissive-2.0",
]
confidence-threshold = 0.9
[bans]
multiple-versions = "warn" # transitive dupes (digest 0.10/0.11 etc.) are noise, not a gate
wildcards = "deny"
# The freemkv siblings resolve as path deps under CI's path-patch layout, which
# reads as a wildcard version req. That is internal, not a supply-chain wildcard.
allow-wildcard-paths = true
[sources]
unknown-registry = "deny"
unknown-git = "deny"
# The freemkv siblings are first-party git-tag deps; allow git sources from
# our own org only (CI resolves them as git, not the local path override).
[sources.allow-org]
github = ["freemkv"]
+611 -271
View File
@@ -1,30 +1,138 @@
//! `keydb.cfg` key source (source #1).
//!
//! Parses a local `keydb.cfg` and enumerates the material it holds for a disc
//! as candidate [`Key`]s, most-specific first. It does NO derivation — picking
//! which device key applies, or which media key verifies, is the MKB walk, and
//! that lives in libfreemkv (`Disc::decrypt_with`). The candidate order lets
//! the library try each path the keydb could satisfy:
//!
//! 1. per-disc VUK (hash hit) → `Key::Volume`
//! 2. per-disc unit keys (hash hit) → `Key::Unit`
//! 3. per-disc media key (hash hit) → `Key::Media`
//! 4. device-key pool (universal) → `Key::Device` (lib walks the MKB)
//! 5. processing-key pool → `Key::Processing`
//! 6. media-key pool (all entries) → `Key::Media` (lib brutes vs the MKB)
//! Parses a local `keydb.cfg`, looks the disc up by hash, and derives the disc's terminal
//! **Unit Keys** by composing libfreemkv's raw `aacs::derive` primitives — never
//! re-implementing AES. The path mirrors the OLD candidate order EXACTLY, cheapest-first:
//! per-disc Unit Keys, then VUK, then a Media Key (stored / PK pool / DK pool) via
//! `derive_vuk`.
use std::path::PathBuf;
use std::io::{Read, Write};
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::{Arc, Mutex};
use libfreemkv::aacs::{HostCert, KeyDb};
use libfreemkv::{DiscInputs, Key, KeySource};
use crate::uks_from_vuk;
use libfreemkv::aacs::derive::{derive_media_key_from_dk, derive_media_key_from_pk, derive_vuk};
use libfreemkv::aacs::trace::{KeyNode, MatchedEntry};
use libfreemkv::aacs::types::{HostCert, MediaKey, UnitKey, Vid};
use libfreemkv::keysource::{ResolveCtx, UnitKeyResolution};
use libfreemkv::{Error, KeySource};
use crate::keydb_format::KeyDb;
// Decompression-bomb cap on decompressed keydb size: a tiny zip/gz could
// inflate to GiB and OOM the refresh thread. Same constant `KeyDb::load`
// uses on-disk, defined once in keydb_format so the two can't drift.
use crate::keydb_format::MAX_KEYDB_BYTES;
/// Result of a KEYDB save/update -- path written, entry count, and byte size.
#[derive(Debug)]
pub struct UpdateResult {
pub path: PathBuf,
pub entries: usize,
pub bytes: usize,
}
// Widest observed granularity of a filesystem's stored mtime (HFS+, many container/network
// filesystems record whole seconds); 2 s leaves rounding room.
const MTIME_GRANULARITY: std::time::Duration = std::time::Duration::from_secs(2);
// The identity of the keydb file a cache entry was parsed from. `(len, modified)` alone is not
// an identity; `dev`+`ino` plus CacheEntry::is_settled close the gap.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
struct FileStamp {
/// Filesystem device id. `0` where unavailable.
dev: u64,
/// Inode number — the discriminator an atomic rename always changes. `0`
/// where unavailable.
ino: u64,
len: u64,
modified: Option<std::time::SystemTime>,
}
impl FileStamp {
/// Stamp from a metadata block. Take it from `File::metadata` on an OPEN
/// handle (an `fstat`), never from a second `std::fs::metadata` on the path:
/// only the handle guarantees the stamp and the bytes describe one file.
fn of(meta: &std::fs::Metadata) -> Self {
// dev/ino are a Unix concept; Windows's rough equivalent is documented
// unreliable on some filesystems, so the cross-platform fallback is
// simply "no inode discriminator" (rule 2 alone is correct there).
#[cfg(unix)]
let (dev, ino) = {
use std::os::unix::fs::MetadataExt;
(meta.dev(), meta.ino())
};
#[cfg(not(unix))]
let (dev, ino) = (0u64, 0u64);
Self {
dev,
ino,
len: meta.len(),
modified: meta.modified().ok(),
}
}
}
/// One cached parse: the file identity it came from, WHEN that identity was
/// observed, the parsed database, and the parser's rejection counts.
struct CacheEntry {
stamp: FileStamp,
/// Wall-clock time at which `stamp` was read off the open handle. The whole
/// point of storing it is [`Self::is_settled`].
stamped_at: std::time::SystemTime,
db: Arc<KeyDb>,
/// Retained, not just logged: [`KeydbSource::cached_db`] re-emits the
/// summary on every hit, because a cache hit skips `KeyDb::parse` and with
/// it the one warning that a corrupt keydb is being served.
stats: crate::keydb_format::ParseStats,
}
impl CacheEntry {
// Require a settled identity before reusing a cached parse: coarse mtimes can hide
// in-place edits within one timestamp tick. Missing/future mtimes use the age of
// the observation; inode identity detects atomic replacement.
fn is_settled(&self, granularity: std::time::Duration) -> bool {
match self
.stamp
.modified
.and_then(|m| self.stamped_at.duration_since(m).ok())
{
// The normal case: a real mtime that was already `granularity` in
// the past when we stamped it, so any later write bumps the mtime
// past our stamp and is detected. Trust it.
Some(age) => age >= granularity,
// No mtime (mtime-less FS) or a future mtime (clock skew): `duration_since`
// yields None, so the mtime can't discriminate change — settle via inode
// identity once this observation has itself aged past the granularity.
None => std::time::SystemTime::now()
.duration_since(self.stamped_at)
.is_ok_and(|age| age >= granularity),
}
}
}
/// A [`KeySource`] backed by a local `keydb.cfg` file.
///
/// The parsed database is CACHED behind the file's identity stamp: a single AACS-cert rip calls
/// `host_certs()`, the trait `host_certs(mkb)`, and `get_unit_keys`, and each used to re-read +
/// re-parse the whole ~62 MiB file. A replaced keydb is picked up on the next call, with one
/// narrow exception.
pub struct KeydbSource {
path: PathBuf,
/// Lazily-built candidate list (UK ▸ VK ▸ MK ▸ DK ▸ …) plus its cursor —
/// the keydb owns the order and hands one candidate per `next_key`. `None`
/// until the first `next_key` parses the file.
cursor: Option<std::vec::IntoIter<Key>>,
// Mutex, not RwLock: the guarded section is a stamp compare + Arc clone,
// so there's nothing for concurrent readers to win. Poisoning recovers
// rather than propagates — a panic elsewhere must not poison every lookup.
cache: Mutex<Option<CacheEntry>>,
// Cache MISSES (real reads+parses). The only honest way to assert the
// cache from a test, since timing is flaky and the parsed value is
// identical either way.
parses: AtomicUsize,
// The `G` of CacheEntry::is_settled, as a field so tests can isolate the two staleness
// discriminators one at a time (0 vs. a huge duration).
mtime_granularity: std::time::Duration,
// Corruption summaries emitted (see emit_parse_stats). Same rationale as
// `parses`: avoids pulling a `tracing` subscriber into dev-dependencies.
warnings: AtomicUsize,
}
impl KeydbSource {
@@ -32,286 +140,518 @@ impl KeydbSource {
pub fn new(path: impl Into<PathBuf>) -> Self {
Self {
path: path.into(),
cursor: None,
cache: Mutex::new(None),
parses: AtomicUsize::new(0),
mtime_granularity: MTIME_GRANULARITY,
warnings: AtomicUsize::new(0),
}
}
// Log the parser's rejection summary and count the emission — the count
// is the only way to assert, without a `tracing` dev-dependency, that the
// corruption warning is NOT swallowed by the cache. See `cached_db`.
fn emit_parse_stats(&self, stats: &crate::keydb_format::ParseStats) {
if stats.log() {
self.warnings.fetch_add(1, Ordering::Relaxed);
}
}
/// Corruption summaries emitted so far. Test-only observability, paired with
/// [`Self::parse_count`]: a corrupt keydb must warn once per LOOKUP, while
/// still parsing once.
#[cfg(test)]
fn warning_count(&self) -> usize {
self.warnings.load(Ordering::Relaxed)
}
/// Test-only: override the settle window (see the field's doc).
#[cfg(test)]
fn with_mtime_granularity(mut self, g: std::time::Duration) -> Self {
self.mtime_granularity = g;
self
}
// The parsed keydb, from cache when unchanged (errors mirror KeyDb::load). ONE OPEN, ONE
// IDENTITY: stamp (fstat) and bytes share one handle.
fn cached_db(&self) -> std::io::Result<Arc<KeyDb>> {
let file = std::fs::File::open(&self.path)?;
let stamp = FileStamp::of(&file.metadata()?);
let stamped_at = std::time::SystemTime::now();
// Fast path: a settled cache hit under a brief lock (stamp compare +
// Arc clone). Re-emit the rejection summary on EVERY hit — a hit skips
// `KeyDb::parse`, else a corrupt keydb warns once then serves silently.
{
let guard = self.cache.lock().unwrap_or_else(|e| e.into_inner());
if let Some(entry) = guard.as_ref()
&& entry.stamp == stamp
&& entry.is_settled(self.mtime_granularity)
{
self.emit_parse_stats(&entry.stats);
return Ok(entry.db.clone());
}
}
// Miss: parse the ~62 MiB file OUTSIDE the lock so a reparse can't stall
// every other worker. Stamp (fstat) and bytes still come from the ONE
// open handle, preserving the one-open-one-identity invariant.
let (db, stats) = KeyDb::load_counted(file, &self.path)?;
let db = Arc::new(db);
// Re-acquire and double-check: a peer may have installed the same
// settled stamp while we parsed — adopt theirs and drop our redundant
// parse rather than racing a lost update.
let mut guard = self.cache.lock().unwrap_or_else(|e| e.into_inner());
if let Some(entry) = guard.as_ref()
&& entry.stamp == stamp
&& entry.is_settled(self.mtime_granularity)
{
self.emit_parse_stats(&entry.stats);
return Ok(entry.db.clone());
}
self.emit_parse_stats(&stats);
self.parses.fetch_add(1, Ordering::Relaxed);
*guard = Some(CacheEntry {
stamp,
stamped_at,
db: db.clone(),
stats,
});
Ok(db)
}
/// Drop the cached parse. Called after this source WRITES the file, so the
/// next lookup re-reads it without depending on mtime resolution.
fn invalidate_cache(&self) {
*self.cache.lock().unwrap_or_else(|e| e.into_inner()) = None;
}
/// Cache misses so far — the number of real file reads + parses. Test-only
/// observability for the cache (see [`Self::cached_db`]).
#[cfg(test)]
fn parse_count(&self) -> usize {
self.parses.load(Ordering::Relaxed)
}
// Turn a KeyDb::load failure into this source's verdict: MISSING is the documented benign
// case (Ok/None); everything else is logged and surfaced as an error, not silently reported
// as "no key".
fn load_failure(&self, e: &std::io::Error) -> Option<Error> {
match e.kind() {
std::io::ErrorKind::NotFound => None,
std::io::ErrorKind::InvalidData => {
tracing::warn!(
target: "freemkv::keysource",
path = %self.path.display(),
"keydb.cfg is unusable (over the size cap or not valid UTF-8) — this is a corrupt or truncated keydb, NOT a disc without a key"
);
Some(Error::KeydbInvalid)
}
kind => {
tracing::warn!(
target: "freemkv::keysource",
path = %self.path.display(),
io_kind = ?kind,
"keydb.cfg could not be read — NOT a disc without a key"
);
Some(Error::KeydbLoad {
path: self.path.display().to_string(),
})
}
}
}
/// Validate, decompress, and crash-safely persist raw keydb bytes (plain
/// text, `.zip`, or `.gz`) to THIS source's own [`path`](Self::path).
///
/// The bytes are decompressed (zip / gz / plain), checked for at least
/// one recognisable keydb entry, then atomically written to the source's
/// path (sibling-temp + fsync + rename + parent-dir fsync). Decompressed
/// size is capped at [`MAX_KEYDB_BYTES`] (decompression-bomb guard).
/// Writes to the source's own path, not a hardcoded default, so the
/// caller decides the destination (CLI `--keydb`, the autorip service path).
pub fn save(&self, bytes: &[u8]) -> Result<UpdateResult, Error> {
let text = if bytes.starts_with(b"PK\x03\x04") {
extract_zip(bytes)?
} else if bytes.starts_with(&[0x1f, 0x8b]) {
read_capped_to_string(flate2::read::GzDecoder::new(bytes))?
} else {
// Plain-text body: route through the same capped reader as the
// gz/zip branches so an oversized uncompressed upload can't bypass
// MAX_KEYDB_BYTES.
read_capped_to_string(std::io::Cursor::new(bytes))?
};
let entries = text
.lines()
// Mirror KeyDb::parse by running the same parsers, not a prefix check: a
// right-prefix row with bad hex / short cert parses to nothing, so a prefix count
// let junk past `entries > 0`.
.filter(|l| crate::keydb_format::is_parseable_entry_line(l))
.count();
if entries == 0 {
return Err(Error::KeydbInvalid);
}
write_atomic(&self.path, &text)?;
// The file this source reads was just replaced; drop the parsed copy.
self.invalidate_cache();
Ok(UpdateResult {
path: self.path.clone(),
entries,
bytes: text.len(),
})
}
/// Fetch keydb bytes from `url` via the caller-supplied `fetch` transport,
/// then validate + save them to this source's path.
///
/// The transport is INJECTED: this crate stays transport-agnostic on the
/// update path so the application supplies its own TLS / SSRF-guarded fetch
/// (the `freemkv` CLI passes its `keydb_fetch::fetch`). `fetch` returns the
/// raw response body (plain text, `.zip`, or `.gz`); [`save`](Self::save)
/// does the verify + atomic write.
pub fn update(
&self,
fetch: impl Fn(&str) -> Result<Vec<u8>, Error>,
url: &str,
) -> Result<UpdateResult, Error> {
let bytes = fetch(url)?;
self.save(&bytes)
}
/// The host certificate(s) in this keydb — the second kind of data the one
/// keydb file holds (alongside decryption keys). The app passes these to the
/// live-drive scan as `DriveCredentials` for the AACS handshake. Empty if
/// the keydb is missing/unreadable or carries no host cert.
///
/// Best first ([`KeyDb::host_certs_ranked`]): this is what the scan-options
/// builder hands the live-drive handshake as `DriveCredentials`.
pub fn host_certs(&self) -> Vec<HostCert> {
match KeyDb::load(&self.path) {
Ok(db) => db.host_certs,
Err(_) => Vec::new(),
match self.cached_db() {
Ok(db) => db.host_certs_ranked(),
// No error channel here (the scan-options builder wants a list), so
// the failure can only be LOGGED — but it must not be invisible.
Err(e) => {
let _ = self.load_failure(&e);
Vec::new()
}
}
}
/// Build the ordered candidate list from a parsed keydb. Pure (no I/O), so
/// it is unit-testable without a file on disk.
///
/// Order = cheapest + most authoritative first: **UK ▸ VK ▸ MK ▸ DK**. The
/// UK is the final per-CPS-unit content key — zero derivation, directly
/// usable — so it is tried first; the VUK needs one derivation step, an MK
/// two, and the device-key pool the full MKB walk (AACS-1.0-only, slowest),
/// so it is the last-resort fallback. Trying the UK first is also what lets a
/// stale/wrong per-disc VUK be skipped in favour of a good UK in the SAME
/// entry (`decrypt_with` rejects the VUK; the loop falls through to the UK).
fn candidates_from(db: &KeyDb, inputs: &DiscInputs) -> Vec<Key> {
let mut out = Vec::new();
// Return terminal Unit Keys from a parsed keydb without I/O.
// CPS unit numbers are one-based; the returned vector uses zero-based indexes.
fn unit_keys_from(db: &KeyDb, ctx: &dyn ResolveCtx) -> Vec<UnitKey> {
Self::resolve_from(db, ctx).keys
}
// Per-disc hit (most specific). find_disc normalizes the hash form.
if let Some(entry) = db.find_disc(&inputs.disc_hash) {
// UK first — terminal content key, no derivation.
if !entry.unit_keys.is_empty() {
out.push(Key::Unit(entry.unit_keys.clone()));
// De-conflated resolution: the keys PLUS whether the disc matched and — on a
// keyless match — WHY nothing derived, so a matched-but-underivable disc is
// never reported as a flat "no entry" (issue #46). Pure (no I/O).
fn resolve_from(db: &KeyDb, ctx: &dyn ResolveCtx) -> KeydbResolution {
// Per-disc hit (most specific); find_disc normalizes the hash form. Without a match
// there is no per-disc anchor, so the global PK/DK pools are never consulted.
let entries_loaded = db.disc_entries.len();
let Some(entry) = db.find_disc(ctx.disc_hash()) else {
return KeydbResolution::miss(entries_loaded);
};
// UNION every source of terminal keys, then dedup — never first-hit, since a stored
// `unit_keys` list can be PARTIAL while the VUK boils every declared unit.
let mut keys: Vec<UnitKey> = Vec::new();
// 1. Terminal Unit Keys stored in the entry — directly usable, no
// derivation. Preserve the keydb's CPS numbering (idx = num - 1).
for (num, key) in &entry.unit_keys {
// A valid CPS unit number is >= 1; num 0 would collide with unit 1
// at idx 0 (num - 1), so skip it rather than mis-map two units.
if *num == 0 {
continue;
}
// VK next — one step (decrypt Unit_Key_RO.inf).
if let Some(vuk) = entry.vuk {
out.push(Key::Volume(vuk));
}
// MK — two steps (derive the VUK, then the unit keys).
if let Some(mk) = entry.media_key {
out.push(Key::Media(vec![mk]));
keys.push(UnitKey::new(num - 1, *key));
}
// The disc's encrypted title keys (from Unit_Key_RO.inf). Empty when
// the scan captured none, in which case only the stored list (1)
// contributes.
let enc_title_keys = match ctx.enc_title_keys() {
Ok(k) => k,
Err(e) => {
// Mirror online.rs: surface the read failure, then fall back to
// empty (only the stored unit-key list can contribute).
tracing::warn!(
target: "freemkv::keysource",
error = %e,
"keydb: disc encrypted title keys unreadable; deriving without them"
);
&[]
}
};
// Why the keyless derivation could not finish, for the trace's miss
// path. `None` once any key lands (or when there was no material to try).
let mut miss_reason: Option<KeyNode> = None;
if !enc_title_keys.is_empty() {
// VUK path, else MK path (stored/PK/DK) → VUK.
let derived = if let Some(vuk) = entry.vuk {
uks_from_vuk(&vuk, enc_title_keys)
} else {
let vid = ctx.vid().or_else(|| entry.vid.map(Vid));
let mkb = match ctx.mkb() {
Ok(m) => m,
Err(e) => {
// Mirror online.rs: log the failure, then fall back to
// empty (PK/DK derivation may then find no Media Key).
tracing::warn!(
target: "freemkv::keysource",
error = %e,
"keydb: disc MKB unreadable; media-key derivation may fail"
);
&[]
}
};
let mk: Option<MediaKey> = entry
.media_key
.map(MediaKey)
.or_else(|| derive_media_key_from_pk(mkb, &db.processing_keys).map(MediaKey))
// DK pool: the real Subset-Difference MKB walk. No VID at the MK
// step (it enters at the VUK step below); the VID guard follows.
.or_else(|| derive_media_key_from_dk(mkb, &db.device_keys).map(MediaKey));
match (mk, vid) {
// VUK = derive_vuk(MK, VID), then boil the disc's encrypted
// title keys to the terminal Unit Keys.
(Some(mk), Some(vid)) => {
uks_from_vuk(&derive_vuk(&mk.0, &vid.0), enc_title_keys)
}
// Locked VID-per-path rule: an MK with no VID cannot derive.
// De-conflate the two reasons so the trace can say WHICH.
(Some(_), None) => {
miss_reason = Some(KeyNode::NoVid);
Vec::new()
}
(None, _) => {
miss_reason = Some(KeyNode::NoDerivableKey);
Vec::new()
}
}
};
keys.extend(derived);
}
// Universal material — the library walks/brutes it against this disc's
// MKB and VID.
if !db.device_keys.is_empty() {
out.push(Key::Device(db.device_keys.clone()));
}
if !db.processing_keys.is_empty() {
out.push(Key::Processing(db.processing_keys.clone()));
}
// Unique by key value, first occurrence wins (stored numbering kept).
let mut seen = std::collections::HashSet::new();
keys.retain(|u| seen.insert(u.key));
// Media-key pool across every entry: an MK is MKB-scoped, so a sibling
// disc's MK may verify against this disc (the path-2.5 brute). Hand the
// whole pool; the library picks the one that verifies.
let mk_pool: Vec<[u8; 16]> = db.iter_disc_entries().filter_map(|e| e.media_key).collect();
if !mk_pool.is_empty() {
out.push(Key::Media(mk_pool));
}
// Booleans-and-lengths shape of the matched entry, for the app to log —
// answers "what did the matched entry actually carry?" (issue #46).
let shape = MatchedEntry {
has_vuk: entry.vuk.is_some(),
has_unit_keys: !entry.unit_keys.is_empty(),
unit_keys_len: entry.unit_keys.len(),
has_media_key: entry.media_key.is_some(),
has_keydb_vid: entry.vid.is_some(),
enc_title_keys_len: enc_title_keys.len(),
vid_available: ctx.vid().is_some() || entry.vid.is_some(),
};
out
// On a match with no key: carry the specific reason if we have one, else
// leave it empty for the library to render a bare `NoDerivableKey`.
// KU J23: `NoVid` stays beside partial keys (the VID would derive the rest).
let miss_path = match miss_reason {
Some(n) if keys.is_empty() || n == KeyNode::NoVid => vec![n],
_ => Vec::new(),
};
KeydbResolution {
keys,
matched: true,
shape: Some(shape),
miss_path,
entries_loaded,
}
}
}
impl KeySource for KeydbSource {
/// Expose the keydb's host certs through the trait — the OEM/AACS cert-auth
/// route collects them across every source via this method. Delegates to the
/// inherent [`KeydbSource::host_certs`] (same `| HC |`/`| HC2 |` rows parsed
/// by libfreemkv's keydb parser); no new parsing.
fn host_certs(&self) -> Vec<HostCert> {
KeydbSource::host_certs(self)
// The de-conflated result of a keydb lookup: keys plus enough context that a
// MATCHED-but-underivable disc is reported distinctly from a true miss. See
// [`KeydbSource::resolve_from`] and issue #46.
struct KeydbResolution {
keys: Vec<UnitKey>,
matched: bool,
shape: Option<MatchedEntry>,
miss_path: Vec<KeyNode>,
// Per-disc entries loaded in the keydb — named in a true-miss verdict so a
// reporter can confirm a wrong-pressing (`… not in keydb (N entries loaded)`).
entries_loaded: usize,
}
impl KeydbResolution {
// A true miss: the disc hash was not in the keydb at all.
fn miss(entries_loaded: usize) -> Self {
Self {
keys: Vec::new(),
matched: false,
shape: None,
miss_path: Vec::new(),
entries_loaded,
}
}
}
/// Read a decompressed stream into a `String` with a hard size ceiling.
/// Returns [`Error::KeydbInvalid`] if the input exceeds the cap, or
/// [`Error::KeydbParse`] if the bytes are not valid UTF-8.
fn read_capped_to_string<R: Read>(reader: R) -> Result<String, Error> {
let mut buf = Vec::new();
// Read one byte past the cap so an exactly-at-cap stream is accepted but
// anything larger is rejected.
reader
.take(MAX_KEYDB_BYTES + 1)
.read_to_end(&mut buf)
.map_err(|_| Error::KeydbParse)?;
if buf.len() as u64 > MAX_KEYDB_BYTES {
return Err(Error::KeydbInvalid);
}
String::from_utf8(buf).map_err(|_| Error::KeydbParse)
}
/// Extract the first `*.cfg` member of a zip archive as a capped `String`.
fn extract_zip(data: &[u8]) -> Result<String, Error> {
let cursor = std::io::Cursor::new(data);
let mut archive = zip::ZipArchive::new(cursor).map_err(|_| Error::KeydbParse)?;
for i in 0..archive.len() {
let file = archive.by_index(i).map_err(|_| Error::KeydbParse)?;
if file.name().ends_with(".cfg") || file.name().ends_with(".CFG") {
return read_capped_to_string(file);
}
}
/// The keydb can hand out a per-disc **terminal** `Key::Unit` (a UK entry
/// keyed on `disc_hash` alone — see `candidates_from`). Unlike a derived key
/// (Device/Processing/Media/Volume), a terminal UK is applied as-is by
/// `Disc::decrypt_with`: it is NOT re-derived through the MKB-verified AACS
/// resolver, so a UK entry whose hash matches the disc but whose key bytes
/// are wrong would commit and mux undecryptable video as "success". The only
/// thing that disproves a wrong UK is descrambling real ciphertext, so this
/// source requires content samples — without them `decrypt_with` skips
/// validation and the wrong UK is taken. Returning `true` makes every
/// consumer (autorip resume/mux-worker AND the CLI) sample units before
/// resolving, so a keydb UK is ciphertext-validated on every path.
fn needs_samples(&self) -> bool {
true
Err(Error::KeydbInvalid)
}
// Write `text` to `path` crash-safely (temp file, fsync, atomic rename) so an interrupted
// update never leaves a half-written keydb.
fn write_atomic(path: &Path, text: &str) -> Result<(), Error> {
let werr = || Error::KeydbWrite {
path: path.display().to_string(),
};
if let Some(dir) = path.parent() {
std::fs::create_dir_all(dir).map_err(|e| {
tracing::warn!(error = %e, path = %path.display(), "keydb dir create failed");
werr()
})?;
}
let tmp = {
use std::sync::atomic::{AtomicU64, Ordering};
static TMP_COUNTER: AtomicU64 = AtomicU64::new(0);
path.with_extension(format!(
"tmp.{}.{}",
std::process::id(),
TMP_COUNTER.fetch_add(1, Ordering::Relaxed)
))
};
let write_result = (|| -> std::io::Result<()> {
// The temp file is renamed onto keydb.cfg, which holds AACS key
// material and the host private key/cert — create it 0600 on Unix so
// umask can't leave the keys world-readable. Non-unix keeps create().
#[cfg(unix)]
let mut f = {
use std::os::unix::fs::OpenOptionsExt;
std::fs::OpenOptions::new()
.write(true)
.create(true)
.truncate(true)
.mode(0o600)
.open(&tmp)?
};
#[cfg(not(unix))]
let mut f = std::fs::File::create(&tmp)?;
f.write_all(text.as_bytes())?;
f.sync_all()?;
Ok(())
})();
if let Err(e) = write_result {
let _ = std::fs::remove_file(&tmp);
tracing::warn!(error = %e, path = %path.display(), "keydb write/fsync failed; keydb unchanged");
return Err(werr());
}
if let Err(e) = std::fs::rename(&tmp, path) {
let _ = std::fs::remove_file(&tmp);
tracing::warn!(error = %e, path = %path.display(), "keydb rename failed; keydb unchanged");
return Err(werr());
}
// Durably commit the new dirent: on POSIX filesystems (ext2, some NFS) a
// crash right after the rename can lose the directory entry even though the
// rename returned. Best-effort (swallowed on failure); no-op on Windows.
if let Some(dir) = path.parent() {
libfreemkv::io::fsync::dir(dir);
}
Ok(())
}
impl KeySource for KeydbSource {
// Resolve this disc's base per-CPS-unit Unit Keys from the keydb. A MISSING keydb yields no
// keys (Ok(empty)); an UNUSABLE one is a source failure (Err) — never conflated.
fn get_unit_keys(&self, ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, Error> {
match self.cached_db() {
Ok(db) => Ok(Self::unit_keys_from(&db, ctx)),
Err(e) => match self.load_failure(&e) {
// Missing keydb: the documented benign miss.
None => Ok(Vec::new()),
// Corrupt / unreadable keydb: a SOURCE FAILURE, not a miss.
Some(err) => Err(err),
},
}
}
// De-conflated resolution (issue #46): besides the keys, report whether the
// disc MATCHED, why nothing derived on a keyless match, and the matched
// entry's shape to log — so a matched-no-key disc is not a mute "no entry".
fn resolve_unit_keys(&self, ctx: &dyn ResolveCtx) -> Result<UnitKeyResolution, Error> {
match self.cached_db() {
Ok(db) => {
let r = Self::resolve_from(&db, ctx);
Ok(UnitKeyResolution {
keys: r.keys,
matched: r.matched,
miss_path: r.miss_path,
matched_entry: r.shape,
store_entries: Some(r.entries_loaded),
})
}
Err(e) => match self.load_failure(&e) {
// Missing keydb: the documented benign miss (nothing matched).
None => Ok(UnitKeyResolution::default()),
// Corrupt / unreadable keydb: a SOURCE FAILURE, not a miss.
Some(err) => Err(err),
},
}
}
// The keydb's host certs, best first. `mkb` is ignored: cert selection takes no disc input.
fn host_certs(&self, _mkb: Option<u32>) -> Vec<HostCert> {
match self.cached_db() {
Ok(db) => db.host_certs_ranked(),
// Vec-returning trait method: log the failure, return nothing.
Err(e) => {
let _ = self.load_failure(&e);
Vec::new()
}
}
}
// Keyed by disc hash, not by samples: asked once per resolve, not per
// piece (KU §2.3 step 8, N-KU10; KSK1).
fn answer_depends_on_samples(&self) -> bool {
false
}
fn label(&self) -> &'static str {
"keydb"
}
fn next_key(&mut self, inputs: &DiscInputs) -> Option<Key> {
// On the first ask, parse the keydb once and build the ordered candidate
// list; later asks just advance the cursor. A missing/unreadable keydb
// is not an error — it simply yields no candidates (another source may
// have the key), the same as the library's own loader.
if self.cursor.is_none() {
let cands = match KeyDb::load(&self.path) {
Ok(db) => Self::candidates_from(&db, inputs),
Err(_) => Vec::new(),
};
self.cursor = Some(cands.into_iter());
}
self.cursor.as_mut().and_then(Iterator::next)
}
}
#[cfg(test)]
mod tests {
use super::*;
use libfreemkv::aacs::{DeviceKey, DiscEntry};
use std::collections::HashMap;
fn inputs(hash: &str) -> DiscInputs {
DiscInputs {
disc_hash: hash.into(),
volume_id: [0u8; 16],
mkb: Vec::new(),
unit_key_ro: Vec::new(),
samples: Vec::new(),
volume_label: None,
}
}
fn dk() -> DeviceKey {
DeviceKey {
key: [0x22u8; 16],
node: 1,
uv: 2,
u_mask_shift: 0,
}
}
fn entry_with_vuk(hash: &str, vuk: [u8; 16]) -> DiscEntry {
DiscEntry {
disc_hash: hash.into(),
title: String::new(),
media_key: None,
disc_id: None,
vuk: Some(vuk),
unit_keys: Vec::new(),
}
}
#[test]
fn per_disc_vuk_ranks_before_device_pool() {
let mut entries = HashMap::new();
entries.insert("0xaabb".into(), entry_with_vuk("0xaabb", [0x11u8; 16]));
let db = KeyDb {
device_keys: vec![dk()],
processing_keys: Vec::new(),
host_certs: Vec::new(),
disc_entries: entries,
};
let cands = KeydbSource::candidates_from(&db, &inputs("0xaabb"));
assert!(
matches!(cands.first(), Some(Key::Volume(v)) if *v == [0x11u8; 16]),
"the disc's own VUK must be the first (most specific) candidate"
);
assert!(
cands.iter().any(|k| matches!(k, Key::Device(_))),
"the universal device-key pool is still offered as a fallback"
);
}
#[test]
fn per_disc_uk_ranks_before_vuk() {
// An entry with BOTH a UK and a VUK (the dual-key shape) must hand the
// terminal UK out first, so a stale/wrong VUK never pre-empts a good UK.
let mut entries = HashMap::new();
let mut e = entry_with_vuk("0xaabb", [0x11u8; 16]);
e.unit_keys = vec![(1, [0x22u8; 16])];
entries.insert("0xaabb".into(), e);
let db = KeyDb {
device_keys: Vec::new(),
processing_keys: Vec::new(),
host_certs: Vec::new(),
disc_entries: entries,
};
let cands = KeydbSource::candidates_from(&db, &inputs("0xaabb"));
assert!(
matches!(cands.first(), Some(Key::Unit(_))),
"the terminal UK must be the first candidate"
);
assert!(
matches!(cands.get(1), Some(Key::Volume(v)) if *v == [0x11u8; 16]),
"the VUK follows the UK"
);
}
#[test]
fn no_disc_hit_offers_only_universal_material() {
let db = KeyDb {
device_keys: vec![dk()],
processing_keys: Vec::new(),
host_certs: Vec::new(),
disc_entries: HashMap::new(),
};
// A disc with no per-disc entry: no Volume/Unit candidate, just the pool.
let cands = KeydbSource::candidates_from(&db, &inputs("0xdeadbeef"));
assert!(cands.iter().all(|k| matches!(k, Key::Device(_))));
assert_eq!(cands.len(), 1);
}
#[test]
fn empty_keydb_offers_nothing() {
let db = KeyDb {
device_keys: Vec::new(),
processing_keys: Vec::new(),
host_certs: Vec::new(),
disc_entries: HashMap::new(),
};
assert!(KeydbSource::candidates_from(&db, &inputs("0xaabb")).is_empty());
}
/// Regression: a keydb can hand out a per-disc terminal `Key::Unit` that
/// `Disc::decrypt_with` applies WITHOUT re-deriving through the MKB-verified
/// AACS resolver. The only thing that disproves a wrong UK is descrambling
/// ciphertext, so the source MUST request content samples — otherwise the
/// autorip resume/mux-worker path (which only samples when some source
/// reports `needs_samples()`) resolves with empty samples and commits a
/// wrong UK as success. Was `false` (inherited default); must be `true`.
#[test]
fn keydb_source_needs_samples() {
let src = KeydbSource::new("/nonexistent/path/keydb.cfg");
assert!(
src.needs_samples(),
"keydb emits terminal Key::Unit entries that need ciphertext validation"
);
}
/// No keydb (or a LibreDrive deployment) → no host credentials, not an
/// error. (The positive parse is NOT tested here — it would require host
/// key material, which must never appear in code.)
#[test]
fn host_certs_empty_when_keydb_missing() {
assert!(
KeydbSource::new("/nonexistent/path/keydb.cfg")
.host_certs()
.is_empty()
);
}
/// The KeySource TRAIT method exposes the keydb's host cert(s) — this is the
/// path the OEM/AACS cert-auth route collects certs through. A keydb with a
/// `| HC |` row must surface a HostCert via `KeySource::host_certs`, so the
/// handshake (which iterates `opts.key_sources[..].host_certs()`) finds it.
/// Placeholder all-zero material (never a real key) — same convention as
/// libfreemkv's own `parse_host_cert` test.
#[test]
fn trait_host_certs_returns_keydb_hc_row() {
let dir = std::env::temp_dir().join(format!("fmk_hc_{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let path = dir.join("keydb.cfg");
let line = format!(
"| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{}\n",
"00".repeat(20),
"00".repeat(92)
);
std::fs::write(&path, line).unwrap();
let src = KeydbSource::new(&path);
// Consult through the TRAIT, exactly as the OEM route does.
let certs = KeySource::host_certs(&src);
assert_eq!(certs.len(), 1, "trait host_certs must surface the HC row");
assert_eq!(certs[0].certificate.len(), 92);
std::fs::remove_dir_all(&dir).ok();
}
/// Zero certs from a (missing) keydb through the TRAIT method — the OEM route
/// sees an empty vec here and, with no other source supplying a cert, fails
/// gracefully with `AacsNoHostCert` rather than panicking.
#[test]
fn trait_host_certs_empty_when_keydb_missing() {
let src = KeydbSource::new("/nonexistent/path/keydb.cfg");
assert!(KeySource::host_certs(&src).is_empty());
}
}
#[path = "keydb_tests.rs"]
mod tests;
+869
View File
@@ -0,0 +1,869 @@
//! AACS Key Database parsing — KEYDB.cfg format.
//!
//! Byte-faithful copy of libfreemkv's `aacs::keydb` parser, relocated so the
//! keydb.cfg format lives with the key sources that consume it. The parsing
//! logic is identical; the only deviation is [`KeyDb::load`], which returns a
//! standalone [`std::io::Result`] here instead of `libfreemkv::error::Result`
//! (so the format crate carries no dependency on libfreemkv's error type).
#![allow(dead_code)]
use std::collections::HashMap;
use std::sync::Arc;
use libfreemkv::aacs::types::{DeviceKey, HostCert};
/// A keydb per-disc unit key: the CPS-unit number paired with its 16-byte key.
pub type NumberedUnitKey = (u32, [u8; 16]);
/// Upper bound on the keydb.cfg byte size (the public UHD keydb is ~62 MiB;
/// 128 MiB bounds a hostile/corrupt file). The SINGLE definition for the crate:
/// [`KeyDb::load`] uses it as the on-disk load cap and [`crate::KeydbSource`]
/// re-uses it as the decompression-bomb cap on the save/update path, so the two
/// can never drift out of lockstep.
pub(crate) const MAX_KEYDB_BYTES: u64 = 128 * 1024 * 1024;
/// Parsed AACS key database.
///
/// NO `#[derive(Debug)]`: see the hand-written redacting impl below. The
/// derive dumped every processing key and all ~181k discs' key material into
/// any `{:?}`.
pub struct KeyDb {
/// Device keys for MKB processing
pub device_keys: Vec<DeviceKey>,
/// Processing keys (pre-computed media keys for specific MKB versions)
pub processing_keys: Vec<[u8; 16]>,
/// Host certificate + private key for SCSI authentication, paired with the
/// keydb's revocation metadata (libfreemkv's `HostCert` stays pure; the
/// `Revoked in MKBv<N>` annotation is tracked in this crate).
pub host_certs: Vec<KeydbHostCert>,
/// Per-disc VUK entries indexed by disc hash (hex lowercase).
///
/// The key is `Arc<str>` and is the SAME allocation as the entry's own
/// [`DiscEntry::disc_hash`] — the map used to own a second `String` clone
/// of every hash, ~13 MB of pure duplication across the real keydb's 181k
/// entries (and now that [`crate::KeydbSource`] caches the parsed db for
/// the process lifetime, that duplication is resident, not transient).
/// `Arc<str>: Borrow<str>`, so lookups still take a plain `&str`.
pub disc_entries: HashMap<Arc<str>, DiscEntry>,
}
// Key material must never leak via `{:?}`; both types below carry raw AACS key bytes.
impl std::fmt::Debug for KeyDb {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("KeyDb")
.field("device_keys_len", &self.device_keys.len())
.field("processing_keys_len", &self.processing_keys.len())
.field("host_certs_len", &self.host_certs.len())
.field("disc_entries_len", &self.disc_entries.len())
.finish()
}
}
impl std::fmt::Debug for DiscEntry {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DiscEntry")
.field("disc_hash", &self.disc_hash)
.field("title", &self.title)
.field("media_key", &self.media_key.map(|_| "<redacted>"))
.field("vid", &self.vid.map(|_| "<redacted>"))
.field("vuk", &self.vuk.map(|_| "<redacted>"))
.field("unit_keys_len", &self.unit_keys.len())
.field("mkb_version", &self.mkb_version)
.field("volume_size", &self.volume_size)
.field("is_uhd", &self.is_uhd)
.finish()
}
}
/// A keydb host certificate together with its revocation generation.
///
/// libfreemkv's [`HostCert`] is intentionally crypto-pure and carries no
/// revocation state; the keydb's `; Revoked in MKBv<N>` comment is parsed
/// here and stored alongside the cert so callers can filter by MKB generation
/// without modifying the library type.
#[derive(Debug, Clone)]
pub struct KeydbHostCert {
/// The pure libfreemkv host certificate + private key(s).
pub cert: HostCert,
/// The MKB generation at which this host cert was revoked, parsed from a
/// `; Revoked in MKBv<N>` comment. `None` when the cert carries no such
/// annotation (treated as never-revoked).
pub revoked_at_mkb: Option<u32>,
}
/// A per-disc entry from the key database.
///
/// NO `#[derive(Debug)]`: see the hand-written redacting impl above.
#[derive(Clone)]
pub struct DiscEntry {
/// Disc hash (20 bytes, hex). `Arc<str>` so the entry and the
/// [`KeyDb::disc_entries`] key that points at it share ONE allocation
/// instead of two.
pub disc_hash: Arc<str>,
/// Disc title
pub title: String,
/// Media Key (16 bytes) — from MKB processing
pub media_key: Option<[u8; 16]>,
/// Volume ID — the AACS VID (the keydb `I` token), 16 bytes. NOT the disc's
/// identity (that's `disc_hash`); this is the per-disc Volume ID used to
/// derive the VUK.
pub vid: Option<[u8; 16]>,
/// Volume Unique Key (16 bytes) — decrypts title keys
pub vuk: Option<[u8; 16]>,
/// Unit keys (title keys) indexed by CPS unit number
pub unit_keys: Vec<NumberedUnitKey>,
/// MKB version parsed from the trailing `; MKBv<N>` comment, if present.
pub mkb_version: Option<u32>,
/// Volume size in bytes parsed from `VolumeSize: <N>` in the comment.
pub volume_size: Option<u64>,
/// True if the comment contains the literal `(UHD)` flag.
pub is_uhd: bool,
}
// Parse a hex string like "0xABCD..." into bytes. Operates on bytes, not
// `&str` char boundaries: the keydb is third-party content, so a non-ASCII
// scalar must not panic on a mid-codepoint slice. Non-hex byte -> `None`.
pub(crate) fn parse_hex(s: &str) -> Option<Vec<u8>> {
// The one workspace hex parser (strips an optional 0x/0X, byte-based).
libfreemkv::hex::parse_hex_bytes(s)
}
// Read digits right after the first `marker` in `text`. Byte-based so
// untrusted comment text never panics on a char boundary; skips whitespace
// before the digits, serving both `MKBv<N>` and `VolumeSize: <N>`.
fn parse_digits_after<T: std::str::FromStr>(text: &str, marker: &str) -> Option<T> {
let bytes = text.as_bytes();
let start = text.find(marker)? + marker.len();
let mut i = start;
// Skip any whitespace between the marker and the digits.
while i < bytes.len() && bytes[i].is_ascii_whitespace() {
i += 1;
}
let digit_start = i;
while i < bytes.len() && bytes[i].is_ascii_digit() {
i += 1;
}
if i == digit_start {
return None;
}
// The digit run is pure ASCII, so this slice is a valid str.
std::str::from_utf8(&bytes[digit_start..i])
.ok()?
.parse()
.ok()
}
/// Parse the host-cert revocation generation from a `Revoked in MKBv<N>`
/// comment on a `| HC |`/`| HC2 |` line. `None` when absent.
fn parse_revoked_at_mkb(line: &str) -> Option<u32> {
parse_digits_after(line, "Revoked in MKBv")
}
/// Parse hex into a fixed-size array.
pub(crate) fn parse_hex16(s: &str) -> Option<[u8; 16]> {
libfreemkv::hex::parse_hex_fixed::<16>(s)
}
pub(crate) fn parse_hex20(s: &str) -> Option<[u8; 20]> {
libfreemkv::hex::parse_hex_fixed::<20>(s)
}
// What `KeyDb::parse` threw away, by reason.
#[derive(Debug, Default, Clone, Copy, PartialEq, Eq)]
pub(crate) struct ParseStats {
/// `| DK` rows that parsed as neither a positioned nor an orphan device key.
pub dk_rejected: usize,
/// `| PK` rows whose key field did not yield 16 bytes.
pub pk_rejected: usize,
/// `| HC` rows rejected (bad hex, short cert, missing field).
pub hc_rejected: usize,
/// `| HC2` rows rejected (bad hex, wrong priv length, short cert).
pub hc2_rejected: usize,
/// `0x… = …` disc rows the field parser refused outright.
pub disc_rejected: usize,
/// Disc rows that REPLACED an earlier row with the same hash (last wins).
/// Not necessarily corruption, but never something to discover silently:
/// a duplicated hash means one of the two rows' keys is now unreachable.
pub disc_duplicate: usize,
}
impl ParseStats {
/// Total rows the parser refused. Excludes duplicates, which were parsed
/// successfully and merely overwrote a sibling.
fn rejected(&self) -> usize {
self.dk_rejected
+ self.pk_rejected
+ self.hc_rejected
+ self.hc2_rejected
+ self.disc_rejected
}
// Emit the one summary line, if there is anything to say; returns whether
// it fired (tests use this). `pub(crate)` so `KeydbSource` can re-emit it
// on a cache hit too, keeping the warning from going silent forever.
pub(crate) fn log(&self) -> bool {
if self.rejected() == 0 && self.disc_duplicate == 0 {
return false;
}
tracing::warn!(
target: "freemkv::keysource",
dk_rejected = self.dk_rejected,
pk_rejected = self.pk_rejected,
hc_rejected = self.hc_rejected,
hc2_rejected = self.hc2_rejected,
disc_rejected = self.disc_rejected,
disc_duplicate = self.disc_duplicate,
"keydb.cfg lines were rejected while parsing; the file may be truncated or corrupt (keys past the damage will not resolve)"
);
true
}
}
// True when `line` opens a per-disc row. Matches `0x`/`0X` case-insensitively
// (like `parse_hex`) so an uppercase row isn't silently dropped. Shared with
// `KeydbSource::save`, whose entry count must match what this parser accepts.
pub(crate) fn is_disc_entry_line(line: &str) -> bool {
(line.starts_with("0x") || line.starts_with("0X")) && line.contains(" = ")
}
/// True when `line` is a row `KeyDb::parse` would actually ACCEPT into the db — a disc row OR a
/// DK/PK/HC/HC2 row that its real parser accepts, not merely one carrying the right `| XX`
/// prefix. `KeydbSource::save`'s "won't persist unparseable content" guard counts entries with
/// THIS, so a syntactically prefixed but malformed DK/PK/HC row (right prefix, bad hex/short
/// cert) can no longer inflate the entry count past the `entries > 0` check. Mirrors the
/// dispatch in [`KeyDb::parse_counted`] exactly.
pub(crate) fn is_parseable_entry_line(line: &str) -> bool {
let line = line.trim();
if line.is_empty() || line.starts_with(';') || line.starts_with('#') {
return false;
}
// Order matches `parse_counted`: HC2 before HC (both share the `| HC`
// prefix); an orphan DK (no position fields) still counts as a DK.
if line.starts_with("| DK") {
return KeyDb::parse_device_key(line).is_some() || KeyDb::parse_orphan_dk(line).is_some();
}
if line.starts_with("| PK") {
return KeyDb::parse_processing_key(line).is_some();
}
if line.starts_with("| HC2") {
return KeyDb::parse_host_cert_v2(line).is_some();
}
if line.starts_with("| HC") {
return KeyDb::parse_host_cert(line).is_some();
}
is_disc_entry_line(line) && KeyDb::parse_disc_entry(line).is_some()
}
impl KeyDb {
/// Construct an empty KeyDb. Used by unit tests; production code
/// reaches a populated KeyDb via [`KeyDb::load`] or [`KeyDb::parse`].
pub fn empty() -> Self {
KeyDb {
device_keys: Vec::new(),
processing_keys: Vec::new(),
host_certs: Vec::new(),
disc_entries: HashMap::new(),
}
}
/// Parse a KEYDB.cfg file from a string.
///
/// Lines the parser cannot use are counted and summarised in ONE
/// `tracing::warn!` (see [`ParseStats`]) — they are never dropped silently.
pub fn parse(data: &str) -> Self {
let (db, stats) = Self::parse_counted(data);
stats.log();
db
}
/// [`Self::parse`] without the log — the seam the rejection-counting tests
/// assert on (a `tracing` subscriber would be the only other way to observe
/// a count, and the count is the thing under test, not its formatting).
pub(crate) fn parse_counted(data: &str) -> (Self, ParseStats) {
let mut stats = ParseStats::default();
let mut db = KeyDb {
device_keys: Vec::new(),
processing_keys: Vec::new(),
host_certs: Vec::new(),
disc_entries: HashMap::new(),
};
// Strip a leading UTF-8 BOM (U+FEFF): it is NOT trim()-able whitespace, so
// it would cling to line 1's `0x…` hash and silently drop the first disc
// row (a keydb saved by a Windows editor).
let data = data.strip_prefix('\u{feff}').unwrap_or(data);
for line in data.lines() {
let line = line.trim();
// Skip comments and empty lines
if line.is_empty() || line.starts_with(';') || line.starts_with('#') {
continue;
}
// Device Key: positioned rows -> `device_keys` (tree walk); orphan
// rows (no position fields) -> `processing_keys` (brute walker).
// Per AACS a "PK" IS a DK at terminal position, so both are DKs here.
if line.starts_with("| DK") {
if let Some(dk) = Self::parse_device_key(line) {
db.device_keys.push(dk);
} else if let Some(key) = Self::parse_orphan_dk(line) {
db.processing_keys.push(key);
} else {
stats.dk_rejected += 1;
}
continue;
}
// Processing Key
if line.starts_with("| PK") {
if let Some(pk) = Self::parse_processing_key(line) {
db.processing_keys.push(pk);
} else {
stats.pk_rejected += 1;
}
continue;
}
// Host Certificate (AACS 2.0). Normally augments the preceding HC
// (AACS 1.0) row, but an HC2 row may appear first (third-party line
// ordering); then it's carried on a fresh HostCert with an empty v1 cert.
if line.starts_with("| HC2") {
if let Some((pk, cert, revoked_at_mkb)) = Self::parse_host_cert_v2(line) {
if let Some(hc) = db.host_certs.last_mut() {
hc.cert.private_key_v2 = Some(pk);
hc.cert.certificate_v2 = Some(cert);
// The revocation annotation can live on the HC2 line
// instead of the HC line; carry it onto the combined
// cert if the HC line had none, so it isn't lost.
if hc.revoked_at_mkb.is_none() {
hc.revoked_at_mkb = revoked_at_mkb;
}
} else {
db.host_certs.push(KeydbHostCert {
cert: HostCert {
private_key: [0u8; 20],
certificate: Vec::new(),
private_key_v2: Some(pk),
certificate_v2: Some(cert),
},
revoked_at_mkb,
});
}
} else {
stats.hc2_rejected += 1;
}
continue;
}
// Host Certificate (AACS 1.0)
if line.starts_with("| HC") {
if let Some(hc) = Self::parse_host_cert(line) {
db.host_certs.push(hc);
} else {
stats.hc_rejected += 1;
}
continue;
}
// Gate on the `0x` PREFIX alone (not `is_disc_entry_line`, which also
// needs " = "): a malformed `0x…` row is COUNTED as rejected, never
// silently dropped. No entry cap — a keydb must hold every row.
if line.starts_with("0x") || line.starts_with("0X") {
match Self::parse_disc_entry(line) {
// The map key IS the entry's own `disc_hash` allocation
// (`Arc` clone, no second copy of the string).
Some(entry) => {
if db
.disc_entries
.insert(entry.disc_hash.clone(), entry)
.is_some()
{
stats.disc_duplicate += 1;
}
}
None => stats.disc_rejected += 1,
}
}
}
(db, stats)
}
/// Load a KEYDB.cfg from disk.
///
/// A read failure (missing/unreadable file, non-UTF-8 content) surfaces
/// as an [`std::io::Error`] (the cap-exceeded case as
/// [`std::io::ErrorKind::InvalidData`]). Note that [`Self::parse`] itself
/// is lenient: a syntactically valid but key-less file parses to an empty
/// [`KeyDb`] rather than an error — callers needing a non-empty db must
/// check the parsed contents.
pub fn load(path: &std::path::Path) -> std::io::Result<Self> {
let (db, stats) = Self::load_counted(std::fs::File::open(path)?, path)?;
stats.log();
Ok(db)
}
// `Self::load` from an ALREADY-OPEN file, without the log.
pub(crate) fn load_counted(
f: std::fs::File,
path: &std::path::Path,
) -> std::io::Result<(Self, ParseStats)> {
// Read through a hard `take` cap rather than trusting a stat: a
// `metadata` pre-check reports len 0 for a FIFO/char device (e.g. would
// read `/dev/zero` forever). One byte past cap matches `read_capped_to_string`.
use std::io::Read;
let mut buf = Vec::new();
f.take(MAX_KEYDB_BYTES + 1).read_to_end(&mut buf)?;
if buf.len() as u64 > MAX_KEYDB_BYTES {
return Err(std::io::Error::new(
std::io::ErrorKind::InvalidData,
format!(
"keydb.cfg exceeds {MAX_KEYDB_BYTES} byte cap: {}",
path.display()
),
));
}
let data = String::from_utf8(buf).map_err(|_| {
std::io::Error::new(
std::io::ErrorKind::InvalidData,
format!("keydb.cfg is not valid UTF-8: {}", path.display()),
)
})?;
Ok(Self::parse_counted(&data))
}
/// Look up a disc by its hash. Returns the VUK if found.
pub fn find_vuk(&self, disc_hash: &str) -> Option<[u8; 16]> {
let hash = disc_hash
.trim()
.to_lowercase()
.trim_start_matches("0x")
.to_string();
// Try with 0x prefix and without; every stored key currently carries
// the prefix, so the no-prefix fallback is a defensive match kept for
// the prefix-agnostic lookup contract.
self.disc_entries
.get(format!("0x{hash}").as_str())
.or_else(|| self.disc_entries.get(hash.as_str()))
.and_then(|e| e.vuk)
}
/// Look up a disc by its hash. Returns the full entry.
pub fn find_disc(&self, disc_hash: &str) -> Option<&DiscEntry> {
let hash = disc_hash
.trim()
.to_lowercase()
.trim_start_matches("0x")
.to_string();
// No-prefix fallback below (every stored key carries "0x", see
// find_vuk); kept as a defensive match for the lookup contract.
self.disc_entries
.get(format!("0x{hash}").as_str())
.or_else(|| self.disc_entries.get(hash.as_str()))
}
/// Iterate every disc entry. Used by Path 3 (scan for matching VID).
pub fn iter_disc_entries(&self) -> impl Iterator<Item = &DiscEntry> {
self.disc_entries.values()
}
/// The host certs usable at MKB generation `mkb`.
///
/// A cert annotated `Revoked in MKBv<R>` is unusable once the disc's MKB
/// generation reaches `R` (an AACS MKB revokes a cert from its own
/// generation onward), so it is included only while `gen < R`. When `mkb`
/// is `None` the disc's generation is unknown and cannot be filtered, so
/// every cert is returned; certs with no revocation annotation are always
/// returned.
#[deprecated(note = "use host_certs_ranked; host cert selection takes no disc input")]
pub fn host_certs(&self, mkb: Option<u32>) -> Vec<HostCert> {
self.host_certs
.iter()
.filter(|hc| match (hc.revoked_at_mkb, mkb) {
(None, _) => true,
(Some(_), None) => true,
(Some(revoked), Some(disc_gen)) => disc_gen < revoked,
})
.map(|hc| hc.cert.clone())
.collect()
}
/// Every host cert, best first, with no disc input: certs with no `Revoked in
/// MKBv<N>` note in file order, then revoked ones by latest revocation (a later
/// revocation is the better bet without knowing the disc's generation).
pub fn host_certs_ranked(&self) -> Vec<HostCert> {
let mut ranked: Vec<&KeydbHostCert> = self.host_certs.iter().collect();
// Stable sort: ties (every unannotated cert, equal revocations) keep file order.
ranked.sort_by_key(|hc| {
let r = hc.revoked_at_mkb;
(r.is_some(), std::cmp::Reverse(r.unwrap_or(0)))
});
ranked.into_iter().map(|hc| hc.cert.clone()).collect()
}
/// Standalone keydb accessor: the disc's Volume ID (the keydb `I` token),
/// looked up by the same disc-hash form [`Self::find_disc`] accepts. Pure
/// file lookup; no crypto/derivation.
pub fn get_vid(&self, disc_hash: &str) -> Option<[u8; 16]> {
self.find_disc(disc_hash).and_then(|e| e.vid)
}
/// Standalone keydb accessor: the disc's stored unit (title) keys, cloned.
/// Empty when the disc is absent or carries no unit keys. Pure file lookup.
pub fn get_uk(&self, disc_hash: &str) -> Vec<NumberedUnitKey> {
self.find_disc(disc_hash)
.map(|e| e.unit_keys.clone())
.unwrap_or_default()
}
/// Standalone keydb accessor: `(disc_hash, unit_keys)` for every disc entry
/// that carries at least one unit key. Pure file lookup.
pub fn get_uks(&self) -> Vec<(String, Vec<NumberedUnitKey>)> {
self.disc_entries
.values()
.filter(|e| !e.unit_keys.is_empty())
.map(|e| (e.disc_hash.to_string(), e.unit_keys.clone()))
.collect()
}
/// Serialize back to keydb.cfg text — the INVERSE of [`Self::parse`], so the
/// keydb wire format lives in ONE place (parse + emit together). Emits, in a
/// deterministic order: host certs, device keys, processing keys, then one
/// line per disc entry (sorted by hash). `parse(to_keydb_cfg(kd))` reproduces
/// every field (see `round_trips_through_parse`). Used by the key-import tool
/// to export a complete keydb.cfg (keys + host certs + VIDs).
pub fn to_keydb_cfg(&self) -> String {
fn hx(b: &[u8]) -> String {
use std::fmt::Write;
let mut s = String::with_capacity(b.len() * 2);
for x in b {
let _ = write!(s, "{x:02x}");
}
s
}
let mut out = String::new();
// Host certs (AACS 1.0): | HC | HOST_PRIV_KEY 0x.. | HOST_CERT 0x.. ; Revoked in MKBv<N>
// AACS 2.0 credentials ride a sibling `| HC2 |` line; emit it too so a
// round-trip through `to_keydb_cfg` never silently drops v2 host certs.
for hc in &self.host_certs {
out.push_str("| HC | HOST_PRIV_KEY 0x");
out.push_str(&hx(&hc.cert.private_key));
out.push_str(" | HOST_CERT 0x");
out.push_str(&hx(&hc.cert.certificate));
if let Some(n) = hc.revoked_at_mkb {
out.push_str(" ; Revoked in MKBv");
out.push_str(&n.to_string());
}
out.push('\n');
// AACS 2.0 (HC2): inverse of `parse_host_cert_v2`.
if let (Some(pk2), Some(cert2)) = (
hc.cert.private_key_v2.as_ref(),
hc.cert.certificate_v2.as_ref(),
) {
out.push_str("| HC2 | HOST_PRIV_KEY 0x");
out.push_str(&hx(pk2));
out.push_str(" | HOST_CERT 0x");
out.push_str(&hx(cert2));
out.push('\n');
}
}
// Device keys: | DK | DEVICE_KEY 0x.. | DEVICE_NODE 0x.. | KEY_UV 0x.. | KEY_U_MASK_SHIFT 0x..
for dk in &self.device_keys {
out.push_str("| DK | DEVICE_KEY 0x");
out.push_str(&hx(&dk.key));
out.push_str(&format!(
" | DEVICE_NODE 0x{:04x} | KEY_UV 0x{:08x} | KEY_U_MASK_SHIFT 0x{:02x}\n",
dk.node, dk.uv, dk.u_mask_shift
));
}
// Processing keys: | PK | 0x..
for pk in &self.processing_keys {
out.push_str("| PK | 0x");
out.push_str(&hx(pk));
out.push('\n');
}
// Per-disc entries, sorted by hash for a deterministic, diff-friendly file.
let mut hashes: Vec<&Arc<str>> = self.disc_entries.keys().collect();
hashes.sort();
for h in hashes {
let d = &self.disc_entries[&**h];
// `parse` keeps the `hash_part` verbatim, so the stored `disc_hash`
// already carries its `0x` prefix — emit it as-is (prefixing another
// `0x` would double it on re-parse).
out.push_str(h);
out.push_str(" = ");
// Parse stores the title VERBATIM (parens and all), so emitting it
// bare round-trips through parse. Empty → "Unknown".
if d.title.is_empty() {
out.push_str("Unknown");
} else {
out.push_str(&d.title);
}
if let Some(mk) = d.media_key {
out.push_str(" | M | 0x");
out.push_str(&hx(&mk));
}
if let Some(id) = d.vid {
out.push_str(" | I | 0x");
out.push_str(&hx(&id));
}
if let Some(vuk) = d.vuk {
out.push_str(" | V | 0x");
out.push_str(&hx(&vuk));
}
if !d.unit_keys.is_empty() {
out.push_str(" | U |");
for (n, k) in &d.unit_keys {
out.push_str(&format!(" {}-0x{}", n, hx(k)));
}
// Comment only after U (the one ;-split field) so it can't corrupt
// a preceding hex value on re-parse.
if d.mkb_version.is_some() || d.volume_size.is_some() || d.is_uhd {
out.push_str(" ;");
if let Some(v) = d.mkb_version {
out.push_str(&format!(" MKBv{v}"));
}
if let Some(sz) = d.volume_size {
out.push_str(&format!(" VolumeSize: {sz}"));
}
if d.is_uhd {
out.push_str(" (UHD)");
}
}
}
out.push('\n');
}
out
}
}
// ── Private parsers (re-open the inherent impl) ─────────────────────────────
impl KeyDb {
fn parse_device_key(line: &str) -> Option<DeviceKey> {
// | DK | DEVICE_KEY 0x... | DEVICE_NODE 0x... | KEY_UV 0x... | KEY_U_MASK_SHIFT 0x...
let key_str = line.split("DEVICE_KEY").nth(1)?.split('|').next()?.trim();
let node_str = line.split("DEVICE_NODE").nth(1)?.split('|').next()?.trim();
let uv_str = line.split("KEY_UV").nth(1)?.split('|').next()?.trim();
let shift_str = line
.split("KEY_U_MASK_SHIFT")
.nth(1)?
.split(';')
.next()?
.split('|')
.next()?
.trim();
Some(DeviceKey {
key: parse_hex16(key_str)?,
// Canonical hex parsers (one prefix/case rule for the whole workspace)
// — NOT an ad-hoc `from_str_radix(trim_start_matches("0x"))`, whose
// case-sensitive strip silently dropped an uppercase-`0X` value.
node: libfreemkv::hex::parse_hex_u16(node_str)?,
uv: libfreemkv::hex::parse_hex_u32(uv_str)?,
u_mask_shift: libfreemkv::hex::parse_hex_u8(shift_str)?,
})
}
fn parse_processing_key(line: &str) -> Option<[u8; 16]> {
// | PK | 0x...
let parts: Vec<&str> = line.split('|').collect();
if parts.len() >= 3 {
let key_str = parts[2].split(';').next()?.trim();
return parse_hex16(key_str);
}
None
}
// Parse an orphan DK row: `| DK |` with only `DEVICE_KEY` (no position
// metadata), treated as terminal/unpositioned by the brute walker. `None`
// if a position field is present — those are positioned DKs (`parse_device_key`).
fn parse_orphan_dk(line: &str) -> Option<[u8; 16]> {
if line.contains("DEVICE_NODE")
|| line.contains("KEY_UV")
|| line.contains("KEY_U_MASK_SHIFT")
{
return None;
}
let key_str = line
.split("DEVICE_KEY")
.nth(1)?
.split('|')
.next()?
.split(';')
.next()?
.trim();
parse_hex16(key_str)
}
fn parse_host_cert(line: &str) -> Option<KeydbHostCert> {
// | HC | HOST_PRIV_KEY 0x... | HOST_CERT 0x... ; Revoked in MKBv<N>
let priv_str = line
.split("HOST_PRIV_KEY")
.nth(1)?
.split('|')
.next()?
.trim();
let cert_str = line
.split("HOST_CERT")
.nth(1)?
.split(';')
.next()?
.split('|')
.next()?
.trim();
let certificate = parse_hex(cert_str)?;
// AACS 1.0 host certs are 92 bytes; drop malformed/short rows at
// parse time so the handshake never attempts junk (mirrors the v2
// path, which enforces >= 132).
if certificate.len() < 92 {
return None;
}
Some(KeydbHostCert {
cert: HostCert {
private_key: parse_hex20(priv_str)?,
certificate,
private_key_v2: None,
certificate_v2: None,
},
revoked_at_mkb: parse_revoked_at_mkb(line),
})
}
/// Parse AACS 2.0 host cert: `| HC2 | HOST_PRIV_KEY 0x... | HOST_CERT 0x...`
/// Returns the private key, the cert bytes, and the `Revoked in MKBv<N>`
/// generation (if the line carries that comment).
fn parse_host_cert_v2(line: &str) -> Option<([u8; 32], Vec<u8>, Option<u32>)> {
let priv_str = line
.split("HOST_PRIV_KEY")
.nth(1)?
.split('|')
.next()?
.trim();
let cert_str = line
.split("HOST_CERT")
.nth(1)?
.split(';')
.next()?
.split('|')
.next()?
.trim();
let priv_bytes = parse_hex(priv_str)?;
if priv_bytes.len() != 32 {
return None;
}
let mut pk = [0u8; 32];
pk.copy_from_slice(&priv_bytes);
let cert = parse_hex(cert_str)?;
if cert.len() < 132 {
return None;
}
Some((pk, cert, parse_revoked_at_mkb(line)))
}
fn parse_disc_entry(line: &str) -> Option<DiscEntry> {
// 0x<hash> = <title> | D | <date> | M | 0x<mk> | I | 0x<id> | V | 0x<vuk> | U | <unit_keys> ; <comment>
let (hash_part, rest) = line.split_once(" = ")?;
let disc_hash: Arc<str> = Arc::from(hash_part.trim().to_lowercase());
// The trailing `;` comment (e.g. "; MKBv76/BEE/FindVUK 1.74 -
// VolumeSize: 81309007872 (UHD)") carries metadata the key fields
// don't; capture everything after the FIRST ';' on the line.
let comment = line.split_once(';').map(|(_, c)| c).unwrap_or("");
let mkb_version: Option<u32> = parse_digits_after(comment, "MKBv");
let volume_size: Option<u64> = parse_digits_after(comment, "VolumeSize:");
let is_uhd = comment.contains("(UHD)");
// Title kept VERBATIM (trimmed), a faithful copy for exact round-trip.
let before_fields = rest.split(" | ").next().unwrap_or("");
// A title-only entry (no key fields) carries its `;` comment on the same
// chunk — strip it so the comment doesn't leak into the title.
let title = before_fields
.split(';')
.next()
.unwrap_or("")
.trim()
.to_string();
// Parse fields by tag
let mut media_key = None;
let mut vid = None;
let mut vuk = None;
let mut unit_keys = Vec::new();
let parts: Vec<&str> = rest.split(" | ").collect();
// Field scan starts at index 1: `parts[0]` is ALWAYS the title chunk;
// excluding it stops a title matching a field-tag letter ("M"/"I"/"V"/
// "U"/"D") from being eaten as a tag and shadowing the real field.
let mut i = 1;
while i < parts.len() {
match parts[i].trim() {
// M/I/V strip a trailing `; comment` before hex-parsing (like the
// U arm below): when one of these is the row's LAST field, the
// comment gets glued on and parse_hex16 would otherwise fail.
"M" => {
if i + 1 < parts.len() {
media_key =
parse_hex16(parts[i + 1].split(';').next().unwrap_or("").trim());
i += 1;
}
}
"I" => {
if i + 1 < parts.len() {
vid = parse_hex16(parts[i + 1].split(';').next().unwrap_or("").trim());
i += 1;
}
}
"V" => {
if i + 1 < parts.len() {
vuk = parse_hex16(parts[i + 1].split(';').next().unwrap_or("").trim());
i += 1;
}
}
"U" if i + 1 < parts.len() => {
// Unit keys: "1-0xKEY" or "1-0xKEY ; comment"
let uk_str = parts[i + 1].split(';').next().unwrap_or("").trim();
for uk in uk_str.split(' ') {
let uk = uk.trim();
if let Some((num, key)) = uk.split_once('-')
&& let Ok(n) = num.parse::<u32>()
&& let Some(k) = parse_hex16(key)
{
unit_keys.push((n, k));
}
}
i += 1;
}
_ => {}
}
i += 1;
}
Some(DiscEntry {
disc_hash,
title,
media_key,
vid,
vuk,
unit_keys,
mkb_version,
volume_size,
is_uhd,
})
}
}
#[cfg(test)]
#[path = "keydb_format_tests.rs"]
mod tests;
File diff suppressed because it is too large Load Diff
+1311
View File
File diff suppressed because it is too large Load Diff
+143 -44
View File
@@ -1,75 +1,174 @@
//! Pluggable AACS key sources for libfreemkv.
//!
//! libfreemkv performs no key lookup — it is handed a [`Key`] and derives down
//! the AACS chain to decrypt. This crate provides the published [`KeySource`]
//! implementations that do the lookup:
//!
//! - [`KeydbSource`] — a local `keydb.cfg` (source #1).
//! - [`OnlineSource`] — a remote key service (source #2).
//! - [`MapfileSource`] — the persisted unit key from a rip mapfile (source #3).
//!
//! Applications (autorip, the `freemkv` CLI) choose and order the sources from
//! their own config — the local-vs-online policy is just which impls they plug
//! in — then resolve and hand the resulting key to `Disc::decrypt_with`.
//!
//! Sources are dumb and stateful: each hands its candidate keys out one at a
//! time via [`KeySource::next_key`], in its own best order, and reports
//! exhaustion. Compose several with [`MultiSource`] in the caller's chosen
//! order. Resolving those candidates against a disc, and reading the encrypted
//! content-sample units a key server validates on, is decryption *mechanism* —
//! it lives in the library (`libfreemkv::resolve_and_apply`,
//! `libfreemkv::read_encrypted_units`), not here. A source only ever looks a key
//! up and hands it back; what's done with the key is not its concern.
//! libfreemkv owns the AACS crypto; this crate provides [`KeySource`] impls that look a disc up
//! and drive the boil-down primitives to terminal Unit Keys: [`KeydbSource`] (local
//! `keydb.cfg`) and [`OnlineSource`] (remote key service). Applications choose and order
//! sources, then resolve and hand the key to `Disc::decrypt_with`; compose several with
//! [`MultiSource`].
mod keydb;
mod mapfile;
/// The `keydb.cfg` parser (`KeyDb`, `DiscEntry`, …). Public: parsing the keydb
/// is not secret — freemkv uses it, and so do tools that build a disc registry
/// from it (e.g. a per-disc Volume-ID index).
pub mod keydb_format;
mod online;
mod paths;
pub use keydb::KeydbSource;
pub use mapfile::MapfileSource;
pub use online::{OnlineSource, validate_keyserver_url};
pub use keydb::{KeydbSource, UpdateResult};
pub use keydb_format::{DiscEntry, KeyDb};
#[cfg(feature = "test-hooks")]
pub use online::set_last_decode_reachability;
pub use online::{
DecodeReachability, KeyserverUrlFault, KeyserverUrlRejection, MIN_SAMPLE_UNITS, OnlineSource,
check_keyserver_url, check_keyserver_url_static, take_last_decode_reachability,
validate_keyserver_url,
};
pub use paths::{default_keydb_path, existing_keydb_path, keydb_search_paths};
// Re-exported for downstream convenience so apps need only depend on this crate
// for the source-side types.
pub use libfreemkv::{DiscInputs, Key, KeySource};
pub use libfreemkv::aacs::types::UnitKey;
pub use libfreemkv::keysource::ResolveCtx;
pub use libfreemkv::{DiscInputs, KeySource};
/// An ordered composition of key sources, driven as one. `next_key` exhausts
/// the first source (one candidate per call), then the next, … then `None`.
/// **The caller supplies the list AND the order** — local-first `[Keydb,
/// Online]`, online-first `[Online, Keydb]`, resume `[Mapfile, Keydb]`, etc. —
/// so the "which sources, in what order" policy lives entirely with the
/// application, not the library. `MultiSource` is itself a [`KeySource`], so it
/// nests and composes.
// VUK -> the disc's terminal Unit Keys (positional index), one AES-ECB-decrypt
// per encrypted title key, via `aacs::derive::decrypt_unit_key`. Replaces the
// removed libfreemkv `aacs::boil::uk_from_vuk` wrapper.
pub(crate) fn uks_from_vuk(vuk: &[u8; 16], enc_title_keys: &[[u8; 16]]) -> Vec<UnitKey> {
enc_title_keys
.iter()
.enumerate()
.map(|(i, e)| UnitKey::new(i as u32, libfreemkv::aacs::derive::decrypt_unit_key(vuk, e)))
.collect()
}
/// An ordered composition of key sources, driven as one.
///
/// [`MultiSource::get_unit_keys`] tries each inner source in order and returns the first
/// non-empty Unit Key set (and [`MultiSource::get_fmts_indexes`] does the same for the forensic
/// set). The caller supplies the list AND the order — local-first `[Keydb, Online]`,
/// online-first `[Online, Keydb]`, etc. `MultiSource` is itself a [`KeySource`], so it nests
/// and composes.
pub struct MultiSource {
sources: Vec<Box<dyn KeySource>>,
idx: usize,
}
impl MultiSource {
/// Compose the given sources, tried in the order supplied.
pub fn new(sources: Vec<Box<dyn KeySource>>) -> Self {
Self { sources, idx: 0 }
Self { sources }
}
}
// Drive `sources` in order, returning the first non-empty result and preserving Ok/Err when
// nothing resolves. `get` selects the trait method so the base and forensic paths share one
// implementation.
fn first_non_empty(
sources: &[Box<dyn KeySource>],
get: impl Fn(&dyn KeySource, &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error>,
ctx: &dyn ResolveCtx,
) -> Result<Vec<UnitKey>, libfreemkv::Error> {
let mut first_failure: Option<libfreemkv::Error> = None;
for s in sources {
match get(s.as_ref(), ctx) {
Ok(uks) if !uks.is_empty() => return Ok(uks),
Ok(_) => {}
Err(e) => {
if first_failure.is_none() {
first_failure = Some(e);
}
}
}
}
match first_failure {
// At least one source could not answer, and nothing else had a key: the
// composition does NOT know that this disc has no key.
Some(e) => Err(e),
// Every source answered; none holds a key. The genuine miss.
None => Ok(Vec::new()),
}
}
impl KeySource for MultiSource {
fn next_key(&mut self, inputs: &DiscInputs) -> Option<Key> {
while self.idx < self.sources.len() {
if let Some(key) = self.sources[self.idx].next_key(inputs) {
return Some(key);
// First inner source to return a non-empty base Unit Key set wins. All
// exhausted -> `Ok(empty)` if all answered, else the first failure (see
// [`MultiSource`]).
fn get_unit_keys(&self, ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
first_non_empty(&self.sources, |s, c| s.get_unit_keys(c), ctx)
}
// De-conflated counterpart: first inner source with keys wins; else the
// richest miss survives composition (a MATCHED inner source beats a plain
// miss). Same Err/Ok failure contract as `get_unit_keys`.
fn resolve_unit_keys(
&self,
ctx: &dyn ResolveCtx,
) -> Result<libfreemkv::keysource::UnitKeyResolution, libfreemkv::Error> {
let mut first_failure: Option<libfreemkv::Error> = None;
let mut matched: Option<libfreemkv::keysource::UnitKeyResolution> = None;
for s in &self.sources {
match s.resolve_unit_keys(ctx) {
Ok(r) if !r.keys.is_empty() => return Ok(r),
Ok(r) => {
if r.matched && matched.is_none() {
matched = Some(r);
}
}
Err(e) => {
if first_failure.is_none() {
first_failure = Some(e);
}
}
}
self.idx += 1; // this source is spent — advance to the next
}
None
if let Some(r) = matched {
return Ok(r);
}
match first_failure {
Some(e) => Err(e),
None => Ok(libfreemkv::keysource::UnitKeyResolution::default()),
}
}
fn needs_samples(&self) -> bool {
self.sources.iter().any(|s| s.needs_samples())
// Forensic-index counterpart to `get_unit_keys`, same failure-preserving
// rule. A source with no forensic material (keydb, via the trait default)
// contributes empty and is skipped; today the online source answers.
fn get_fmts_indexes(&self, ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
first_non_empty(&self.sources, |s, c| s.get_fmts_indexes(c), ctx)
}
fn errored(&self) -> bool {
self.sources.iter().any(|s| s.errored())
/// UNION every inner source's host certs, in source order (`mkb` is passed
/// through unchanged). Without this a composed source would hide an inner
/// source's cert from the OEM cert-auth route — the gap this fixes.
fn host_certs(&self, mkb: Option<u32>) -> Vec<libfreemkv::aacs::types::HostCert> {
self.sources
.iter()
.flat_map(|s| s.host_certs(mkb))
.collect()
}
fn label(&self) -> &'static str {
"multi"
}
// KU3-6: depends on samples if ANY inner source does, so a nested
// sample-dependent source keeps getting the per-piece ask.
fn answer_depends_on_samples(&self) -> bool {
self.sources.iter().any(|s| s.answer_depends_on_samples())
}
// Same `any(inner)` rule (KU-K1 review): `resolve` retries the composition
// whenever any inner source's last failure was transport-class.
fn last_failure_was_transport(&self) -> bool {
self.sources.iter().any(|s| s.last_failure_was_transport())
}
// KU J23: any inner source that consumes the VID makes the composition one.
fn uses_vid(&self) -> bool {
self.sources.iter().any(|s| s.uses_vid())
}
}
#[cfg(test)]
#[path = "lib_tests.rs"]
mod tests;
+172
View File
@@ -0,0 +1,172 @@
use super::*;
use libfreemkv::aacs::types::HostCert;
// A minimal KeySource that only supplies host certs — the sole MultiSource
// behaviour under test. It never resolves a unit key.
struct CertOnly {
certs: Vec<HostCert>,
}
impl KeySource for CertOnly {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
Ok(Vec::new())
}
fn host_certs(&self, _mkb: Option<u32>) -> Vec<HostCert> {
self.certs.clone()
}
}
// A host cert tagged with a distinct marker byte so a returned set can be
// identified exactly. Synthetic bytes only; no real key material.
fn cert(marker: u8) -> HostCert {
HostCert {
private_key: [marker; 20],
certificate: vec![marker; 92],
private_key_v2: None,
certificate_v2: None,
}
}
fn boxed(certs: Vec<HostCert>) -> Box<dyn KeySource> {
Box::new(CertOnly { certs }) as Box<dyn KeySource>
}
/// `MultiSource::host_certs` must UNION every inner source's certs. Without
/// this a composed source hides an inner source's cert from the OEM
/// cert-auth route — the gap the union closes.
#[test]
fn multi_source_host_certs_unions_every_inner_source() {
let multi = MultiSource::new(vec![
boxed(vec![cert(0x01)]),
boxed(vec![cert(0x02), cert(0x03)]),
]);
let mut markers: Vec<u8> = multi
.host_certs(Some(50))
.iter()
.map(|c| c.certificate[0])
.collect();
markers.sort_unstable();
assert_eq!(
markers,
vec![0x01, 0x02, 0x03],
"certs from every inner source must be unioned"
);
}
/// With no inner source holding a cert, the union is empty (not an error).
#[test]
fn multi_source_host_certs_empty_when_no_source_has_one() {
let multi = MultiSource::new(vec![boxed(vec![]), boxed(vec![])]);
assert!(multi.host_certs(None).is_empty());
}
// A source whose `answer_depends_on_samples` is fixed to the given value —
// the only behaviour KSK2 exercises.
struct FixedDependency(bool);
impl KeySource for FixedDependency {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
Ok(Vec::new())
}
fn answer_depends_on_samples(&self) -> bool {
self.0
}
}
// KSK2: `MultiSource::answer_depends_on_samples` is `any(inner)`, so nesting
// a sample-dependent online source under a sample-independent keydb still
// gets the per-piece ask (`resolve` must not treat the pair as keydb-only).
#[test]
fn multi_source_answer_depends_on_samples_is_any_inner() {
let all_independent = MultiSource::new(vec![
Box::new(FixedDependency(false)) as Box<dyn KeySource>,
Box::new(FixedDependency(false)) as Box<dyn KeySource>,
]);
assert!(!all_independent.answer_depends_on_samples());
let keydb_then_online = MultiSource::new(vec![
Box::new(FixedDependency(false)) as Box<dyn KeySource>,
Box::new(FixedDependency(true)) as Box<dyn KeySource>,
]);
assert!(keydb_then_online.answer_depends_on_samples());
}
// A source whose `uses_vid` is fixed to the given value.
struct FixedVid(bool);
impl KeySource for FixedVid {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
Ok(Vec::new())
}
fn uses_vid(&self) -> bool {
self.0
}
}
// KU J23: like `answer_depends_on_samples`, a composed source uses the VID if any
// inner one does, so a nested online source still makes a Missing piece VID-derivable.
#[test]
fn multi_source_uses_vid_is_any_inner() {
let neither = MultiSource::new(vec![
Box::new(FixedVid(false)) as Box<dyn KeySource>,
Box::new(FixedVid(false)) as Box<dyn KeySource>,
]);
assert!(!neither.uses_vid());
let keydb_then_online = MultiSource::new(vec![
Box::new(FixedVid(false)) as Box<dyn KeySource>,
Box::new(FixedVid(true)) as Box<dyn KeySource>,
]);
assert!(keydb_then_online.uses_vid());
}
// A source whose `last_failure_was_transport` is fixed to the given value.
struct FixedTransport(bool);
impl KeySource for FixedTransport {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
Ok(Vec::new())
}
fn last_failure_was_transport(&self) -> bool {
self.0
}
}
// For consistency with `answer_depends_on_samples` (KU3-6): a composed
// source's last-failure verdict is `any(inner)` too, so `resolve` retries
// the composition whenever ANY inner source's last failure was transport.
#[test]
fn multi_source_last_failure_was_transport_is_any_inner() {
let none_transport = MultiSource::new(vec![
Box::new(FixedTransport(false)) as Box<dyn KeySource>,
Box::new(FixedTransport(false)) as Box<dyn KeySource>,
]);
assert!(!none_transport.last_failure_was_transport());
let one_transport = MultiSource::new(vec![
Box::new(FixedTransport(false)) as Box<dyn KeySource>,
Box::new(FixedTransport(true)) as Box<dyn KeySource>,
]);
assert!(one_transport.last_failure_was_transport());
}
/// The `mkb` generation must reach each inner source so its OWN revocation
/// filter can act; a source that drops everything at a given generation is
/// honoured by the union.
#[test]
fn multi_source_host_certs_passes_mkb_through_to_inner_sources() {
struct GenAware;
impl KeySource for GenAware {
fn get_unit_keys(&self, _c: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
Ok(Vec::new())
}
fn host_certs(&self, mkb: Option<u32>) -> Vec<HostCert> {
match mkb {
// Modelled on a cert revoked in MKBv72.
Some(g) if g >= 72 => Vec::new(),
_ => vec![cert(0xaa)],
}
}
}
let multi = MultiSource::new(vec![Box::new(GenAware) as Box<dyn KeySource>]);
assert_eq!(multi.host_certs(Some(50)).len(), 1, "usable below gen 72");
assert!(
multi.host_certs(Some(72)).is_empty(),
"the gen must reach the inner source's filter"
);
}
-46
View File
@@ -1,46 +0,0 @@
//! Mapfile cache source (source #3).
//!
//! A rip's ddrescue-style mapfile persists the resolved unit keys in its
//! `# freemkv-uk:` header (written at sweep time when the disc was keyed). On
//! resume / deferred mux, that mapfile is the fastest source — the keys are
//! already resolved, no keydb parse and no network round-trip. This source
//! reads them back as a terminal [`Key::Unit`] candidate.
//!
//! It is keyed by the mapfile path (the disc identity is implicit in which
//! mapfile belongs to which rip), so it ignores [`DiscInputs`].
use std::path::PathBuf;
use libfreemkv::disc::mapfile::Mapfile;
use libfreemkv::{DiscInputs, Key, KeySource};
/// A [`KeySource`] backed by a rip mapfile's persisted unit keys.
pub struct MapfileSource {
path: PathBuf,
/// The mapfile holds exactly one (terminal) UK set, so it is read once —
/// this flips true after the first `next_key`.
asked: bool,
}
impl MapfileSource {
/// A mapfile source reading the given `*.mapfile` path.
pub fn new(path: impl Into<PathBuf>) -> Self {
Self {
path: path.into(),
asked: false,
}
}
}
impl KeySource for MapfileSource {
fn next_key(&mut self, _inputs: &DiscInputs) -> Option<Key> {
if self.asked {
return None;
}
self.asked = true;
// A missing/unreadable/keyless mapfile simply offers nothing.
let map = Mapfile::load(&self.path).ok()?;
let uks = map.unit_keys();
(!uks.is_empty()).then(|| Key::Unit(uks.to_vec()))
}
}
+1030 -394
View File
File diff suppressed because it is too large Load Diff
+1936
View File
File diff suppressed because it is too large Load Diff
+23 -197
View File
@@ -1,78 +1,32 @@
//! Where the `keydb.cfg` lives, per OS.
//! Where the `keydb.cfg` lives: local to the executable, local ONLY.
//!
//! Key-path policy belongs with the key sources (this crate), not the library:
//! libfreemkv is handed a path and reads it. The CLI/app asks here for the
//! ordered list of locations to *search* (first existing wins) and for the
//! single *default* location to write to (e.g. `update-keys`/save).
//! Key-path policy belongs here (key sources), not in libfreemkv, which
//! just reads a handed-in path. This module gives the *search* list (first
//! existing wins) and the single *default* write location.
//!
//! Resolution order:
//!
//! - **Windows**: `%APPDATA%\freemkv\keydb.cfg` FIRST (the idiomatic per-user
//! roaming config dir), then the legacy `%USERPROFILE%\.config\freemkv\keydb.cfg`
//! for back-compat with installs that predate this fix.
//! - **Linux / macOS**: `$XDG_CONFIG_HOME/freemkv/keydb.cfg` (if `XDG_CONFIG_HOME`
//! is set), then `$HOME/.config/freemkv/keydb.cfg` — the long-standing default,
//! unchanged so existing users keep working.
//!
//! Pure `std::env` — `%APPDATA%`, `%USERPROFILE%`, `$HOME`, `$XDG_CONFIG_HOME`
//! are all environment variables, so no `dirs`-style crate is pulled in.
//! freemkv is portable: `keydb.cfg` lives next to the exe — `<dir of current exe>/keydb.cfg` —
//! with no OS config-dir fallback. `--keydb PATH` bypasses this module.
use std::path::PathBuf;
/// The keydb filename plus its `freemkv` subdir, joined onto a base dir.
fn keydb_under(base: PathBuf) -> PathBuf {
base.join("freemkv").join("keydb.cfg")
}
/// The ordered list of `keydb.cfg` locations to search, most-idiomatic first.
/// The single `keydb.cfg` location to search: next to the current executable.
///
/// Returns exactly one path — `<dir of current exe>/keydb.cfg` — on success.
/// Returns an empty list if the executable's own directory can't be determined
/// (`std::env::current_exe()` fails or has no parent); there is deliberately no
/// OS config-dir fallback (portable / local-only).
///
/// The caller picks the first path that exists on disk (see
/// [`existing_keydb_path`]); for writing a freshly-downloaded keydb, use
/// [`default_keydb_path`] (the first entry — the canonical location).
///
/// Always returns at least one entry on a normally-configured system; returns
/// an empty list only if none of the relevant env vars are set.
/// [`default_keydb_path`].
pub fn keydb_search_paths() -> Vec<PathBuf> {
let mut paths = Vec::new();
if cfg!(windows) {
// Idiomatic Windows location first.
if let Ok(appdata) = std::env::var("APPDATA") {
if !appdata.is_empty() {
paths.push(keydb_under(PathBuf::from(appdata)));
}
}
// Legacy XDG-style dotfolder under the user profile, for back-compat.
if let Ok(profile) = std::env::var("USERPROFILE") {
if !profile.is_empty() {
paths.push(
PathBuf::from(profile)
.join(".config")
.join("freemkv")
.join("keydb.cfg"),
);
}
}
} else {
// Honour XDG_CONFIG_HOME if the user set it, then the historical default.
if let Ok(xdg) = std::env::var("XDG_CONFIG_HOME") {
if !xdg.is_empty() {
paths.push(keydb_under(PathBuf::from(xdg)));
}
}
if let Ok(home) = std::env::var("HOME") {
if !home.is_empty() {
paths.push(
PathBuf::from(home)
.join(".config")
.join("freemkv")
.join("keydb.cfg"),
);
}
}
match std::env::current_exe()
.ok()
.and_then(|exe| exe.parent().map(|dir| dir.join("keydb.cfg")))
{
Some(path) => vec![path],
None => Vec::new(),
}
paths
}
/// The first search path that exists on disk, if any.
@@ -85,140 +39,12 @@ pub fn existing_keydb_path() -> Option<PathBuf> {
/// The canonical default location to WRITE the keydb to (e.g. after a download).
///
/// This is the first (most idiomatic) entry of [`keydb_search_paths`]:
/// `%APPDATA%\freemkv\keydb.cfg` on Windows, `~/.config/freemkv/keydb.cfg`
/// elsewhere. Returns `None` only when the relevant env vars are unset.
/// This is the sole entry of [`keydb_search_paths`]: `<dir of current exe>/keydb.cfg`.
/// Returns `None` only when the executable's own directory can't be determined.
pub fn default_keydb_path() -> Option<PathBuf> {
keydb_search_paths().into_iter().next()
}
#[cfg(test)]
mod tests {
use super::*;
/// Serialize env-mutating tests: they share the process environment.
fn lock() -> std::sync::MutexGuard<'static, ()> {
static M: std::sync::Mutex<()> = std::sync::Mutex::new(());
M.lock().unwrap_or_else(|e| e.into_inner())
}
/// Build the search list under an explicit env, restoring the prior env
/// afterwards. Avoids depending on the host's real HOME/APPDATA.
fn with_env(vars: &[(&str, Option<&str>)], f: impl FnOnce()) {
let _g = lock();
let keys = ["APPDATA", "USERPROFILE", "HOME", "XDG_CONFIG_HOME"];
let saved: Vec<(&str, Option<String>)> =
keys.iter().map(|k| (*k, std::env::var(k).ok())).collect();
// Clear all, then apply the requested overrides.
for k in keys {
unsafe { std::env::remove_var(k) };
}
for (k, v) in vars {
match v {
Some(val) => unsafe { std::env::set_var(k, val) },
None => unsafe { std::env::remove_var(k) },
}
}
f();
// Restore.
for (k, v) in saved {
match v {
Some(val) => unsafe { std::env::set_var(k, val) },
None => unsafe { std::env::remove_var(k) },
}
}
}
#[cfg(windows)]
#[test]
fn windows_prefers_appdata_then_legacy_userprofile() {
with_env(
&[
("APPDATA", Some(r"C:\Users\matt\AppData\Roaming")),
("USERPROFILE", Some(r"C:\Users\matt")),
],
|| {
let paths = keydb_search_paths();
assert_eq!(paths.len(), 2, "APPDATA + legacy USERPROFILE");
assert_eq!(
paths[0],
PathBuf::from(r"C:\Users\matt\AppData\Roaming")
.join("freemkv")
.join("keydb.cfg"),
"APPDATA location must be searched first on Windows"
);
assert_eq!(
paths[1],
PathBuf::from(r"C:\Users\matt")
.join(".config")
.join("freemkv")
.join("keydb.cfg"),
"legacy .config dotfolder is the back-compat fallback"
);
assert_eq!(default_keydb_path(), Some(paths[0].clone()));
},
);
}
#[cfg(windows)]
#[test]
fn windows_falls_back_to_legacy_when_appdata_unset() {
with_env(
&[("APPDATA", None), ("USERPROFILE", Some(r"C:\Users\matt"))],
|| {
let paths = keydb_search_paths();
assert_eq!(paths.len(), 1, "only the legacy USERPROFILE path");
assert_eq!(
paths[0],
PathBuf::from(r"C:\Users\matt")
.join(".config")
.join("freemkv")
.join("keydb.cfg")
);
},
);
}
#[cfg(not(windows))]
#[test]
fn unix_default_is_home_dotconfig() {
with_env(
&[("HOME", Some("/u/me")), ("XDG_CONFIG_HOME", None)],
|| {
let paths = keydb_search_paths();
assert_eq!(paths.len(), 1, "just the $HOME/.config default");
assert_eq!(
paths[0],
PathBuf::from("/u/me")
.join(".config")
.join("freemkv")
.join("keydb.cfg"),
"Linux/macOS default must remain ~/.config/freemkv/keydb.cfg"
);
assert_eq!(default_keydb_path(), Some(paths[0].clone()));
},
);
}
#[cfg(not(windows))]
#[test]
fn unix_honours_xdg_config_home_first() {
with_env(
&[
("XDG_CONFIG_HOME", Some("/u/me/.cfg")),
("HOME", Some("/u/me")),
],
|| {
let paths = keydb_search_paths();
assert_eq!(paths.len(), 2, "XDG dir + $HOME/.config fallback");
assert_eq!(
paths[0],
PathBuf::from("/u/me/.cfg")
.join("freemkv")
.join("keydb.cfg"),
"XDG_CONFIG_HOME, when set, is searched before ~/.config"
);
},
);
}
}
#[path = "paths_tests.rs"]
mod tests;
+40
View File
@@ -0,0 +1,40 @@
use super::*;
/// The expected exe-local keydb path, computed the same way the code does.
/// Under `cargo test`, `current_exe()` is the test binary under `target/…`.
fn expected_local() -> Option<PathBuf> {
std::env::current_exe()
.ok()
.and_then(|exe| exe.parent().map(|dir| dir.join("keydb.cfg")))
}
#[test]
fn search_paths_is_exactly_exe_local_keydb() {
let paths = keydb_search_paths();
match expected_local() {
Some(expected) => {
assert_eq!(
paths,
vec![expected],
"search list must be exactly [<exe dir>/keydb.cfg]"
);
}
None => {
// No exe dir available → empty, no OS fallback (local only).
assert!(
paths.is_empty(),
"no exe dir means an empty search list, never an OS fallback"
);
}
}
}
#[test]
fn default_path_matches_search_head() {
// The write default is the single search entry, or None if unavailable.
assert_eq!(default_keydb_path(), expected_local());
assert_eq!(
default_keydb_path(),
keydb_search_paths().into_iter().next()
);
}
+510
View File
@@ -0,0 +1,510 @@
//! Fixture-based integration tests for the published key sources.
//!
//! These exercise the *public* surface of `freemkv-keysources` end-to-end —
//! real files on disk, the real `KeyDb` parser from libfreemkv, and the
//! `KeySource` trait (`get_unit_keys` over a `ResolveCtx`) the applications
//! drive. Covered: `KeydbSource` terminal unit-key lookup by disc hash
//! (plus exe-local path helpers and MKB-aware host-cert serving);
//! `OnlineSource` SSRF/scheme validation and the unconfigured no-op; and
//! `MultiSource` ordering/precedence, host-cert UNION, and nesting.
use std::io::Write;
use std::path::{Path, PathBuf};
use freemkv_keysources::{
DiscInputs, KeySource, KeydbSource, MultiSource, OnlineSource, UnitKey, default_keydb_path,
existing_keydb_path, keydb_search_paths, validate_keyserver_url,
};
use libfreemkv::keysource::{DiscInputsCtx, ResolveCtx};
// ── fixture helpers ─────────────────────────────────────────────────────────
/// A unique scratch dir for one test, removed on `Drop` so fixtures never leak.
struct Scratch(PathBuf);
impl Scratch {
fn new(tag: &str) -> Self {
let dir = std::env::temp_dir().join(format!(
"fmk_ks_{}_{}_{:?}",
tag,
std::process::id(),
std::thread::current().id()
));
std::fs::create_dir_all(&dir).unwrap();
Scratch(dir)
}
fn write(&self, name: &str, body: &str) -> PathBuf {
let p = self.0.join(name);
let mut f = std::fs::File::create(&p).unwrap();
f.write_all(body.as_bytes()).unwrap();
p
}
fn path(&self, name: &str) -> PathBuf {
self.0.join(name)
}
}
impl Drop for Scratch {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
/// Bare `DiscInputs` keyed only by disc hash — the lookup key a keydb uses.
fn inputs(hash: &str) -> DiscInputs {
DiscInputs {
disc_hash: hash.into(),
volume_id: [0u8; 16],
version: libfreemkv::aacs::mkb::AACS_MAJOR_UHD,
mkb: Vec::new(),
unit_key_ro: Vec::new(),
samples: Vec::new(),
volume_label: None,
}
}
/// Resolve a source through the public trait over a `DiscInputsCtx`.
fn resolve(src: &dyn KeySource, inp: &DiscInputs) -> Vec<UnitKey> {
let ctx = DiscInputsCtx::new(inp);
src.get_unit_keys(&ctx)
.expect("get_unit_keys must not error for these fixtures")
}
// ── KeydbSource: real-file lookup by disc hash ──────────────────────────────
const DISC_HASH: &str = "0xaabbccddaabbccddaabbccddaabbccddaabbccdd";
/// A `keydb.cfg` with one per-disc entry carrying a **terminal** unit key (the
/// `U` token) plus a universal DK pool. The terminal UK path needs no on-disc
/// crypto inputs, so it round-trips through a bare `DiscInputs`.
fn keydb_with_unit_key() -> String {
format!(
"; fixture keydb\n\
| DK | DEVICE_KEY 0x{dk} | DEVICE_NODE 0x0001 | KEY_UV 0x00000002 | KEY_U_MASK_SHIFT 0x00\n\
{hash} = FIXTURE_DISC | U | 1-0x{uk}\n",
dk = "22".repeat(16),
hash = DISC_HASH,
uk = "11".repeat(16),
)
}
#[test]
fn keydb_source_resolves_terminal_unit_key_by_hash() {
let s = Scratch::new("keydb_hit");
let path = s.write("keydb.cfg", &keydb_with_unit_key());
let src = KeydbSource::new(&path);
let uks = resolve(&src, &inputs(DISC_HASH));
assert_eq!(uks.len(), 1, "the disc's terminal unit key is resolved");
assert_eq!(
uks[0].key, [0x11u8; 16],
"key bytes come straight from the keydb"
);
// CPS number 1 in the keydb → positional idx 0 (resolver re-adds the +1).
assert_eq!(uks[0].idx, 0, "stored CPS num 1 maps to positional idx 0");
}
#[test]
fn keydb_source_hash_miss_yields_nothing() {
let s = Scratch::new("keydb_miss");
let path = s.write("keydb.cfg", &keydb_with_unit_key());
let src = KeydbSource::new(&path);
let uks = resolve(&src, &inputs("0xdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef"));
assert!(
uks.is_empty(),
"a hash miss resolves no keys from the keydb"
);
}
#[test]
fn keydb_source_missing_file_is_silent_ok_empty() {
// A missing keydb is not an error — it simply offers no keys, so a later
// source in the chain can still supply them.
let src = KeydbSource::new("/nonexistent/path/keydb.cfg");
let inp = inputs(DISC_HASH);
let ctx = DiscInputsCtx::new(&inp);
assert!(
src.get_unit_keys(&ctx)
.expect("missing keydb is Ok, not Err")
.is_empty()
);
}
#[test]
fn keydb_source_label_is_keydb() {
assert_eq!(KeydbSource::new("/nonexistent/keydb.cfg").label(), "keydb");
}
// ── KeydbSource: host-cert serving from a file ──────────────────────────────
#[test]
fn keydb_source_serves_host_cert_from_hc_row() {
let s = Scratch::new("keydb_hc");
// `| HC |` row with all-zero placeholder material (never a real key).
let line = format!(
"| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{}\n",
"00".repeat(20),
"00".repeat(92)
);
let path = s.write("keydb.cfg", &line);
let src = KeydbSource::new(&path);
// Inherent (scan-options) form.
assert_eq!(
src.host_certs().len(),
1,
"inherent host_certs sees the HC row"
);
// Trait form: `mkb` is ignored; the ranked list is returned.
let via_trait = KeySource::host_certs(&src, Some(70));
assert_eq!(via_trait.len(), 1, "trait host_certs sees the HC row");
assert_eq!(via_trait[0].certificate.len(), 92);
}
#[test]
fn keydb_source_host_certs_empty_when_file_missing() {
let src = KeydbSource::new("/nonexistent/keydb.cfg");
assert!(src.host_certs().is_empty());
assert!(KeySource::host_certs(&src, None).is_empty());
}
// ── path policy: exe-local, local-only ──────────────────────────────────────
fn expected_local() -> Option<PathBuf> {
std::env::current_exe()
.ok()
.and_then(|exe| exe.parent().map(|dir| dir.join("keydb.cfg")))
}
#[test]
fn search_paths_is_exactly_exe_local() {
let paths = keydb_search_paths();
match expected_local() {
Some(expected) => assert_eq!(
paths,
vec![expected],
"search list must be exactly [<exe dir>/keydb.cfg], no OS fallback"
),
None => assert!(paths.is_empty(), "no exe dir → empty, never an OS fallback"),
}
}
#[test]
fn default_path_matches_search_head() {
assert_eq!(default_keydb_path(), expected_local());
assert_eq!(
default_keydb_path(),
keydb_search_paths().into_iter().next()
);
}
#[test]
fn existing_keydb_path_reflects_disk_state() {
let head_exists = keydb_search_paths()
.first()
.map(|p| p.exists())
.unwrap_or(false);
assert_eq!(existing_keydb_path().is_some(), head_exists);
}
// ── OnlineSource: validation + unconfigured no-op (no network in CI) ─────────
#[test]
fn online_source_unconfigured_is_silent_no_op() {
// Empty base URL → a clean "no service" empty, no network.
let src = OnlineSource::new("", "");
assert!(resolve(&src, &inputs(DISC_HASH)).is_empty());
}
#[test]
fn online_source_metadata() {
let src = OnlineSource::new("https://example.test/keys", "tok");
assert_eq!(src.label(), "online");
// No host-cert serving today — a no-op empty, no network touched.
assert!(KeySource::host_certs(&src, None).is_empty());
}
#[test]
fn validate_keyserver_url_gates_scheme_and_allows_lan() {
assert!(validate_keyserver_url("https://8.8.8.8/keys").is_ok());
// Home app: LAN/loopback/link-local hosts pass; cleartext http and invalid addresses do not.
assert!(validate_keyserver_url("https://127.0.0.1/keys").is_ok());
assert!(validate_keyserver_url("https://169.254.1.5/keys").is_ok());
assert!(validate_keyserver_url("https://[::1]:9000/keys").is_ok());
assert!(validate_keyserver_url("http://127.0.0.1/keys").is_err());
assert!(validate_keyserver_url("https://0.0.0.0/keys").is_err());
assert!(validate_keyserver_url("ftp://example.com/keys").is_err());
assert!(validate_keyserver_url("").is_err());
}
// ── MultiSource: ordering, precedence, aggregation, nesting ─────────────────
/// A scripted source for composition tests: returns a fixed Unit Key set.
struct ScriptedSource {
keys: Vec<UnitKey>,
label: &'static str,
}
impl ScriptedSource {
fn new(label: &'static str, keys: Vec<UnitKey>) -> Self {
Self { keys, label }
}
}
impl KeySource for ScriptedSource {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
Ok(self.keys.clone())
}
fn label(&self) -> &'static str {
self.label
}
}
fn uk(b: u8) -> UnitKey {
UnitKey::new(0, [b; 16])
}
/// `MultiSource` itself reports a fixed label, distinct from any inner
/// source's own label — this is what a rip's source-attribution log shows
/// when the composition, not one of its members, answered.
#[test]
fn multi_source_label_is_multi() {
let multi = MultiSource::new(vec![Box::new(ScriptedSource::new("A", Vec::new()))]);
assert_eq!(multi.label(), "multi");
}
#[test]
fn multi_source_first_non_empty_wins_in_caller_order() {
// Caller supplies [A, B]; A is non-empty so A's keys win.
let a = ScriptedSource::new("A", vec![uk(0xa1)]);
let b = ScriptedSource::new("B", vec![uk(0xb1)]);
let multi = MultiSource::new(vec![Box::new(a), Box::new(b)]);
let got = resolve(&multi, &inputs("x"));
assert_eq!(got, vec![uk(0xa1)], "A (first, non-empty) wins");
}
#[test]
fn multi_source_order_is_reversible() {
// The SAME two sources in the opposite order yield the opposite precedence.
let a = ScriptedSource::new("A", vec![uk(0xa1)]);
let b = ScriptedSource::new("B", vec![uk(0xb1)]);
let multi = MultiSource::new(vec![Box::new(b), Box::new(a)]);
let got = resolve(&multi, &inputs("x"));
assert_eq!(got, vec![uk(0xb1)], "B-first ordering wins");
}
#[test]
fn multi_source_skips_empty_sources() {
// An empty source in front is transparently skipped to the next.
let empty = ScriptedSource::new("empty", vec![]);
let real = ScriptedSource::new("real", vec![uk(0xc1)]);
let multi = MultiSource::new(vec![Box::new(empty), Box::new(real)]);
assert_eq!(resolve(&multi, &inputs("x")), vec![uk(0xc1)]);
}
#[test]
fn multi_source_unions_host_certs() {
// A real keydb (1 HC row) composed with a cert-less scripted source: the
// composed `host_certs` must UNION — i.e. surface the keydb's cert (the gap
// this migration fixes; previously a composed source hid inner certs).
let s = Scratch::new("multi_hc");
let line = format!(
"| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{}\n",
"00".repeat(20),
"00".repeat(92)
);
let keydb = s.write("keydb.cfg", &line);
let multi = MultiSource::new(vec![
Box::new(ScriptedSource::new("plain", vec![])),
Box::new(KeydbSource::new(&keydb)),
]);
let certs = KeySource::host_certs(&multi, None);
assert_eq!(
certs.len(),
1,
"the inner keydb's cert must be visible through the union"
);
}
#[test]
fn multi_source_nests() {
// MultiSource is itself a KeySource, so it composes inside another. Inner
// [empty, A] then outer B: A (first non-empty) wins.
let inner = MultiSource::new(vec![
Box::new(ScriptedSource::new("empty", vec![])),
Box::new(ScriptedSource::new("A", vec![uk(0xa1)])),
]);
let outer = MultiSource::new(vec![
Box::new(inner),
Box::new(ScriptedSource::new("B", vec![uk(0xb1)])),
]);
assert_eq!(resolve(&outer, &inputs("x")), vec![uk(0xa1)]);
}
#[test]
fn multi_source_real_keydb_resolves_through_chain() {
// End-to-end with the REAL keydb source over a fixture file inside a chain:
// a no-key scripted source first, then the keydb that actually resolves.
let s = Scratch::new("multi_real");
let keydb = s.write("keydb.cfg", &keydb_with_unit_key());
let multi = MultiSource::new(vec![
Box::new(ScriptedSource::new("plain", vec![])),
Box::new(KeydbSource::new(&keydb)),
]);
let got = resolve(&multi, &inputs(DISC_HASH));
assert_eq!(
got.len(),
1,
"the keydb resolves the disc once the empty source is skipped"
);
assert_eq!(
got[0].key, [0x11u8; 16],
"the keydb's terminal UK is returned"
);
}
// ── MultiSource: a source that could not ANSWER is not a source that said no ─.
/// A source that always fails, with a chosen error — the key-service outage.
struct FailingSource {
err: fn() -> libfreemkv::Error,
label: &'static str,
}
impl KeySource for FailingSource {
fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
Err((self.err)())
}
fn get_fmts_indexes(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
Err((self.err)())
}
fn label(&self) -> &'static str {
self.label
}
}
fn unavailable() -> libfreemkv::Error {
libfreemkv::Error::KeyServiceUnavailable
}
fn unauthorized() -> libfreemkv::Error {
libfreemkv::Error::KeyServiceUnauthorized
}
#[test]
fn multi_source_reports_a_source_failure_instead_of_a_clean_no_key() {
let multi = MultiSource::new(vec![
Box::new(FailingSource {
err: unavailable,
label: "down",
}),
Box::new(ScriptedSource::new("empty", vec![])),
]);
let err = multi
.get_unit_keys(&DiscInputsCtx::new(&inputs("x")))
.expect_err("a failed source must not collapse into a clean no-key");
assert_eq!(
err.code(),
libfreemkv::error::E_KEY_SERVICE_UNAVAILABLE,
"the composition must surface the service outage"
);
assert_ne!(
err.code(),
libfreemkv::error::E_NO_DISC_KEY,
"an outage must never be reported as 'this disc has no key'"
);
}
/// The forensic half carried the identical bug and is now driven through the
/// same helper — pinned separately so a future edit cannot fix one and leave
/// the other.
#[test]
fn multi_source_fmts_indexes_also_preserve_a_source_failure() {
let multi = MultiSource::new(vec![Box::new(FailingSource {
err: unavailable,
label: "down",
})]);
assert_eq!(
multi
.get_fmts_indexes(&DiscInputsCtx::new(&inputs("x")))
.expect_err("the forensic path must preserve the failure too")
.code(),
libfreemkv::error::E_KEY_SERVICE_UNAVAILABLE,
);
}
/// A source failure must NOT block the chain: a later source that actually has
/// the key still wins outright. Pins the other half of the contract, so the fix
/// cannot be over-applied into "any error aborts the composition".
#[test]
fn multi_source_failure_never_blocks_a_later_success() {
let multi = MultiSource::new(vec![
Box::new(FailingSource {
err: unavailable,
label: "down",
}),
Box::new(ScriptedSource::new("real", vec![uk(0xc1)])),
]);
assert_eq!(
resolve(&multi, &inputs("x")),
vec![uk(0xc1)],
"a failing first source must not stop the second from resolving"
);
}
/// FIRST failure wins, matching libfreemkv's `source_failure` rule: the
/// most-preferred source's reason is the one the operator is told about, in the
/// same order the caller expressed a preference for successes.
#[test]
fn multi_source_reports_the_first_failure_not_the_last() {
let multi = MultiSource::new(vec![
Box::new(FailingSource {
err: unauthorized,
label: "bad-token",
}),
Box::new(FailingSource {
err: unavailable,
label: "down",
}),
]);
assert_eq!(
multi
.get_unit_keys(&DiscInputsCtx::new(&inputs("x")))
.expect_err("both sources failed")
.code(),
libfreemkv::error::E_KEY_SERVICE_UNAUTHORIZED,
"the first source's failure is the one reported"
);
}
/// The contrast case: every source ANSWERED and none holds a key. That is the
/// genuine miss, and `E7022` is then the truth — so the fix must not turn an
/// honest empty into an error.
#[test]
fn multi_source_all_sources_answering_empty_is_still_a_clean_no_key() {
let multi = MultiSource::new(vec![
Box::new(ScriptedSource::new("a", vec![])),
Box::new(ScriptedSource::new("b", vec![])),
]);
assert!(
multi
.get_unit_keys(&DiscInputsCtx::new(&inputs("x")))
.expect("every source answered; this is a genuine miss")
.is_empty()
);
}
/// The `Path`-typed constructor accepts a borrowed path.
#[test]
fn keydb_source_accepts_borrowed_path() {
let s = Scratch::new("keydb_borrow");
let path: &Path = &s.path("keydb.cfg");
std::fs::write(path, keydb_with_unit_key()).unwrap();
let src = KeydbSource::new(path);
assert_eq!(resolve(&src, &inputs(DISC_HASH)).len(), 1);
}