tests: loopback idle bound T_IDLE is 1 s, so a slow macOS runner's scheduling gaps stay inside every margin

tests: Stop tests against a silent stub idle T_STOP_IDLE (2 s), past the Stop
tests: the stopped reachability query idles 2 s and gets 10 s to drain
2026-10-07 18:13:11 +00:00 · 2026-10-01 18:06:06 -07:00 · 2026-10-01 17:20:54 -07:00 · 2026-10-01 16:54:40 -07:00 · 2026-10-01 16:53:58 -07:00 · 2026-09-30 20:36:55 -07:00
11 changed files with 1955 additions and 362 deletions
@@ -2,10 +2,11 @@ version: 2

 # Dependency updates land on `dev`, never on `main`.
 #
-# `main` here is a RELEASE POINTER that release.sh moves to each tag. A bot
-# commit on it would put work there that no tag contains, which is exactly the
-# state that aborted the 1.6.2 cascade at the last step -- so pointing
-# Dependabot at main would recreate that failure on a schedule.
+# `main` here is a RELEASE POINTER that freemkv's release-orchestrate.yml
+# moves to each tag. A bot commit on it would put work there that no tag
+# contains, which is exactly the state that aborted the 1.6.2 cascade at the
+# last step -- so pointing Dependabot at main would recreate that failure on a
+# schedule.
 updates:
  - package-ecosystem: cargo
    directory: /
@@ -13,9 +14,7 @@ updates:
    schedule:
      interval: weekly
    open-pull-requests-limit: 5
-    # One PR per week for the routine bumps instead of one per crate. Eight
-    # repos times a handful of crates is a volume nobody reads, and an
-    # unread PR queue is indistinguishable from no updates at all.
+    # One PR per week for the routine bumps instead of one per crate.
    groups:
      minor-and-patch:
        update-types:
@@ -23,18 +22,14 @@ updates:
          - patch
    ignore:
      # The freemkv crates depend on each other by GIT TAG, re-pinned by
-      # release.sh as part of the release commit. Dependabot cannot see that
-      # cascade, so a PR bumping one of these would fight the release process
-      # and could pin a version whose tag does not exist yet.
+      # release-orchestrate.yml in the release commit. A bot bump would fight
+      # the release cascade and could pin a tag that does not exist yet.
      - dependency-name: freemkv-unlock
      - dependency-name: libfreemkv
      - dependency-name: freemkv-keysources
      - dependency-name: freemkv-i18n
      - dependency-name: freemkv-engine

-  # The workflows are now real infrastructure -- the release cascade, the
-  # cross-platform hash matrix, the disc gate -- so their actions need the same
-  # attention as the crates.
  - package-ecosystem: github-actions
    directory: /
    target-branch: dev
@@ -47,17 +42,11 @@ updates:
          - minor
          - patch
    ignore:
-      # NOT a dependency: `dtolnay/rust-toolchain` is versioned by the RUST
-      # release it installs, and the tag we pin is the toolchain CI is pinned
-      # to on purpose -- precommit.sh runs the same one locally so a lint that
-      # passes on a developer's newer default cannot pass CI by accident.
-      #
-      # Dependabot reads those tags as semver and proposed 1.97.0 -> 1.100.0,
-      # a Rust version that does not exist. Every such PR 404s on toolchain
-      # download across all eight repos, and they regenerate weekly -- eight
-      # permanently-red PRs that promote.yml then has to special-case when it
-      # decides whether dev is green.
-      #
-      # Bumping the toolchain is a deliberate, all-eight-repos change, made by
-      # hand together with precommit.sh. There is nothing here for a bot.
+      # Versioned by the Rust release it installs; CI picks the toolchain via
+      # `toolchain:` on @master/@stable. Dependabot proposes Rust versions that
+      # do not exist. Toolchain bumps are a deliberate all-repo change.
      - dependency-name: dtolnay/rust-toolchain
+      # The org's own reusable workflows/actions. @v1 and @v2 are independent
+      # moving tags (v2 is only the comment guard), not semver releases: a
+      # "bump" from v1 to v2 would move a caller onto an OLDER guard.
+      - dependency-name: "freemkv/.github*"
@@ -94,7 +94,7 @@ jobs:
      - uses: freemkv/.github/.github/actions/checkout-siblings@v1
        with:
          checkout: libfreemkv freemkv-unlock
-          patch: libfreemkv
+          patch: libfreemkv freemkv-unlock
          self: freemkv-keysources
      # fmt runs on the MSRV pin; clippy runs on stable so new-lint debt is a
      # blocking gate the pinned toolchain would hide. Stable is installed first
@@ -125,7 +125,7 @@ jobs:
      - uses: freemkv/.github/.github/actions/checkout-siblings@v1
        with:
          checkout: libfreemkv freemkv-unlock
-          patch: libfreemkv
+          patch: libfreemkv freemkv-unlock
          self: freemkv-keysources
      - uses: dtolnay/rust-toolchain@master
        with:
@@ -149,13 +149,13 @@ jobs:
      - uses: freemkv/.github/.github/actions/checkout-siblings@v1
        with:
          checkout: libfreemkv freemkv-unlock
-          patch: libfreemkv
+          patch: libfreemkv freemkv-unlock
          self: freemkv-keysources
      - uses: dtolnay/rust-toolchain@stable
-      - uses: EmbarkStudios/cargo-deny-action@v2
-        with:
-          command: check advisories bans licenses sources
-          manifest-path: freemkv-keysources/Cargo.toml
+      - uses: taiki-e/install-action@cargo-deny
+      # Run from the repo dir so the generated .cargo/config.toml path patches apply.
+      - run: cargo deny --all-features check advisories bans licenses sources
+        working-directory: freemkv-keysources

  # This crate ships wherever freemkv and autorip ship, and it drags in all of
  # libfreemkv's platform code (SCSI transports included). Build the tests, not
@@ -182,7 +182,7 @@ jobs:
      - uses: freemkv/.github/.github/actions/checkout-siblings@v1
        with:
          checkout: libfreemkv freemkv-unlock
-          patch: libfreemkv
+          patch: libfreemkv freemkv-unlock
          self: freemkv-keysources
      - uses: dtolnay/rust-toolchain@master
        with:
@@ -209,7 +209,7 @@ jobs:
      - uses: freemkv/.github/.github/actions/checkout-siblings@v1
        with:
          checkout: libfreemkv freemkv-unlock
-          patch: libfreemkv
+          patch: libfreemkv freemkv-unlock
          self: freemkv-keysources
      - uses: dtolnay/rust-toolchain@master
        with:
@@ -250,7 +250,7 @@ jobs:
      - uses: freemkv/.github/.github/actions/checkout-siblings@v1
        with:
          checkout: libfreemkv freemkv-unlock
-          patch: libfreemkv
+          patch: libfreemkv freemkv-unlock
          self: freemkv-keysources
      - uses: dtolnay/rust-toolchain@master
        with:
@@ -96,7 +96,7 @@ jobs:
      - uses: freemkv/.github/.github/actions/checkout-siblings@v1
        with:
          checkout: libfreemkv freemkv-unlock
-          patch: libfreemkv
+          patch: libfreemkv freemkv-unlock
          self: freemkv-keysources
      - uses: dtolnay/rust-toolchain@master
        with:
@@ -119,7 +119,7 @@ jobs:
      - uses: freemkv/.github/.github/actions/checkout-siblings@v1
        with:
          checkout: libfreemkv freemkv-unlock
-          patch: libfreemkv
+          patch: libfreemkv freemkv-unlock
          self: freemkv-keysources
      - uses: dtolnay/rust-toolchain@stable
        with:
@@ -145,7 +145,7 @@ jobs:
      - uses: freemkv/.github/.github/actions/checkout-siblings@v1
        with:
          checkout: libfreemkv freemkv-unlock
-          patch: libfreemkv
+          patch: libfreemkv freemkv-unlock
          self: freemkv-keysources
      - uses: dtolnay/rust-toolchain@master
        with:
@@ -1,5 +1,25 @@
 # Changelog

+## [Unreleased]
+
+### Fixed
+
+- The online key service ignores `HTTP_PROXY`/`HTTPS_PROXY`/`ALL_PROXY`: with one set, every key lookup failed (the pinned resolver also answered the proxy's address, so the proxy request went to the key service).
+
+### Changed
+
+- `KeydbSource` hands out host certs best first, with no disc input: certs with no `Revoked in MKBv<N>` note in keydb order, then revoked ones by latest revocation. The inherent `host_certs()` and the `KeySource::host_certs` trait method both return this order; the trait ignores its `mkb` argument.
+
+### Deprecated
+
+- `KeyDb::host_certs(mkb)`: use `KeyDb::host_certs_ranked`.
+
+### Added
+
+- `KeyDb::host_certs_ranked`.
+- `check_keyserver_url` with `KeyserverUrlRejection`/`KeyserverUrlFault`: a typed Permanent vs Temporary verdict, so callers need not match `validate_keyserver_url`'s text.
+- `test-hooks` feature exposing `set_last_decode_reachability`, to test decode-verdict handling without a network.
+
 ## [1.7.7] — 2026-09-26

 ### Maintenance
@@ -31,6 +31,12 @@ version = "0.23.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5"

+[[package]]
+name = "bitflags"
+version = "2.13.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3ded4057c258ba199e2d26386d3af3780957ecaee6c4ef4041c6b4b8b97c0b06"
+
 [[package]]
 name = "block-buffer"
 version = "0.12.1"
@@ -240,6 +246,16 @@ version = "1.0.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"

+[[package]]
+name = "errno"
+version = "0.3.14"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
+dependencies = [
+ "libc",
+ "windows-sys 0.52.0",
+]
+
 [[package]]
 name = "find-msvc-tools"
 version = "0.1.11"
@@ -497,13 +513,22 @@ dependencies = [
 "memchr",
 "rayon",
 "roxmltree",
+ "rustix",
 "serde",
 "serde_json",
 "sha1",
+ "sha2",
+ "socket2",
 "tracing",
 "zip",
 ]

+[[package]]
+name = "linux-raw-sys"
+version = "0.12.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
+
 [[package]]
 name = "litemap"
 version = "0.8.3"
@@ -677,7 +702,7 @@ dependencies = [
 "getrandom 0.2.17",
 "libc",
 "untrusted",
- "windows-sys",
+ "windows-sys 0.52.0",
 ]

 [[package]]
@@ -686,6 +711,19 @@ version = "0.20.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "6c20b6793b5c2fa6553b250154b78d6d0db37e72700ae35fad9387a46f487c97"

+[[package]]
+name = "rustix"
+version = "1.1.5"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "891efababe418670775f199f0d233d84843c227a0949a883ce15b37c78d6629d"
+dependencies = [
+ "bitflags",
+ "errno",
+ "libc",
+ "linux-raw-sys",
+ "windows-sys 0.52.0",
+]
+
 [[package]]
 name = "rustls"
 version = "0.23.45"
@@ -804,6 +842,16 @@ version = "1.16.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "b9be42f50aa861c555654aa3a37f52f4b1074bacf4e48fe0ef7fa584e80f1f0f"

+[[package]]
+name = "socket2"
+version = "0.6.5"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4"
+dependencies = [
+ "libc",
+ "windows-sys 0.61.2",
+]
+
 [[package]]
 name = "stable_deref_trait"
 version = "1.2.1"
@@ -1021,6 +1069,12 @@ dependencies = [
 "rustls-pki-types",
 ]

+[[package]]
+name = "windows-link"
+version = "0.2.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
+
 [[package]]
 name = "windows-sys"
 version = "0.52.0"
@@ -1030,6 +1084,15 @@ dependencies = [
 "windows-targets",
 ]

+[[package]]
+name = "windows-sys"
+version = "0.61.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
+dependencies = [
+ "windows-link",
+]
+
 [[package]]
 name = "windows-targets"
 version = "0.52.6"
@@ -1242,3 +1305,7 @@ version = "1.7.7"
 [[patch.unused]]
 name = "freemkv-i18n"
 version = "1.7.7"
+
+[[patch.unused]]
+name = "freemkv-keysources"
+version = "1.7.7"
@@ -30,6 +30,15 @@ zip = { version = "8", default-features = false, features = ["deflate"] }
 # key material).
 tracing = "0.1"

+[features]
+# `set_last_decode_reachability`, for consumers' tests only (enable in dev-dependencies).
+test-hooks = []
+
+[dev-dependencies]
+# `test_util` fixtures (`decrypt_unit`, …) for this crate's tests (keys-upfront
+# KU-P1). CI resolves it through the sibling path patch.
+libfreemkv = { version = "1.7", features = ["test-util"] }
+
 [profile.release]
 lto = "thin"
 codegen-units = 1
@@ -41,6 +50,7 @@ codegen-units = 1
 # config-level [patch.crates-io] wins over this manifest one for the same crate).
 [patch.crates-io]
 libfreemkv = { git = "https://github.com/freemkv/libfreemkv", tag = "v1.7.7" }
+freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.7.7" }

 [lints.clippy]
 uninlined_format_args = "allow"
@@ -324,13 +324,11 @@ impl KeydbSource {
    /// live-drive scan as `DriveCredentials` for the AACS handshake. Empty if
    /// the keydb is missing/unreadable or carries no host cert.
    ///
-    /// Inherent, no-MKB form: this is used by the **scan-options** builder,
-    /// which runs before the disc's MKB generation is known, so no revocation
-    /// filtering is applied (passes `None`). The [`KeySource::host_certs`] TRAIT
-    /// method wires the real MKB generation through for revocation filtering.
+    /// Best first ([`KeyDb::host_certs_ranked`]): this is what the scan-options
+    /// builder hands the live-drive handshake as `DriveCredentials`.
    pub fn host_certs(&self) -> Vec<HostCert> {
        match self.cached_db() {
-            Ok(db) => db.host_certs(None),
+            Ok(db) => db.host_certs_ranked(),
            // No error channel here (the scan-options builder wants a list), so
            // the failure can only be LOGGED — but it must not be invisible.
            Err(e) => {
@@ -458,10 +456,10 @@ impl KeydbSource {

        // On a match with no key: carry the specific reason if we have one, else
        // leave it empty for the library to render a bare `NoDerivableKey`.
-        let miss_path = if keys.is_empty() {
-            miss_reason.map(|n| vec![n]).unwrap_or_default()
-        } else {
-            Vec::new()
+        // KU J23: `NoVid` stays beside partial keys (the VID would derive the rest).
+        let miss_path = match miss_reason {
+            Some(n) if keys.is_empty() || n == KeyNode::NoVid => vec![n],
+            _ => Vec::new(),
        };

        KeydbResolution {
@@ -631,11 +629,10 @@ impl KeySource for KeydbSource {
        }
    }

-    // Expose the keydb's host certs through the trait, wiring the disc's MKB generation through
-    // for revocation filtering.
-    fn host_certs(&self, mkb: Option<u32>) -> Vec<HostCert> {
+    // The keydb's host certs, best first. `mkb` is ignored: cert selection takes no disc input.
+    fn host_certs(&self, _mkb: Option<u32>) -> Vec<HostCert> {
        match self.cached_db() {
-            Ok(db) => db.host_certs(mkb),
+            Ok(db) => db.host_certs_ranked(),
            // Vec-returning trait method: log the failure, return nothing.
            Err(e) => {
                let _ = self.load_failure(&e);
@@ -644,6 +641,12 @@ impl KeySource for KeydbSource {
        }
    }

+    // Keyed by disc hash, not by samples: asked once per resolve, not per
+    // piece (KU §2.3 step 8, N-KU10; KSK1).
+    fn answer_depends_on_samples(&self) -> bool {
+        false
+    }
+
    fn label(&self) -> &'static str {
        "keydb"
    }
@@ -789,6 +792,19 @@ mod tests {
        assert!(!shape.vid_available);
    }

+    // KU J23: stored unit keys for SOME CPS units plus a Media Key with no VID. The stored
+    // keys come back, and the miss path still says `NoVid`: the VID would derive the rest.
+    #[test]
+    fn partial_stored_keys_with_a_media_key_and_no_vid_report_no_vid() {
+        let mut e = blank_entry(HASH);
+        e.media_key = Some([0x33u8; 16]);
+        e.unit_keys = vec![(1, [0x55u8; 16])];
+        let db = db_with(e, Vec::new());
+        let r = KeydbSource::resolve_from(&db, &ctx(HASH, vec![[0x44u8; 16]; 2], None));
+        assert_eq!(r.keys.len(), 1, "the stored key");
+        assert_eq!(r.miss_path, vec![KeyNode::NoVid]);
+    }
+
    // A hash that is NOT in the keydb is the one true miss: not matched, no
    // shape, no miss path (the library renders this as `no entry`).
    #[test]
@@ -1145,6 +1161,15 @@ mod tests {
        assert_eq!(KeydbSource::new("/nonexistent/keydb.cfg").label(), "keydb");
    }

+    // KSK1: keyed by disc hash, so `resolve` asks it once per rip, not once per
+    // piece (KU §2.3 step 8, N-KU10).
+    #[test]
+    fn keydb_source_answer_does_not_depend_on_samples() {
+        assert!(!KeydbSource::new("/nonexistent/keydb.cfg").answer_depends_on_samples());
+        // KU J23: a keydb's VID need is per disc (its `NoVid` miss path), not blanket.
+        assert!(!KeydbSource::new("/nonexistent/keydb.cfg").uses_vid());
+    }
+
    // A stamp with no mtime (mtime-less FS) or a future mtime (clock skew) must
    // not be stuck un-settled forever — it now settles via the inode-identity
    // fallback once the observation has aged past the granularity.
@@ -1192,6 +1217,46 @@ mod tests {
        );
    }

+    /// Best-first with no disc input: unannotated certs in file order, then revoked
+    /// ones by latest revocation. The trait ignores its MKB argument.
+    #[test]
+    fn host_certs_ranked_orders_unannotated_then_latest_revocation_first() {
+        let dir = scratch("ranked");
+        let path = dir.join("keydb.cfg");
+        let hc = |marker: u8, note: &str| {
+            format!(
+                "| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{}{}\n",
+                "00".repeat(20),
+                format!("{marker:02x}").repeat(92),
+                note
+            )
+        };
+        let cfg = [
+            hc(0x68, " ; Revoked in MKBv68"),
+            hc(0xA1, ""),
+            hc(0x72, " ; Revoked in MKBv72"),
+            hc(0xB2, ""),
+        ]
+        .concat();
+        std::fs::write(&path, cfg).unwrap();
+        let src = KeydbSource::new(&path);
+        let markers =
+            |certs: Vec<HostCert>| -> Vec<u8> { certs.iter().map(|c| c.certificate[0]).collect() };
+        let want = vec![0xA1, 0xB2, 0x72, 0x68];
+        assert_eq!(
+            markers(src.host_certs()),
+            want,
+            "inherent (production credentials)"
+        );
+        assert_eq!(
+            markers(KeySource::host_certs(&src, Some(99))),
+            want,
+            "trait ignores mkb"
+        );
+        assert_eq!(markers(KeySource::host_certs(&src, None)), want);
+        std::fs::remove_dir_all(&dir).ok();
+    }
+
    /// No keydb → no host credentials, not an error (inherent and trait forms).
    #[test]
    fn host_certs_empty_when_keydb_missing() {
@@ -1201,8 +1266,8 @@ mod tests {
        assert!(KeySource::host_certs(&src, Some(68)).is_empty());
    }

-    /// The TRAIT `host_certs` surfaces a `| HC |` row and now wires the MKB
-    /// generation through. Placeholder all-zero material (never a real key).
+    /// The TRAIT `host_certs` surfaces a `| HC |` row for any `mkb` argument.
+    /// Placeholder all-zero material (never a real key).
    #[test]
    fn trait_host_certs_returns_keydb_hc_row() {
        let dir = std::env::temp_dir().join(format!("fmk_hc_{}", std::process::id()));
@@ -477,6 +477,7 @@ impl KeyDb {
    /// is `None` the disc's generation is unknown and cannot be filtered, so
    /// every cert is returned; certs with no revocation annotation are always
    /// returned.
+    #[deprecated(note = "use host_certs_ranked; host cert selection takes no disc input")]
    pub fn host_certs(&self, mkb: Option<u32>) -> Vec<HostCert> {
        self.host_certs
            .iter()
@@ -489,6 +490,19 @@ impl KeyDb {
            .collect()
    }

+    /// Every host cert, best first, with no disc input: certs with no `Revoked in
+    /// MKBv<N>` note in file order, then revoked ones by latest revocation (a later
+    /// revocation is the better bet without knowing the disc's generation).
+    pub fn host_certs_ranked(&self) -> Vec<HostCert> {
+        let mut ranked: Vec<&KeydbHostCert> = self.host_certs.iter().collect();
+        // Stable sort: ties (every unannotated cert, equal revocations) keep file order.
+        ranked.sort_by_key(|hc| {
+            let r = hc.revoked_at_mkb;
+            (r.is_some(), std::cmp::Reverse(r.unwrap_or(0)))
+        });
+        ranked.into_iter().map(|hc| hc.cert.clone()).collect()
+    }
+
    /// Standalone keydb accessor: the disc's Volume ID (the keydb `I` token),
    /// looked up by the same disc-hash form [`Self::find_disc`] accepts. Pure
    /// file lookup; no crypto/derivation.
@@ -1913,6 +1927,7 @@ mod tests {
    // ── Host-cert revocation: parse + host_certs(mkb) filter ────────────────

    #[test]
+    #[allow(deprecated)]
    fn host_cert_revoked_parses_and_filters_by_mkb() {
        let revoked_line = format!(
            "| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{} ; Revoked in MKBv72",
@@ -1942,6 +1957,7 @@ mod tests {
    }

    #[test]
+    #[allow(deprecated)]
    fn host_cert_without_revocation_included_for_all_mkb() {
        let line = format!(
            "| HC | HOST_PRIV_KEY 0x{} | HOST_CERT 0x{}",
@@ -1958,6 +1974,7 @@ mod tests {
    }

    #[test]
+    #[allow(deprecated)]
    fn hc2_revocation_propagates_when_hc_has_none() {
        // The HC line carries no annotation; the revocation lives on the HC2
        // line. The combined cert must still be filtered by that generation
@@ -2178,6 +2195,7 @@ mod tests {
    /// so the returned SET is identified exactly, not merely counted. A cert
    /// revoked at generation R is usable iff the disc's generation is `< R`.
    #[test]
+    #[allow(deprecated)]
    fn host_certs_filters_several_certs_across_mkb_generations() {
        let hc = |marker: u8, note: &str| {
            format!(
@@ -2405,7 +2423,7 @@ mod tests {
        for entry in &candidate_entries {
            for (_, key) in &entry.unit_keys {
                let mut unit = original.clone();
-                libfreemkv::aacs::content::decrypt_unit(&mut unit, key);
+                libfreemkv::test_util::decrypt_unit(&mut unit, key);
                attempts += 1;
                any_key_changed_the_unit |= unit != original;
                if libfreemkv::aacs::content::is_clean(&unit, libfreemkv::disc::ContentFormat::BdTs)
@@ -16,8 +16,11 @@ mod paths;

 pub use keydb::{KeydbSource, UpdateResult};
 pub use keydb_format::{DiscEntry, KeyDb};
+#[cfg(feature = "test-hooks")]
+pub use online::set_last_decode_reachability;
 pub use online::{
-    DecodeReachability, MIN_SAMPLE_UNITS, OnlineSource, take_last_decode_reachability,
+    DecodeReachability, KeyserverUrlFault, KeyserverUrlRejection, MIN_SAMPLE_UNITS, OnlineSource,
+    check_keyserver_url, check_keyserver_url_static, take_last_decode_reachability,
    validate_keyserver_url,
 };
 pub use paths::{default_keydb_path, existing_keydb_path, keydb_search_paths};
@@ -134,9 +137,9 @@ impl KeySource for MultiSource {
        first_non_empty(&self.sources, |s, c| s.get_fmts_indexes(c), ctx)
    }

-    /// UNION every inner source's host certs (filtered at the given MKB
-    /// generation). Without this a composed source would hide an inner source's
-    /// cert from the OEM cert-auth route — the gap this fixes.
+    /// UNION every inner source's host certs, in source order (`mkb` is passed
+    /// through unchanged). Without this a composed source would hide an inner
+    /// source's cert from the OEM cert-auth route — the gap this fixes.
    fn host_certs(&self, mkb: Option<u32>) -> Vec<libfreemkv::aacs::types::HostCert> {
        self.sources
            .iter()
@@ -147,6 +150,23 @@ impl KeySource for MultiSource {
    fn label(&self) -> &'static str {
        "multi"
    }
+
+    // KU3-6: depends on samples if ANY inner source does, so a nested
+    // sample-dependent source keeps getting the per-piece ask.
+    fn answer_depends_on_samples(&self) -> bool {
+        self.sources.iter().any(|s| s.answer_depends_on_samples())
+    }
+
+    // Same `any(inner)` rule (KU-K1 review): `resolve` retries the composition
+    // whenever any inner source's last failure was transport-class.
+    fn last_failure_was_transport(&self) -> bool {
+        self.sources.iter().any(|s| s.last_failure_was_transport())
+    }
+
+    // KU J23: any inner source that consumes the VID makes the composition one.
+    fn uses_vid(&self) -> bool {
+        self.sources.iter().any(|s| s.uses_vid())
+    }
 }

 #[cfg(test)]
@@ -212,6 +232,92 @@ mod tests {
        assert!(multi.host_certs(None).is_empty());
    }

+    // A source whose `answer_depends_on_samples` is fixed to the given value —
+    // the only behaviour KSK2 exercises.
+    struct FixedDependency(bool);
+    impl KeySource for FixedDependency {
+        fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
+            Ok(Vec::new())
+        }
+        fn answer_depends_on_samples(&self) -> bool {
+            self.0
+        }
+    }
+
+    // KSK2: `MultiSource::answer_depends_on_samples` is `any(inner)`, so nesting
+    // a sample-dependent online source under a sample-independent keydb still
+    // gets the per-piece ask (`resolve` must not treat the pair as keydb-only).
+    #[test]
+    fn multi_source_answer_depends_on_samples_is_any_inner() {
+        let all_independent = MultiSource::new(vec![
+            Box::new(FixedDependency(false)) as Box<dyn KeySource>,
+            Box::new(FixedDependency(false)) as Box<dyn KeySource>,
+        ]);
+        assert!(!all_independent.answer_depends_on_samples());
+
+        let keydb_then_online = MultiSource::new(vec![
+            Box::new(FixedDependency(false)) as Box<dyn KeySource>,
+            Box::new(FixedDependency(true)) as Box<dyn KeySource>,
+        ]);
+        assert!(keydb_then_online.answer_depends_on_samples());
+    }
+
+    // A source whose `uses_vid` is fixed to the given value.
+    struct FixedVid(bool);
+    impl KeySource for FixedVid {
+        fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
+            Ok(Vec::new())
+        }
+        fn uses_vid(&self) -> bool {
+            self.0
+        }
+    }
+
+    // KU J23: like `answer_depends_on_samples`, a composed source uses the VID if any
+    // inner one does, so a nested online source still makes a Missing piece VID-derivable.
+    #[test]
+    fn multi_source_uses_vid_is_any_inner() {
+        let neither = MultiSource::new(vec![
+            Box::new(FixedVid(false)) as Box<dyn KeySource>,
+            Box::new(FixedVid(false)) as Box<dyn KeySource>,
+        ]);
+        assert!(!neither.uses_vid());
+        let keydb_then_online = MultiSource::new(vec![
+            Box::new(FixedVid(false)) as Box<dyn KeySource>,
+            Box::new(FixedVid(true)) as Box<dyn KeySource>,
+        ]);
+        assert!(keydb_then_online.uses_vid());
+    }
+
+    // A source whose `last_failure_was_transport` is fixed to the given value.
+    struct FixedTransport(bool);
+    impl KeySource for FixedTransport {
+        fn get_unit_keys(&self, _ctx: &dyn ResolveCtx) -> Result<Vec<UnitKey>, libfreemkv::Error> {
+            Ok(Vec::new())
+        }
+        fn last_failure_was_transport(&self) -> bool {
+            self.0
+        }
+    }
+
+    // For consistency with `answer_depends_on_samples` (KU3-6): a composed
+    // source's last-failure verdict is `any(inner)` too, so `resolve` retries
+    // the composition whenever ANY inner source's last failure was transport.
+    #[test]
+    fn multi_source_last_failure_was_transport_is_any_inner() {
+        let none_transport = MultiSource::new(vec![
+            Box::new(FixedTransport(false)) as Box<dyn KeySource>,
+            Box::new(FixedTransport(false)) as Box<dyn KeySource>,
+        ]);
+        assert!(!none_transport.last_failure_was_transport());
+
+        let one_transport = MultiSource::new(vec![
+            Box::new(FixedTransport(false)) as Box<dyn KeySource>,
+            Box::new(FixedTransport(true)) as Box<dyn KeySource>,
+        ]);
+        assert!(one_transport.last_failure_was_transport());
+    }
+
    /// The `mkb` generation must reach each inner source so its OWN revocation
    /// filter can act; a source that drops everything at a given generation is
    /// honoured by the union.
@@ -138,7 +138,7 @@ fn keydb_source_label_is_keydb() {
    assert_eq!(KeydbSource::new("/nonexistent/keydb.cfg").label(), "keydb");
 }

-// ── KeydbSource: MKB-aware host-cert serving from a file ────────────────────
+// ── KeydbSource: host-cert serving from a file ──────────────────────────────

 #[test]
 fn keydb_source_serves_host_cert_from_hc_row() {
@@ -152,14 +152,13 @@ fn keydb_source_serves_host_cert_from_hc_row() {
    let path = s.write("keydb.cfg", &line);

    let src = KeydbSource::new(&path);
-    // Inherent (no-MKB, scan-options) form.
+    // Inherent (scan-options) form.
    assert_eq!(
        src.host_certs().len(),
        1,
        "inherent host_certs sees the HC row"
    );
-    // Trait form now wires the MKB generation through (no revocation annotation
-    // → always returned).
+    // Trait form: `mkb` is ignored; the ranked list is returned.
    let via_trait = KeySource::host_certs(&src, Some(70));
    assert_eq!(via_trait.len(), 1, "trait host_certs sees the HC row");
    assert_eq!(via_trait[0].certificate.len(), 92);
@@ -229,13 +228,14 @@ fn online_source_metadata() {
 }

 #[test]
-fn validate_keyserver_url_gates_scheme_and_ssrf() {
+fn validate_keyserver_url_gates_scheme_and_allows_lan() {
    assert!(validate_keyserver_url("https://8.8.8.8/keys").is_ok());
+    // Home app: LAN/loopback/link-local hosts pass; cleartext http and invalid addresses do not.
+    assert!(validate_keyserver_url("https://127.0.0.1/keys").is_ok());
+    assert!(validate_keyserver_url("https://169.254.1.5/keys").is_ok());
+    assert!(validate_keyserver_url("https://[::1]:9000/keys").is_ok());
    assert!(validate_keyserver_url("http://127.0.0.1/keys").is_err());
-    // SSRF blocking is IP-based, not scheme-gated: https://<private-IP> is rejected too.
-    assert!(validate_keyserver_url("https://127.0.0.1/keys").is_err());
-    assert!(validate_keyserver_url("http://169.254.169.254/latest/meta-data/").is_err());
-    assert!(validate_keyserver_url("http://[::1]:9000/keys").is_err());
+    assert!(validate_keyserver_url("https://0.0.0.0/keys").is_err());
    assert!(validate_keyserver_url("ftp://example.com/keys").is_err());
    assert!(validate_keyserver_url("").is_err());
 }
Author	SHA1	Message	Date
Matthew Jackson	3c6ef54cf3	tests: loopback idle bound T_IDLE is 1 s, so a slow macOS runner's scheduling gaps stay inside every margin	2026-10-01 18:06:06 -07:00
Matthew Jackson	73c7f05b4b	tests: Stop tests against a silent stub idle T_STOP_IDLE (2 s), past the Stop	2026-10-01 17:20:54 -07:00
Matthew Jackson	e6320d52c9	tests: the stopped reachability query idles 2 s and gets 10 s to drain	2026-10-01 16:54:40 -07:00
Matthew Jackson	49cf396c07	tests: the stopped reachability query idles past the Stop, so only the Stop ends it	2026-10-01 16:53:58 -07:00
Matthew Jackson	940bb215ab	online: follow libfreemkv's halt::Progress rename to Liveness	2026-09-30 20:36:55 -07:00
Matthew Jackson	18b590407c	Cargo.lock: restore (accidentally changed by local patch)	2026-09-30 20:24:57 -07:00
Matthew Jackson	116b33739b	tests: halt-landing tests assert Halted, not a tight wall-clock bound	2026-09-30 20:24:46 -07:00
Matthew Jackson	d935fa6ffd	online: allow LAN/loopback/link-local key-service hosts, reuse libfreemkv is_blocked_ip	2026-09-30 20:03:38 -07:00
Matthew Jackson	b65959d3a1	ci: run cargo-deny from the repo dir so the .cargo patches apply	2026-09-30 16:09:03 -07:00
Matthew Jackson	3f08e980a0	ci: path-patch freemkv-unlock; dependabot ignores org workflow refs libfreemkv now deps freemkv-unlock by version with a git-tag override, so every root that reaches it patches it. Dependabot config unified; it no longer proposes freemkv/.github v1->v2.	2026-09-30 15:53:05 -07:00
Matthew Jackson	6c8e2c5c46	Allow forensic phase fallback after definitive key service misses	2026-09-29 17:20:43 -07:00
Matthew Jackson	ba357e46f5	online: wire ctx.halt()/ctx.progress() into the query worker (ST-K1b)	2026-09-28 08:18:38 -07:00
Matthew Jackson	a815a0a578	online: ST-K1b tests for ctx.halt()/ctx.progress() wiring (RED)	2026-09-28 08:15:20 -07:00
Matthew Jackson	ba6f213be3	KU-E1b: the key-service query is not yet Stop-aware (wording) check_keyserver_url_static's rustdoc and its test comment no longer call the first query halt-aware: OnlineSource runs it under its own Halt until ST-K1b wires ctx.halt() (J10).	2026-09-28 07:18:27 -07:00
Matthew Jackson	37568b1002	test: use the RFC 2606 .test TLD for the unresolvable host (leak-guard flags .invalid)	2026-09-28 07:04:48 -07:00
Matthew Jackson	40389acd3f	KU-E1 review minor 4: partial stored keys plus a Km still report NoVid The keydb keeps a NoVid miss reason beside the keys it did return, so resolve sets vid_would_help for the CPS units only the VID could derive. Other miss reasons are still reported only with no key.	2026-09-28 06:28:54 -07:00
Matthew Jackson	54508d8ec2	KU-E1 review minor 4 red: partial stored keys plus a Km still report NoVid A keydb entry with stored unit keys for some CPS units and a Media Key but no VID returns the stored keys with an empty miss path, so resolve cannot tell that the VID would derive the rest (J23).	2026-09-28 06:28:47 -07:00
Matthew Jackson	2c93420970	KU-E1: a DNS-free key-service URL check for factory builds check_keyserver_url_static rejects every Config fault (scheme, host, port, a literal non-public address) with no lookup. A factory build, which no Stop can reach, uses it; the host lookup and its SSRF guard run at the first query on the halt-aware worker, as they already did. A test-only counter proves the check starts no lookup.	2026-09-28 06:24:52 -07:00
Matthew Jackson	c48faa042b	KU-E1 red: a DNS-free key-service URL check for factory builds The engine checks the key-service URL, a DNS lookup with a fixed 10 s timeout that Stop cannot interrupt, on every factory build, and no Halt reaches a factory. The test asks for a static check that rejects Config faults with no lookup.	2026-09-28 05:45:58 -07:00
Matthew Jackson	ab93818012	KU-E1 J23: OnlineSource and MultiSource report uses_vid OnlineSource::uses_vid is true (it sends vid_b64); MultiSource is any(inner); KeydbSource keeps the default false.	2026-09-28 05:31:51 -07:00
Matthew Jackson	f3b66307cc	KU-E1 J23 red: which key sources consume the disc's VID OnlineSource sends the VID (vid_b64), so it uses it; MultiSource uses it if any inner source does; KeydbSource does not (its Km-without-VID case is reported per disc, as a NoVid miss path).	2026-09-28 05:31:23 -07:00
Matthew Jackson	ad59f3ce45	KU-K1 review: peek the reachability slot, MultiSource forwards last_failure record_last_failure_transport now reads LAST_DECODE_REACHABILITY.with(\|c\| c.get()) directly instead of take_last_decode_reachability(), so it no longer erases the slot freemkv-library's server reads after resolve (server/ripper/mod.rs:505, :1361) to avoid a second key-service call. Green: the regression test now finds the verdict still in the slot after get_unit_keys/get_fmts_indexes. MultiSource::last_failure_was_transport forwards any(inner), matching answer_depends_on_samples (KU3-6).	2026-09-28 04:19:04 -07:00
Matthew Jackson	4e83b3c1f8	KU-K1 review: red — record_last_failure_transport erases the reachability slot Opus's review: freemkv-library's server calls take_online_decode_reachability() after resolve (server/ripper/mod.rs:505, :1361) to classify a no-key without a second probe. record_last_failure_transport currently TAKES the thread-local slot, so the server sees None and fires probe_online_reachability — a second key-service call, and a definite 422 misread as an outage. Red: the slot is gone after get_unit_keys/get_fmts_indexes. Also adds the forensic-path, mid-body-stall, transport-then-5xx-reset and MultiSource forwarding tests requested in review; the latter fails since MultiSource has no override yet.	2026-09-28 04:18:02 -07:00
Matthew Jackson	c5985d1c87	KU-K1: keydb/multi sample-dependence and online transport-class failures Per keys-upfront-design KU-K1 (§8.2): KeydbSource::answer_depends_on_samples is false (keyed by disc hash, asked once per resolve, N-KU10); MultiSource's is any(inner) (KU3-6), so a nested sample-dependent source still gets the per-piece ask. OnlineSource::last_failure_was_transport (J15) is true only when the most recent query recorded DecodeReachability::Transport (DNS, connect refused, or an idle stall — never an HTTP answer, 5xx included), and resets on the next success, so resolve's J13 retry (stop-design-v5 T31) targets exactly the requests that never got an answer.	2026-09-28 04:13:03 -07:00
Matthew Jackson	8f923fd5f9	KU-K1: tests for answer_depends_on_samples and last_failure_was_transport KeydbSource::answer_depends_on_samples (KSK1) and MultiSource::answer_depends_on_samples (KSK2, KU3-6 "any(inner)") both default to `true` today, so a keydb-then-online composition would be re-asked per piece instead of once. OnlineSource::last_failure_was_transport (J15) defaults to `false`, so a transient DNS/connect/idle failure would never be retried by `resolve` (J13). Red: these six assertions fail against today's defaults.	2026-09-28 04:12:12 -07:00
Matthew Jackson	18fdab6b77	KU-P1: decrypt the real-unit test through test_util::decrypt_unit aacs::content::decrypt_unit becomes pub(crate) at KU-X2 (keys-upfront design v3.4 §2.2); the ignored real-unit test now uses the test-util door. The feature comes in as a dev-dependency (resolved through the sibling path patch).	2026-09-28 00:22:08 -07:00
Matthew Jackson	dfa02adbe2	online: a stopped key query never sends and builds no body Review fixes, green half: - PostJob carries the caller's Halt and re-checks it after the host lookup, before connecting. A Stop during DNS then never uploads the key material or bearer token, so a reopen's query is the only one the service sees. - query_with returns Halted before reading or base64-encoding the MKB when a Stop is already pending.	2026-09-27 23:50:33 -07:00
Matthew Jackson	cbab284c85	online: tests that a stopped key query sends nothing and reads no disc material Review fixes, red half: a Stop during the host lookup must never reach the service (never call the key service twice), and a pending Stop returns before the body is built. KT7a's u64::MAX check now asserts saturation.	2026-09-27 23:49:08 -07:00
Matthew Jackson	0ffdb2a9f6	online: stall-only key-service timeouts and mid-flight Stop (ST-K1a) Commit 2 of 2 (stop-design-v5 §2.7, T15-T17, D3, D4): - query_with builds the body on the caller's thread and runs DNS + POST on a worker, capped at 4 per host. The caller waits for a slot and for the result halt-aware in 20 ms slices, so Stop returns Halted within a slice at any stage. The abandoned worker frees its slot at its own stall bound. - Reachability is recorded on the caller's thread, only when the worker's result arrives. A Stop records nothing. - IdleConnector, chained after DefaultConnector, owns every post-connect bound. Each write and read gets a fresh 60 s stall bound (T16), and the wait for a reply's first byte gets 60 s + request bytes / 256 KiB/s (T17). The 180 s phase totals are gone: no total is configured. - A reply body that stalls or drops records Transport reachability (T16). An over-cap body still records its Status. The trait path passes a fresh Halt. ctx.halt()/ctx.progress() is ST-K1b, after KU-X2. No public API change.	2026-09-27 23:41:43 -07:00
Matthew Jackson	adbb1a12a0	online: ST-K1a tests for stall-only key-service timeouts and mid-flight Stop Commit 1 of 2 (stop-design-v5 §5.0): KT1-KT10 plus guards G12/G13, with signatures that keep today's behaviour (query_with ignores the halt, hardened_agent_with maps idle onto the old phase totals, the first-byte budget does not scale), so each test compiles and fails on its assertion. KT9 replaces the 180 s totals test. The ctx.halt()/ctx.progress() wiring is ST-K1b (after KU-X2).	2026-09-27 23:37:22 -07:00
Matthew Jackson	05281249d1	keydb: hand out host certs best first, with no disc input Unannotated certs in keydb order, then revoked ones by latest revocation. The trait ignores its mkb hint; KeyDb::host_certs(mkb) is deprecated.	2026-09-27 17:24:04 -07:00
Matthew Jackson	cedbc4b1fa	online: offline Temporary-verdict test; proxy wording Drop the real-DNS .invalid test for an offline check of every GuardFail::Unreachable reason; say every key lookup failed.	2026-09-27 16:44:34 -07:00
Matthew Jackson	1606ff3ca3	online: review nits for the proxy fix and keyserver URL verdict Correct the rationale: the env proxy was resolved to the key service's pinned address, so every lookup failed. Gate set_last_decode_reachability behind a test-hooks feature, mark the verdict types non_exhaustive, test a .invalid host, add CHANGELOG.	2026-09-27 12:40:09 -07:00
Matthew Jackson	10b0348db8	online: never use an env proxy; typed keyserver URL verdict and reachability test hook The key-service agent disables ureq's env proxy, which would re-resolve the host and bypass the pinned, SSRF-guarded addresses. check_keyserver_url returns a Permanent/Temporary KeyserverUrlRejection; set_last_decode_reachability plants a verdict for caller tests.	2026-09-27 12:26:39 -07:00