Compare commits

...
10 Commits
Author SHA1 Message Date
Matthew Jackson f49ef023cf changelog: enrich 1.1.0-beta.1 and rc.5.2 entries with concrete mechanisms (version label SoT, per-extent AACS anchor, E7017/E7022 no-key split, 3-sector unit grid)
leak-guard / leak-guard (push) Successful in 28s
2026-06-26 22:26:15 -07:00
Matthew Jackson ba5e4fdafa changelog: add 1.1.0-beta.1 version-stamp consistency, DTS-HD MA core-sync, and TrueHD DTS-monotonicity fixes 2026-06-26 22:16:45 -07:00
Matthew Jackson ced89133cc docs(fvi): standardize table headers and multi-line dense cells
Use 'Semantics / reference' as the last-column header across all spec
tables, and break enum/option/note cells onto separate lines with <br>
for readability. Formatting only; no semantic changes.
2026-06-26 21:42:31 -07:00
Matthew Jackson aefd6b6342 mux: single source of truth for the version label (VERSION_LABEL/MUX_APP)
The CLI --version, the MKV muxing/writing-application field, and the FVI generator
all derive from one libfreemkv const, so a binary reports the exact same label it
stamps into the files it produces — no split-brain where an MKV claims one version
and the binary another.
2026-06-26 21:33:19 -07:00
Matthew Jackson 78f78d285e docs(fvi): pretty-print JSON blocks in FVI format spec
Reformat all three JSON code blocks (section 8 ext example, Appendix A
Header schema, Appendix A Record schema) with consistent 2-space
indentation and valid syntax. No field or semantic changes.
2026-06-26 21:21:26 -07:00
Matthew Jackson 80ecb671fd mux: allow build-time version label override (FREEMKV_BUILD_LABEL)
build.rs emits FREEMKV_VERSION = FREEMKV_BUILD_LABEL when that env is set
(non-empty), else the Cargo package version. The muxing/writing-application
field and the FVI generator tag use it, so a pre-release/test build can be
stamped without bumping Cargo.toml and disturbing the tag-pinned [patch]
version matching. The git short hash is still appended either way.
2026-06-26 19:38:13 -07:00
Matthew Jackson c49a180ce7 mux: fix non-monotonic audio DTS (TrueHD + DTS-HD MA) and stamp builds with git hash
TrueHD: when the PES PTS lags the access-unit cadence, resync to the PTS
but never snap the running timestamp backward, so the emitted DTS stays
monotonic across the resync (next_pts_ns = max(next_pts_ns, pts)).

DTS-HD MA: size each EXSS extension substream exactly from its header
(exss_frame_size) and skip it as a unit, so a false 0x7FFE8001 core sync
inside the lossless extension payload can no longer split the access unit
and truncate the extension. Falls back to a bounded scan when the header
is unparseable.

Provenance: build.rs bakes the git short hash into GIT_SUFFIX; the muxing/
writing-application field and the FVI generator tag now record the exact
build (e.g. "freemkv 1.1.0-beta.1 (g835cc99)"), so any output file is
traceable to the revision that produced it.
2026-06-26 19:26:24 -07:00
Matthew Jackson afa218fc8f Remove keydb download/save from the library
keydb I/O moves out of libfreemkv into freemkv-keysources
(KeydbSource::save / ::update). Delete src/keydb.rs entirely (save,
http_get, default_path, write_atomic, UpdateResult) and drop `pub mod
keydb;` — http_get had no real callers. The shared Keydb* Error variants
stay in error.rs (keysources raises them; the every-error-has-a-code
contract depends on them). flate2 is no longer used here, so drop it
from Cargo.toml (zip stays for labels/jar.rs).

CHANGELOG: note the keydb-I/O move; reword the rc.5.2
DefaultDecodedFieldDuration entry to state only the action taken (the
revert) rather than an unverified Windows-fps outcome.
2026-06-26 17:33:57 -07:00
Matthew Jackson 835cc990ad DVD vob_start absolute rebase + rc.5.3 audit fixes
- ifo.rs: rebase VTS title VOBS to absolute disc LBA (file_start_lba +
  vtstt_vobs); fixes DVD rips opening on the menu region instead of the
  movie (e.g. SOTL). Adds absolute-placement regression test.
- aacs/boil.rs: add mk_from_pk primitive (PK -> MK via MKB walk).
- dvdnav/: nav-VM command decoder + start-cell resolver seam, parked
  behind USE_NAV_RESOLVER (kept compiled, never executed).
- mux: FVI src.byte within-sector per spec; Unknown colour -> CICP
  unspecified (2,2,2,1); demux clear PCS -> NORMAL; ts.rs feed() base
  reset + boundary provenance fix.
- Assorted audit fixes (doc/comment/test accuracy) across the crate.
2026-06-26 17:03:58 -07:00
Matthew Jackson d8c323bf9f Magic-number/taxonomy pass: central wire-format + sector + unit consts
- libfreemkv::consts: coding_type::* (ES coding-type bytes), pes_stream_id::*
  + PAYLOAD_RANGE, SECTOR_BYTES (usize) + SECTOR_BYTES_U64 (offset math)
- replace bare wire-code/sector literals across disc, mpls, clpi, labels,
  m2ts_mux, ps, tsmux, file_sector_source, extract
- remove two unreachable secondary-stream match arms in mpls parse_stream_entry
2026-06-26 13:20:21 -07:00
44 changed files with 1723 additions and 1187 deletions
+71 -28
View File
@@ -1,6 +1,6 @@
# Changelog # Changelog
## [1.0.0-rc.6] — UNRELEASED ## [1.1.0-beta.1] — UNRELEASED
### Added ### Added
@@ -11,15 +11,46 @@
all seven languages, and a Codes-page entry. Messages are source-agnostic all seven languages, and a Codes-page entry. Messages are source-agnostic
("key source", never a specific database). ("key source", never a specific database).
### Changed
- keydb download/save moved out of the library into freemkv-keysources;
libfreemkv no longer has any keydb I/O (it already held no keys).
### Fixed ### Fixed
- **Multi-extent AACS alignment in `dir://` extraction.** The decrypted - **DVD rips now start on the movie, not the disc menu.** A VTS title VOB's
file-tree extractor anchored the AACS unit grid to the first extent only, so a start sector was read from the IFO as a VTS-relative pointer but used as an
file spanning multiple clip extents could mis-align and corrupt. Each extent absolute disc address, so a DVD title's read extents began `ifo_lba` sectors
now anchors on its own encrypted-region start (the same class as the rc.5.2 too early — the rip opened on the disc's menu / VMGI region and only drifted
clip-anchor fix). Decryption math is unchanged. into the feature minutes later (Silence of the Lambs, for example, showed
- **Distinct "no key" reasons.** A disc whose key needs a Volume ID that several minutes of the main menu before the movie). The title VOB is now
couldn't be obtained now reports a distinct error from "no key at all". rebased to its absolute on-disc location, so the rip begins at the first frame
of the feature. Aspect ratio and chapter timing were already correct; only the
starting sector was wrong. (Covered by a new absolute-placement regression
test.)
- **Container metadata correctness.** Unknown colorimetry now emits the CICP
"unspecified" code point (2) consistently across the MKV track and the FVI
sidecar (previously 0); PGS subtitle wipes use the NORMAL composition state
rather than a full epoch reset; and FVI source-byte offsets are written
within-sector per the format spec.
- **Multi-extent AACS alignment in `dir://` extraction.** AACS encrypts in
aligned units of 3 sectors (6 KiB), and the decrypt-on-read gate accepts a read
only when its LBA is unit-aligned against a base. The `dir://` file-tree
extractor set that base once, to the file's first extent. A fragmented file
(Long-AD / continuation-ICB allocation) has later extents starting at arbitrary
LBAs whose distance from the first extent is generally not a multiple of 3
sectors, so the first read of every later extent failed the gate, returned a
decrypt error, and the whole extent was written as a zero-filled hole — even
though the sectors were readable. The unit base is now re-anchored per extent
(matching the mux read paths), so each extent gates on its own unit grid.
Decryption math is unchanged. Same class as the rc.5.2 clip-anchor fix.
- **Distinct "no key" reasons.** When AACS key resolution has usable material
(device or processing keys) but cannot obtain the disc's Volume ID — needed to
derive the unit key — freemkv now reports a distinct "AACS Volume ID
unavailable" error (E7017) instead of collapsing it into the generic "no key"
error (E7022), which is now reserved for a genuine absence of any key material.
No key derivation or descramble logic changed — only the reason reported on a
resolution failure.
- **autorip keydb writes go to the right path.** Auto-download, daily refresh, - **autorip keydb writes go to the right path.** Auto-download, daily refresh,
the "Update KEYDB" button, and the startup existence-check now resolve to the the "Update KEYDB" button, and the startup existence-check now resolve to the
service's config path (matching where reads look); they previously used the service's config path (matching where reads look); they previously used the
@@ -28,6 +59,22 @@
files and parent directories around rename). files and parent directories around rename).
- **Windows-reserved filenames** (`CON`, `NUL`, `COM1`…) inside a disc's file - **Windows-reserved filenames** (`CON`, `NUL`, `COM1`…) inside a disc's file
tree are safely renamed on extraction instead of aborting the walk. tree are safely renamed on extraction instead of aborting the walk.
- **`--version` now matches the build stamped into MKVs.** The CLI's `--version`
string and the `MuxingApp` / `WritingApp` fields written into every MKV now
derive from a single libfreemkv constant — the package version plus the git
short hash (e.g. `freemkv 1.1.0-beta.1 (g835cc99)`). The muxer previously kept
its own copy of that string, so the two could drift; a binary and the files it
produces can no longer report different versions.
- **DTS-HD Master Audio: a false core-sync inside the lossless extension no
longer splits an audio frame.** A byte pattern in the extension substream that
resembled the `0x7FFE8001` core sync word could truncate the lossless
extension and produce decode errors on the affected frames. The extension
substream is now sized exactly from its header, so that pattern is skipped as
data.
- **TrueHD: decode timestamps no longer step backward.** In a case where the
source PES timing lagged the audio access-unit cadence, the muxed decode
timestamp could regress (non-monotonic-DTS warnings to the muxer); the running
timestamp is now clamped so it never goes backward.
### Tests ### Tests
@@ -59,21 +106,16 @@
### Fixed ### Fixed
- **Windows Explorer now reports the full 25 fps for interlaced SD-DVD.** - **Reverted the rc.5.1 `DefaultDecodedFieldDuration` experiment for interlaced
rc.5.1 added a `DefaultDecodedFieldDuration` (20 ms field) element to the SD-DVD.** rc.5.1 added a 20 ms `DefaultDecodedFieldDuration` field element to
576i/480i track header on the theory that Windows derives fps from it. The the 576i/480i track header on the theory that Windows derives fps from it.
captured Silence-of-the-Lambs evidence proved the opposite: with Captured evidence showed that element made Windows Explorer report 12.5 fps
`FlagInterlaced=1` + `DefaultDuration=40 ms` + `DefaultDecodedFieldDuration=20 ms`, (half) and MediaInfo flip the track to "Frame rate mode: Variable", while
Windows Explorer reported 12.5 fps (half) and MediaInfo flipped the track to MakeMKV's rip of the same disc omits it. The element is therefore no longer
"Frame rate mode: Variable". MakeMKV's correct rip of the same disc OMITS written (`MkvTrack::video` now passes `field_duration_ns == 0`); the track
`DefaultDecodedFieldDuration`, keeps `FlagInterlaced=1` + `FieldOrder=TFF` + keeps `FlagInterlaced=1` + `FieldOrder=TFF` and the full-frame 40 ms
full-frame `DefaultDuration` (40 ms), and Explorer shows the full 25 fps with `DefaultDuration` (`1/DefaultDuration` = 25 fps), matching MakeMKV. How a given
MediaInfo "Constant". The element is no longer written (`MkvTrack::video` now player or shell handler chooses to display interlaced fps is not guaranteed.
passes `field_duration_ns == 0`); the only frame-rate signal tools trust,
`1/DefaultDuration` = 25 fps, is the full-frame value. Interlace signalling
(`FlagInterlaced=1`, `FieldOrder=TFF`) is retained, and MediaInfo still
reports "Interlaced / Top Field First" because it reads scan type from the
MPEG-2 elementary stream's picture coding extension, not the container flag.
- **Correct AC-3 audio track selected on DVDs with non-standard sub-stream - **Correct AC-3 audio track selected on DVDs with non-standard sub-stream
ordering.** freemkv assigned each declared audio stream a physical sub-stream ordering.** freemkv assigned each declared audio stream a physical sub-stream
by ordinal (`0x80+n`), assuming the IFO's first stream lives at `0x80`. On by ordinal (`0x80+n`), assuming the IFO's first stream lives at `0x80`. On
@@ -83,11 +125,12 @@
the disc — scanning every AC-3 frame and taking the maximum, so a brief 2.0 the disc — scanning every AC-3 frame and taking the maximum, so a brief 2.0
logo bed at the feature head can't mask the real 5.1 — and routes each declared logo bed at the feature head can't mask the real 5.1 — and routes each declared
stream onto the sub-stream that genuinely matches. stream onto the sub-stream that genuinely matches.
- **"Decryption failed" on large AACS Blu-ray titles fixed.** The unit-alignment - **"Decryption failed" on large AACS Blu-ray titles fixed.** AACS encrypts in
gate measured `lba % 3` against absolute disc LBA 0, but AACS aligned units are aligned units of 3 sectors (6 KiB); the unit-alignment gate measured `lba % 3`
anchored at each clip's encrypted-region start. A clip whose start is not against absolute disc LBA 0, but the unit grid is actually anchored at each
3-aligned had its readable units wrongly rejected — failing the feature/large clip's encrypted-region start. A clip whose start is not 3-sector-aligned had
titles of some discs while short clips passed. The gate is now clip-anchored. its readable units wrongly rejected — failing the feature/large titles of some
discs while short clips passed. The gate is now clip-anchored.
- **Single-pass disc→MKV recovers marginal/transient sectors before failing.** - **Single-pass disc→MKV recovers marginal/transient sectors before failing.**
The direct-to-MKV path now gives the drive its full ECC recovery budget on a The direct-to-MKV path now gives the drive its full ECC recovery budget on a
bad sector (matching the multipass rip) instead of reporting a read failure a bad sector (matching the multipass rip) instead of reporting a read failure a
-1
View File
@@ -22,7 +22,6 @@ sha1 = "0.10"
sha2 = "0.10" sha2 = "0.10"
aes = "0.8" aes = "0.8"
cbc = "0.1" cbc = "0.1"
flate2 = "1"
num-bigint = "0.4" num-bigint = "0.4"
num-traits = "0.2" num-traits = "0.2"
num-integer = "0.1" num-integer = "0.1"
+47
View File
@@ -1,4 +1,6 @@
fn main() { fn main() {
emit_git_suffix();
let target = std::env::var("CARGO_CFG_TARGET_OS").unwrap_or_default(); let target = std::env::var("CARGO_CFG_TARGET_OS").unwrap_or_default();
if target == "macos" { if target == "macos" {
println!("cargo:rustc-link-lib=framework=IOKit"); println!("cargo:rustc-link-lib=framework=IOKit");
@@ -48,3 +50,48 @@ fn main() {
println!("cargo:rerun-if-changed=src/scsi/macos_shim.c"); println!("cargo:rerun-if-changed=src/scsi/macos_shim.c");
} }
} }
/// Bake the git short hash into the build as `GIT_SUFFIX` so any muxed MKV or
/// FVI index is traceable to the exact source revision (e.g. ` (g835cc99)`).
/// Empty when git or the repo is unavailable (e.g. a crates.io tarball build),
/// leaving just the package version. Always emitted so `env!("GIT_SUFFIX")`
/// resolves on every target.
fn emit_git_suffix() {
// Version label for the muxing-app / FVI generator tag. `FREEMKV_BUILD_LABEL`
// overrides the Cargo package version when set (non-empty) — used to stamp a
// pre-release/test build without bumping Cargo.toml and disturbing the
// tag-pinned [patch] version matching. Unset → the package version.
let version = std::env::var("FREEMKV_BUILD_LABEL")
.ok()
.filter(|s| !s.trim().is_empty())
.or_else(|| std::env::var("CARGO_PKG_VERSION").ok())
.unwrap_or_default();
println!("cargo:rustc-env=FREEMKV_VERSION={version}");
println!("cargo:rerun-if-env-changed=FREEMKV_BUILD_LABEL");
let suffix = git_short_hash()
.map(|h| format!(" (g{h})"))
.unwrap_or_default();
println!("cargo:rustc-env=GIT_SUFFIX={suffix}");
// Re-run when HEAD (or the branch it points at) moves so the stamp stays
// current without a clean rebuild.
println!("cargo:rerun-if-changed=.git/HEAD");
if let Ok(head) = std::fs::read_to_string(".git/HEAD") {
if let Some(ref_path) = head.strip_prefix("ref: ") {
println!("cargo:rerun-if-changed=.git/{}", ref_path.trim());
}
}
}
fn git_short_hash() -> Option<String> {
let out = std::process::Command::new("git")
.args(["rev-parse", "--short=7", "HEAD"])
.output()
.ok()?;
if !out.status.success() {
return None;
}
let h = String::from_utf8(out.stdout).ok()?.trim().to_string();
if h.is_empty() { None } else { Some(h) }
}
+38 -26
View File
@@ -1,7 +1,8 @@
# FVI — Freemkv Video Index Format # FVI — Freemkv Video Index Format
**Specification version:** 1.0 (DRAFT) **Specification version:** 1.0 (DRAFT)\
**File extension:** `.fvi` **Media type:** `application/vnd.freemkv.fvi+jsonl` **File extension:** `.fvi`\
**Media type:** `application/vnd.freemkv.fvi+jsonl`\
**Status:** Draft for review. This document is the normative reference for the FVI **Status:** Draft for review. This document is the normative reference for the FVI
format; implementations and downstream tools cite it by section. format; implementations and downstream tools cite it by section.
@@ -25,13 +26,6 @@ FVI is **not** a container, a codec, or a copy of the bitstream. It indexes; it
never stores coded samples. It is the serialized form of an indexer's per-picture never stores coded samples. It is the serialized form of an indexer's per-picture
truth — carried from the demuxer, **never reconstructed** (§9). truth — carried from the demuxer, **never reconstructed** (§9).
### 1.1 Relationship to prior art
Legacy MPEG-only project-index formats from the AviSynth frameserving ecosystem
solve a narrow version of (1) and (2) for MPEG-1/2 only, in a bespoke,
single-tool text encoding. FVI generalizes that idea: codec-agnostic, JSON-based,
provenance-native, and openly specified so any tool may read or write it.
## 2. Conformance ## 2. Conformance
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**, The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**,
@@ -105,15 +99,15 @@ start of a new stream section.
| `width`,`height` | integer | MUST | Coded luma dimensions in pixels. | | `width`,`height` | integer | MUST | Coded luma dimensions in pixels. |
| `dar` | `[int,int]` | SHOULD | Display aspect ratio as `[num,den]`. | | `dar` | `[int,int]` | SHOULD | Display aspect ratio as `[num,den]`. |
| `frame_rate` | `[int,int]` | SHOULD | Nominal rate as exact rational `[num,den]` (e.g. `[24000,1001]`). | | `frame_rate` | `[int,int]` | SHOULD | Nominal rate as exact rational `[num,den]` (e.g. `[24000,1001]`). |
| `scan` | string | MUST | `"progressive"` \| `"interlaced"` \| `"mbaff"`. | | `scan` | string | MUST | `"progressive"`<br>`"interlaced"`<br>`"mbaff"` |
| `colour` | object | SHOULD | CICP per ITU-T H.273: `primaries`,`transfer`,`matrix` (integer CICP codes or registered names), `range` (`"limited"`\|`"full"`). HDR: `mastering_display`, `max_cll`, `max_fall` per ITU-T H.273 / SMPTE ST 2086. | | `colour` | object | SHOULD | CICP per ITU-T H.273: `primaries`, `transfer`, `matrix` (integer CICP codes or registered names)<br>`range`: `"limited"` \| `"full"`<br>HDR: `mastering_display`, `max_cll`, `max_fall` per ITU-T H.273 / SMPTE ST 2086. |
| `language` | string | MAY | BCP 47 tag, if known. | | `language` | string | MAY | BCP 47 tag, if known. |
### 6.2 `source` object ### 6.2 `source` object
| Member | JSON type | Req | Semantics | | Member | JSON type | Req | Semantics / reference |
|---|---|---|---| |---|---|---|---|
| `medium` | string | MUST | `"disc"` \| `"iso"` \| `"file"` \| `"stream"`. | | `medium` | string | MUST | `"disc"`<br>`"iso"`<br>`"file"`<br>`"stream"` |
| `path` | string | MAY | Source path/label. | | `path` | string | MAY | Source path/label. |
| `title` | integer | MAY | Title/program number. | | `title` | integer | MAY | Title/program number. |
| `playlist` | string | MAY | Playlist/PGC identifier. | | `playlist` | string | MAY | Playlist/PGC identifier. |
@@ -128,13 +122,13 @@ One JSON object per coded picture, in coded order.
|---|---|---|---| |---|---|---|---|
| `n` | integer | MUST | Coded-order index, 0-based, contiguous. | | `n` | integer | MUST | Coded-order index, 0-based, contiguous. |
| `src` | object | MUST | Provenance: `{ "file": int?, "sector": uint, "byte": uint }` — the offset of this AU's **first byte** in the source (§9). MUST be carried from demux, never reconstructed. | | `src` | object | MUST | Provenance: `{ "file": int?, "sector": uint, "byte": uint }` — the offset of this AU's **first byte** in the source (§9). MUST be carried from demux, never reconstructed. |
| `type` | string | MUST | Coding type: `"I"` \| `"P"` \| `"B"` (ISO/IEC 13818-2 §6.3.9; H.264/H.265 slice types collapsed to frame type). | | `type` | string | MUST | Coding type:<br>`"I"`<br>`"P"`<br>`"B"`<br>_ISO/IEC 13818-2 §6.3.9; H.264/H.265 slice types collapsed to frame type._ |
| `key` | boolean | MUST | `true` iff this picture is an intra (I) picture / parser-flagged decode-restart point (IDR / IRAP / I-picture). MPEG-2 open-GOP clean-RAP precision (`closed_gop`) is not currently distinguished — see note below. | | `key` | boolean | MUST | `true` iff this picture is an intra (I) picture / parser-flagged decode-restart point (IDR / IRAP / I-picture).<br>_MPEG-2 open-GOP clean-RAP precision (`closed_gop`) is not currently distinguished — see note below._ |
| `gop` | boolean | SHOULD | `true` iff this picture begins a GOP / coded video sequence. Omitted when the implementation does not carry a distinct GOP-boundary signal. | | `gop` | boolean | SHOULD | `true` iff this picture begins a GOP / coded video sequence.<br>_Omitted when the implementation does not carry a distinct GOP-boundary signal._ |
| `pts` | integer\|null | SHOULD | Presentation timestamp in `timescale` ticks; `null` if unknown. | | `pts` | integer\|null | SHOULD | Presentation timestamp in `timescale` ticks; `null` if unknown. |
| `dts` | integer\|null | MAY | Decode timestamp in `timescale` ticks. | | `dts` | integer\|null | MAY | Decode timestamp in `timescale` ticks. |
| `size` | integer | MAY | AU length in bytes; enables byte-range extraction with `src`. | | `size` | integer | MAY | AU length in bytes; enables byte-range extraction with `src`. |
| `recovered` | boolean | MAY | `true` iff any byte of this AU came from a retried/marginal read (§9.1). Default `false`. | | `recovered` | boolean | MAY | `true` iff any byte of this AU came from a retried/marginal read (§9.1).<br>_Default `false`._ |
| codec ext | object | MAY | Codec-specific members under the codec's namespace (§8). | | codec ext | object | MAY | Codec-specific members under the codec's namespace (§8). |
The `type` and `key` members are **codec-agnostic** and MUST be populated for The `type` and `key` members are **codec-agnostic** and MUST be populated for
@@ -162,11 +156,11 @@ per-picture coding accessors (MPEG-2: ISO/IEC 13818-2 §6.3.10). Emitted as
top-level members of the record, and ONLY when the codec actually measured the top-level members of the record, and ONLY when the codec actually measured the
signal — an OPTIONAL member that is omitted (not defaulted) when unknown: signal — an OPTIONAL member that is omitted (not defaulted) when unknown:
| Member | JSON type | Req | Semantics | | Member | JSON type | Req | Semantics / reference |
|---|---|---|---| |---|---|---|---|
| `field_order` | string | MAY | Display field order: `"tff"` (top field first) \| `"bff"` (bottom field first) \| `"progressive"` (no field order applies). Omitted when the codec did not signal it. | | `field_order` | string | MAY | Display field order:<br>`"tff"` top field first<br>`"bff"` bottom field first<br>`"progressive"` no field order applies<br>_Omitted when the codec did not signal it._ |
| `progressive` | boolean | MAY | `true` iff the picture is progressive. Omitted when the codec did not signal it. | | `progressive` | boolean | MAY | `true` iff the picture is progressive.<br>_Omitted when the codec did not signal it._ |
| `nb_fields` | integer | MAY | Number of displayed field periods this picture occupies (the soft-telecine / 2:3 pulldown basis): `1` for a single field picture, `2` for a normal frame, `3`/`4`/`6` for `repeat_first_field` pulldown per §6.3.10. | | `nb_fields` | integer | MAY | Number of displayed field periods this picture occupies (the soft-telecine / 2:3 pulldown basis):<br>`1` for a single field picture<br>`2` for a normal frame<br>`3`/`4`/`6` for `repeat_first_field` pulldown per §6.3.10 |
Codecs that carry only a coding type (e.g. H.264 / HEVC / VC-1 through this Codecs that carry only a coding type (e.g. H.264 / HEVC / VC-1 through this
pipeline) omit `field_order` and `progressive` rather than guessing a default. pipeline) omit `field_order` and `progressive` rather than guessing a default.
@@ -179,8 +173,21 @@ registered set per codec profile (e.g. MPEG-2 §7.1), or (b) placed under an
`ext` object keyed by codec id for richer/optional data: `ext` object keyed by codec id for richer/optional data:
```json ```json
{"n":42,"type":"P","key":false,"src":{"sector":17,"byte":924}, {
"ext":{"hevc":{"temporal_id":0,"nal_type":1}}} "n": 42,
"type": "P",
"key": false,
"src": {
"sector": 17,
"byte": 924
},
"ext": {
"hevc": {
"temporal_id": 0,
"nal_type": 1
}
}
}
``` ```
New codecs and members are added through Appendix B (codec registry) without a New codecs and members are added through Appendix B (codec registry) without a
@@ -260,10 +267,15 @@ Record:
"n": { "type": "integer", "minimum": 0 }, "n": { "type": "integer", "minimum": 0 },
"type": { "enum": ["I", "P", "B"] }, "type": { "enum": ["I", "P", "B"] },
"key": { "type": "boolean" }, "key": { "type": "boolean" },
"src": {"type":"object","required":["sector","byte"], "src": {
"properties":{"file":{"type":"integer"}, "type": "object",
"required": ["sector", "byte"],
"properties": {
"file": { "type": "integer" },
"sector": { "type": "integer", "minimum": 0 }, "sector": { "type": "integer", "minimum": 0 },
"byte":{"type":"integer","minimum":0}}} "byte": { "type": "integer", "minimum": 0 }
}
}
} }
} }
``` ```
+109 -1
View File
@@ -15,11 +15,17 @@
//! //!
//! ```text //! ```text
//! mk_from_dk(device_keys, mkb, vid) → MediaKey (Km) //! mk_from_dk(device_keys, mkb, vid) → MediaKey (Km)
//! mk_from_pk(processing_keys, mkb) → MediaKey (Km)
//! vuk_from_mk(MediaKey, Vid) → Vuk (= AES-G(Km, VID)) //! vuk_from_mk(MediaKey, Vid) → Vuk (= AES-G(Km, VID))
//! uk_from_vuk(Vuk, enc_title_keys) → [UnitKey] (decrypt_unit_key each) //! uk_from_vuk(Vuk, enc_title_keys) → [UnitKey] (decrypt_unit_key each)
//! ``` //! ```
//!
//! `mk_from_dk` and `mk_from_pk` are two entry points to the SAME Media Key:
//! the device-key path walks the MKB's Media-Key-Variant chain, the
//! processing-key path walks the MKB's Subset-Difference cvalue tables. Neither
//! needs a VID (the VID enters at `vuk_from_mk`).
use super::keys::{decrypt_unit_key, derive_vuk}; use super::keys::{decrypt_unit_key, derive_media_key_from_pk, derive_vuk};
use super::types::DeviceKey; use super::types::DeviceKey;
use super::variants::{KEY_CORRECTION_DATA_PLACEHOLDER, derive_media_key_variant, walk_mkb}; use super::variants::{KEY_CORRECTION_DATA_PLACEHOLDER, derive_media_key_variant, walk_mkb};
@@ -111,6 +117,31 @@ pub fn mk_from_dk(
} }
} }
/// Derive the Media Key (Km) from one or more Processing Keys and the disc MKB.
///
/// Wraps [`derive_media_key_from_pk`] — the Subset-Difference PK→MK walk: each
/// processing key is validated (and tree-walked) against the MKB's cvalue tables
/// (records `0x04`/`0x05`) until one yields the Media Key whose verify record
/// (`0x81`/`0x86`) matches. Unlike [`mk_from_dk`] this path is reachable for
/// real discs — a leaked/precomputed AACS Processing Key in the keydb resolves
/// the Media Key directly. No VID is involved at this step; the VID enters at
/// [`vuk_from_mk`].
///
/// Returns [`Error::AacsMkUnavailable`] (E7018) when no processing key resolves
/// the MKB — the same terminal error as [`mk_from_dk`]; no numeric distinction
/// is load-bearing at this boundary.
///
/// [`Error::AacsMkUnavailable`]: crate::error::Error::AacsMkUnavailable
pub fn mk_from_pk(
processing_keys: &[[u8; 16]],
mkb: &[u8],
) -> Result<MediaKey, crate::error::Error> {
match derive_media_key_from_pk(mkb, processing_keys) {
Some(km) => Ok(MediaKey(km)),
None => Err(crate::error::Error::AacsMkUnavailable),
}
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -197,4 +228,81 @@ mod tests {
let e2 = mk_from_dk(&[dk], &mkb, Vid([0x09; 16])); let e2 = mk_from_dk(&[dk], &mkb, Vid([0x09; 16]));
assert!(matches!(e2, Err(crate::error::Error::AacsMkUnavailable))); assert!(matches!(e2, Err(crate::error::Error::AacsMkUnavailable)));
} }
/// Build a 4-byte MKB record header (type + 3-byte big-endian total length,
/// header included) and append `body`. Mirrors the MKB record framing the
/// parser expects; no crypto.
fn mkb_record(rec_type: u8, body: &[u8]) -> Vec<u8> {
let total = 4 + body.len();
let mut rec = Vec::with_capacity(total);
rec.push(rec_type);
rec.push(((total >> 16) & 0xFF) as u8);
rec.push(((total >> 8) & 0xFF) as u8);
rec.push((total & 0xFF) as u8);
rec.extend_from_slice(body);
rec
}
/// `mk_from_pk` resolves a planted Processing Key against a synthetic MKB and
/// drives the FULL boil chain PK → MK → VUK → UK. The MKB is built with the
/// same (pk, cv, mk_dv, uv) construction the production SD walk validates, so
/// this proves a PK entry yields real Unit Keys — not just an `Ok`.
#[test]
fn mk_from_pk_drives_full_chain_to_uks() {
let pk: [u8; 16] = [
0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xAA, 0xBB, 0xCC, 0xDD, 0xEE,
0xFF, 0x00,
];
let mk: [u8; 16] = [
0xA0, 0xA1, 0xA2, 0xA3, 0xA4, 0xA5, 0xA6, 0xA7, 0xA8, 0xA9, 0xAA, 0xAB, 0xAC, 0xAD,
0xAE, 0xAF,
];
let uv: [u8; 4] = [0x00, 0x00, 0x04, 0x00];
// cv = AES-E(pk, mk_raw), where mk_raw is mk with the last-4-bytes-uv XOR
// pre-undone, so the validate step XORs uv back in and recovers mk.
let mut mk_raw = mk;
for a in 0..4 {
mk_raw[12 + a] ^= uv[a];
}
let cv = aes_ecb_encrypt(&pk, &mk_raw);
// mk_dv = AES-E(mk, magic||pad): AES-D(mk, mk_dv) starts with the AACS
// verify sentinel.
let mut vd = [0x11u8; 16];
vd[..8].copy_from_slice(&[0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF]);
let mk_dv = aes_ecb_encrypt(&mk, &vd);
// Synthetic MKB: type/version (0x10), verify record (0x86 = mk_dv),
// one-entry SD index (0x04 = [u_mask_shift=0][uv]), one-entry cvalue
// table (0x05 = cv).
let mut sd = vec![0u8];
sd.extend_from_slice(&uv);
let mut mkb = Vec::new();
mkb.extend_from_slice(&mkb_record(0x10, &[0, 0, 0, 0x20, 0, 0, 0, 0x52]));
mkb.extend_from_slice(&mkb_record(0x86, &mk_dv));
mkb.extend_from_slice(&mkb_record(0x04, &sd));
mkb.extend_from_slice(&mkb_record(0x05, &cv));
// PK → MK.
let got_mk = mk_from_pk(std::slice::from_ref(&pk), &mkb).expect("planted PK resolves MK");
assert_eq!(got_mk, MediaKey(mk), "mk_from_pk recovers the planted MK");
// MK → VUK → UK over an encrypted title key.
let vid = Vid([0x42u8; 16]);
let plain_uk = [0x7Eu8; 16];
let vuk = vuk_from_mk(got_mk, vid);
let enc = aes_ecb_encrypt(&vuk.0, &plain_uk);
let uks = uk_from_vuk(vuk, std::slice::from_ref(&enc));
assert_eq!(uks.len(), 1);
assert_eq!(uks[0].key, plain_uk, "PK chain recovers the title key");
// A corrupt PK resolves nothing.
let mut bad = pk;
bad[0] ^= 0xFF;
assert!(matches!(
mk_from_pk(std::slice::from_ref(&bad), &mkb),
Err(crate::error::Error::AacsMkUnavailable)
));
}
} }
+6 -1
View File
@@ -24,7 +24,7 @@ pub mod types;
pub mod variants; pub mod variants;
// Boil-down derivation primitives (thin newtypes + wrappers over the crypto). // Boil-down derivation primitives (thin newtypes + wrappers over the crypto).
pub use boil::{MediaKey, UnitKey, Vid, Vuk, mk_from_dk, uk_from_vuk, vuk_from_mk}; pub use boil::{MediaKey, UnitKey, Vid, Vuk, mk_from_dk, mk_from_pk, uk_from_vuk, vuk_from_mk};
// Structured, English-free resolution trace. // Structured, English-free resolution trace.
pub use trace::{KeyNode, KeyOutcome, KeyStep, ResolutionTrace, UnlockOutcome, UnlockStep}; pub use trace::{KeyNode, KeyOutcome, KeyStep, ResolutionTrace, UnlockOutcome, UnlockStep};
@@ -35,6 +35,10 @@ pub use decrypt::{
decrypt_unit_full, decrypt_unit_try_keys, is_aacs_scrambled, is_unit_aligned, ts_packet_total, decrypt_unit_full, decrypt_unit_try_keys, is_aacs_scrambled, is_unit_aligned, ts_packet_total,
ts_sync_count, unit_key_validates, ts_sync_count, unit_key_validates,
}; };
// `probe` is a reproduction-harness helper (see keys.rs), not part of the
// documented 1.0 surface; keep it reachable but off the rendered docs so we
// don't commit semver stability to test primitives.
#[doc(hidden)]
pub use keys::probe; pub use keys::probe;
pub use keys::{ pub use keys::{
AacsVersion, ContentCert, MKB_20_CATEGORY_C, MKB_21_CATEGORY_C, MKB_TYPE_3_RECORDABLE, AacsVersion, ContentCert, MKB_20_CATEGORY_C, MKB_21_CATEGORY_C, MKB_TYPE_3_RECORDABLE,
@@ -97,5 +101,6 @@ mod tests {
let _ = mkb_content_len(&[]); let _ = mkb_content_len(&[]);
let _ = is_variant_mkb(&walk_mkb(&[])); let _ = is_variant_mkb(&walk_mkb(&[]));
let _ = disc_hash_hex(&disc_hash(b"x")); let _ = disc_hash_hex(&disc_hash(b"x"));
let _ = mk_from_pk(&[[0u8; 16]], &[]);
} }
} }
+14 -14
View File
@@ -6,7 +6,7 @@
//! //!
//! Reference: https://github.com/lw/BluRay/wiki/CLPI //! Reference: https://github.com/lw/BluRay/wiki/CLPI
use crate::consts::{BD_SOURCE_PACKET_BYTES, SECTOR_BYTES}; use crate::consts::{BD_SOURCE_PACKET_BYTES, SECTOR_BYTES_U64};
use crate::disc::Extent; use crate::disc::Extent;
use crate::error::{Error, Result}; use crate::error::{Error, Result};
@@ -175,8 +175,8 @@ impl ClipInfo {
// a sub-sector-aligned range still spans every sector it touches. // a sub-sector-aligned range still spans every sector it touches.
let start_byte = start_spn as u64 * BD_SOURCE_PACKET_BYTES as u64; let start_byte = start_spn as u64 * BD_SOURCE_PACKET_BYTES as u64;
let end_byte = end_spn as u64 * BD_SOURCE_PACKET_BYTES as u64; let end_byte = end_spn as u64 * BD_SOURCE_PACKET_BYTES as u64;
let start_sector = (start_byte / SECTOR_BYTES as u64) as u32; let start_sector = (start_byte / SECTOR_BYTES_U64) as u32;
let end_sector = end_byte.div_ceil(SECTOR_BYTES as u64) as u32; let end_sector = end_byte.div_ceil(SECTOR_BYTES_U64) as u32;
vec![Extent { vec![Extent {
start_lba: start_sector, // relative to m2ts file start start_lba: start_sector, // relative to m2ts file start
@@ -261,6 +261,7 @@ pub fn parse(data: &[u8]) -> Result<ClipInfo> {
/// errors because the EP map is the primary CLPI output, and a corrupt /// errors because the EP map is the primary CLPI output, and a corrupt
/// program_info shouldn't break sector-range lookups. /// program_info shouldn't break sector-range lookups.
fn parse_program_info(data: &[u8]) -> Vec<ClpiStream> { fn parse_program_info(data: &[u8]) -> Vec<ClpiStream> {
use crate::consts::coding_type as c;
let mut out = Vec::new(); let mut out = Vec::new();
if data.len() < 6 { if data.len() < 6 {
return out; return out;
@@ -299,16 +300,15 @@ fn parse_program_info(data: &[u8]) -> Vec<ClpiStream> {
let mut language = String::new(); let mut language = String::new();
match coding_type { match coding_type {
// Video — MPEG-2 (0x02), H.264 (0x1B), HEVC (0x24) // Video — MPEG-2, H.264, HEVC
0x02 | 0x1B | 0x24 => { c::MPEG2_VIDEO | c::H264 | c::HEVC => {
if sci.len() >= 2 { if sci.len() >= 2 {
video_format = (sci[1] >> 4) & 0x0F; video_format = (sci[1] >> 4) & 0x0F;
video_rate = sci[1] & 0x0F; video_rate = sci[1] & 0x0F;
} }
} }
// Primary audio — LPCM(0x80), AC-3(0x81), DTS(0x82), // Primary audio — LPCM, AC-3, DTS, TrueHD, AC-3+, DTS-HD HR, DTS-HD MA
// TrueHD(0x83), AC-3+(0x84), DTS-HD(0x85), DTS-HD MA(0x86) c::LPCM..=c::DTS_HD_MA => {
0x80..=0x86 => {
if sci.len() >= 2 { if sci.len() >= 2 {
audio_format = (sci[1] >> 4) & 0x0F; audio_format = (sci[1] >> 4) & 0x0F;
audio_rate = sci[1] & 0x0F; audio_rate = sci[1] & 0x0F;
@@ -317,8 +317,8 @@ fn parse_program_info(data: &[u8]) -> Vec<ClpiStream> {
language = String::from_utf8_lossy(&sci[2..5]).to_string(); language = String::from_utf8_lossy(&sci[2..5]).to_string();
} }
} }
// Secondary audio (0xA1 AC-3+, 0xA2 DTS-HD) // Secondary audio (AC-3+ secondary, DTS-HD secondary)
0xA1 | 0xA2 => { c::AC3_PLUS_SECONDARY | c::DTS_HD_SECONDARY => {
if sci.len() >= 2 { if sci.len() >= 2 {
audio_format = (sci[1] >> 4) & 0x0F; audio_format = (sci[1] >> 4) & 0x0F;
audio_rate = sci[1] & 0x0F; audio_rate = sci[1] & 0x0F;
@@ -327,8 +327,8 @@ fn parse_program_info(data: &[u8]) -> Vec<ClpiStream> {
language = String::from_utf8_lossy(&sci[2..5]).to_string(); language = String::from_utf8_lossy(&sci[2..5]).to_string();
} }
} }
// PG (0x90), IG (0x91): coding_type + 3-byte language [+ char_code for PG] // PG, IG: coding_type + 3-byte language [+ char_code for PG]
0x90 | 0x91 => { c::PG | c::IG => {
if sci.len() >= 4 { if sci.len() >= 4 {
language = String::from_utf8_lossy(&sci[1..4]).to_string(); language = String::from_utf8_lossy(&sci[1..4]).to_string();
} }
@@ -1125,8 +1125,8 @@ mod tests {
let end_spn = big_spn as u64; let end_spn = big_spn as u64;
let start_byte = start_spn * BD_SOURCE_PACKET_BYTES as u64; let start_byte = start_spn * BD_SOURCE_PACKET_BYTES as u64;
let end_byte = end_spn * BD_SOURCE_PACKET_BYTES as u64; let end_byte = end_spn * BD_SOURCE_PACKET_BYTES as u64;
let start_sector = (start_byte / SECTOR_BYTES as u64) as u32; let start_sector = (start_byte / SECTOR_BYTES_U64) as u32;
let end_sector = end_byte.div_ceil(SECTOR_BYTES as u64) as u32; let end_sector = end_byte.div_ceil(SECTOR_BYTES_U64) as u32;
assert_eq!(extents[0].start_lba, start_sector); assert_eq!(extents[0].start_lba, start_sector);
assert_eq!(extents[0].sector_count, end_sector - start_sector); assert_eq!(extents[0].sector_count, end_sector - start_sector);
// Concretely: 0x20000 × 192 / 2048 = 12288 sectors. // Concretely: 0x20000 × 192 / 2048 = 12288 sectors.
+88
View File
@@ -8,8 +8,18 @@
/// Bytes per logical sector on every optical medium freemkv reads /// Bytes per logical sector on every optical medium freemkv reads
/// (Blu-ray, DVD-Video, CD-ROM Mode 1). Universal — hence unprefixed. /// (Blu-ray, DVD-Video, CD-ROM Mode 1). Universal — hence unprefixed.
///
/// `usize` because its dominant use is buffer sizing and slice indexing, where
/// Rust *requires* `usize` (`vec![0u8; SECTOR_BYTES]`, `buf.len() < SECTOR_BYTES`).
/// For byte-offset / capacity arithmetic — which is `u64` because a disc can
/// exceed 4 GiB — use [`SECTOR_BYTES_U64`] instead of casting at each site.
pub const SECTOR_BYTES: usize = 2048; pub const SECTOR_BYTES: usize = 2048;
/// [`SECTOR_BYTES`] as `u64`, for byte-offset and capacity arithmetic. The
/// single `usize → u64` boundary cast lives here, once, so offset math across
/// the workspace reads as `sectors * SECTOR_BYTES_U64` with no per-site cast.
pub const SECTOR_BYTES_U64: u64 = SECTOR_BYTES as u64;
/// Bytes per MPEG-2 transport-stream packet. Common to all MPEG-TS, not just /// Bytes per MPEG-2 transport-stream packet. Common to all MPEG-TS, not just
/// Blu-ray — prefixed by the format, not a disc type. /// Blu-ray — prefixed by the format, not a disc type.
pub const TS_PACKET_BYTES: usize = 188; pub const TS_PACKET_BYTES: usize = 188;
@@ -31,3 +41,81 @@ pub const TS_PAYLOAD_BYTES: usize = TS_PACKET_BYTES - TS_HEADER_BYTES;
/// prefixed with the [`BD_TIMESTAMP_PREFIX_BYTES`] arrival-timestamp header. /// prefixed with the [`BD_TIMESTAMP_PREFIX_BYTES`] arrival-timestamp header.
/// A BDAV/M2TS construct only — DVD VOBs have no source packets — hence `BD_`. /// A BDAV/M2TS construct only — DVD VOBs have no source packets — hence `BD_`.
pub const BD_SOURCE_PACKET_BYTES: usize = TS_PACKET_BYTES + BD_TIMESTAMP_PREFIX_BYTES; pub const BD_SOURCE_PACKET_BYTES: usize = TS_PACKET_BYTES + BD_TIMESTAMP_PREFIX_BYTES;
/// Elementary-stream coding-type codes — the single source of truth for the
/// byte that identifies a stream's codec.
///
/// This is one registry used in two places that share the same value space:
/// the MPEG-TS PMT `stream_type` (ISO/IEC 13818-1 Table 2-34) and the Blu-ray
/// STN/CLPI `stream_coding_type` (BD-ROM Part 3). The standardized video codes
/// (`0x02`, `0x1B`, `0x24`) are ISO assignments (ISO/IEC 13818-1 Table 2-34);
/// `0xEA` (VC-1) is a BD-ROM convention in the ISO user-private range. The
/// `0x80..=0xA2` audio/graphics codes also sit in the user-private range and follow the
/// Blu-ray Disc Association / ATSC A/52 convention. Because every consumer
/// reads or writes this single byte, the family is unprefixed — the scope is
/// "any elementary stream freemkv parses or muxes".
///
/// Each constant is `u8`: the spec defines an 8-bit field and the code compares
/// it directly against a byte read from the buffer, so no casts are needed.
pub mod coding_type {
/// MPEG-2 video (ISO/IEC 13818-1 Table 2-34).
pub const MPEG2_VIDEO: u8 = 0x02;
/// H.264 / AVC video (ISO/IEC 13818-1 Table 2-34).
pub const H264: u8 = 0x1B;
/// HEVC / H.265 video (ISO/IEC 13818-1 Table 2-34, 2015 amendment).
pub const HEVC: u8 = 0x24;
/// SMPTE VC-1 video (BD-ROM convention, ISO user-private range).
pub const VC1: u8 = 0xEA;
/// LPCM audio (BD-ROM convention).
pub const LPCM: u8 = 0x80;
/// Dolby Digital (AC-3) audio (BD-ROM / ATSC A/52 convention).
pub const AC3: u8 = 0x81;
/// DTS audio (BD-ROM convention).
pub const DTS: u8 = 0x82;
/// Dolby TrueHD audio (BD-ROM convention).
pub const TRUEHD: u8 = 0x83;
/// Dolby Digital Plus (E-AC-3 / AC-3+) audio (BD-ROM convention).
pub const AC3_PLUS: u8 = 0x84;
/// DTS-HD High Resolution audio (BD-ROM Part 3-1).
pub const DTS_HD_HR: u8 = 0x85;
/// DTS-HD Master Audio (lossless) (BD-ROM Part 3-1).
pub const DTS_HD_MA: u8 = 0x86;
/// Presentation Graphics — PG subtitle stream (BD-ROM HDMV).
pub const PG: u8 = 0x90;
/// Interactive Graphics — IG / BD-J menu overlay, NOT a subtitle (BD-ROM HDMV).
pub const IG: u8 = 0x91;
/// Text subtitle stream (BD-ROM HDMV).
pub const TEXT_SUBTITLE: u8 = 0x92;
/// Secondary Dolby Digital Plus audio (BD-ROM convention).
pub const AC3_PLUS_SECONDARY: u8 = 0xA1;
/// Secondary DTS-HD audio (lossless MA, not lossy HR) (BD-ROM convention).
pub const DTS_HD_SECONDARY: u8 = 0xA2;
}
/// MPEG PES `stream_id` codes — the byte after the `00 00 01` start-code prefix
/// that identifies an elementary stream's role in a PES packet (ISO/IEC
/// 13818-1 Table 2-22). Shared by the program-stream demuxer and the TS/M2TS
/// muxers, so defined here once. Each is `u8` (matches the byte on the wire).
pub mod pes_stream_id {
/// Video stream (`110x xxxx`; freemkv emits the base id `0xE0`).
pub const VIDEO: u8 = 0xE0;
/// private_stream_1 — AC-3 / DTS / LPCM / PGS subtitle payloads.
pub const PRIVATE_STREAM_1: u8 = 0xBD;
/// padding_stream — stuffing bytes only, no payload to demux.
pub const PADDING_STREAM: u8 = 0xBE;
/// private_stream_2 — DVD navigation (PCI/DSI); carries no muxable ES.
pub const PRIVATE_STREAM_2: u8 = 0xBF;
/// Highest video stream_id — the `110x xxxx` video range tops out at 0xEF.
pub const VIDEO_MAX: u8 = 0xEF;
/// Inclusive range of every PES `stream_id` that carries demuxable payload:
/// [`PRIVATE_STREAM_1`] (0xBD) through [`VIDEO_MAX`] (0xEF) — i.e. private
/// stream 1/2, padding, MPEG audio (0xC0-0xDF) and video (0xE0-0xEF). The
/// pack (0xBA), system-header (0xBB) and program-end (0xB9) codes sit below
/// this range and are deliberately excluded: they're structural, not ES.
pub const PAYLOAD_RANGE: core::ops::RangeInclusive<u8> = PRIVATE_STREAM_1..=VIDEO_MAX;
}
+2 -2
View File
@@ -402,7 +402,7 @@ mod tests {
#[test] #[test]
fn crack_unscrambled_returns_none() { fn crack_unscrambled_returns_none() {
let sector = vec![0u8; 2048]; let sector = vec![0u8; SECTOR_BYTES];
assert!(crack_title_key(&sector).is_none()); assert!(crack_title_key(&sector).is_none());
} }
@@ -414,7 +414,7 @@ mod tests {
#[test] #[test]
fn recover_needs_min_plain() { fn recover_needs_min_plain() {
let sector = vec![0u8; 2048]; let sector = vec![0u8; SECTOR_BYTES];
let short_plain = [0u8; 4]; let short_plain = [0u8; 4];
assert!(recover_title_key(&sector, &short_plain).is_none()); assert!(recover_title_key(&sector, &short_plain).is_none());
} }
+9 -2
View File
@@ -373,9 +373,16 @@ pub fn decrypt_sectors(
continue; continue;
} }
let crib = css::stevenson::attack_crib(chunk); let crib = css::stevenson::attack_crib(chunk);
let original: Option<Vec<u8>> = crib.as_ref().map(|_| chunk.to_vec()); // Snapshot the ciphertext into a stack buffer (chunk is exactly
// 2048 here — guaranteed by the `< 2048` continue above) only
// when there's a crib to validate against, so the common
// cache-hit path costs no per-sector heap allocation.
let mut original = [0u8; 2048];
if crib.is_some() {
original.copy_from_slice(chunk);
}
css::lfsr::descramble_sector(title_key, chunk); css::lfsr::descramble_sector(title_key, chunk);
if let (Some(crib), Some(original)) = (crib, original) { if let Some(crib) = crib {
if chunk[0x80..0x80 + 10] != crib[..] { if chunk[0x80..0x80 + 10] != crib[..] {
// Cached key is stale for this region — restore the // Cached key is stale for this region — restore the
// ciphertext and crack this sector's own key. // ciphertext and crack this sector's own key.
+119 -24
View File
@@ -106,7 +106,7 @@ impl Disc {
}) })
.collect(); .collect();
for dvd_title in &ts.titles { for (vts_title_idx, dvd_title) in ts.titles.iter().enumerate() {
title_number += 1; title_number += 1;
// Diagnostic dump (--log-level 3): per-cell category table + // Diagnostic dump (--log-level 3): per-cell category table +
@@ -114,13 +114,32 @@ impl Disc {
// per-cell IFO detail. No-op unless freemkv::diag is enabled. // per-cell IFO detail. No-op unless freemkv::diag is enabled.
crate::diag::dump_dvd_cells(ts.vts_number, title_number, dvd_title); crate::diag::dump_dvd_cells(ts.vts_number, title_number, dvd_title);
// Bug-4 leading-cell filter: drop any leading scene-index / // Feature start cell. Prefer the DVD nav-VM resolver, which
// interleaved-angle sub-block cells so the feature starts at the // PARKED (#40, menu-at-start playback). The "menu at the start"
// movie. Conservative — `feature_start_cell` only ever skips a // symptom (e.g. SOTL) was a sector-mapping fault — the absolute
// prefix of secondary-block cells and never truncates a normal // VOB rebase in `ifo::parse_vts` (`vob_start_sector =
// feature (category 0x00 on cell 0 → no-op). See // file_start_lba + vtstt_vobs`) — NOT a navigation problem, so
// `ifo::DvdTitle::feature_start_cell`. // feature-start resolution is unnecessary for correct rips. The
let feature_start = dvd_title.feature_start_cell(); // nav resolver + verified VM decoder (`dvdnav`) are kept compiled
// but deliberately bypassed; flip `USE_NAV_RESOLVER` to re-enable
// once the nav executor is finished. The fallback is the
// structural leading-cell filter (`feature_start_cell`), which
// drops leading scene-index / interleaved-angle sub-block cells
// and is a no-op for a normal feature (category 0x00 on cell 0).
// See `dvdnav::resolve_feature_start`.
const USE_NAV_RESOLVER: bool = false;
let feature_start = if USE_NAV_RESOLVER {
crate::dvdnav::resolve_feature_start(
reader,
udf_fs,
ts.vts_number as u16,
(vts_title_idx + 1) as u16,
)
.unwrap_or_else(|| dvd_title.feature_start_cell())
} else {
dvd_title.feature_start_cell()
}
.min(dvd_title.cells.len());
let dropped_secs: f64 = dvd_title.cells[..feature_start] let dropped_secs: f64 = dvd_title.cells[..feature_start]
.iter() .iter()
.map(|c| c.duration_secs) .map(|c| c.duration_secs)
@@ -136,9 +155,9 @@ impl Disc {
); );
} }
// Build extents from cell sector ranges (absolute = vob_start + cell offset) // Build extents from cell sector ranges (absolute = vob_start + cell offset),
let extents: Vec<Extent> = dvd_title // starting at the resolved feature-start cell.
.feature_cells() let extents: Vec<Extent> = dvd_title.cells[feature_start..]
.iter() .iter()
.map(|cell| { .map(|cell| {
let start = ts.vob_start_sector.saturating_add(cell.first_sector); let start = ts.vob_start_sector.saturating_add(cell.first_sector);
@@ -553,8 +572,10 @@ mod tests {
assert_eq!(titles.len(), 1); assert_eq!(titles.len(), 1);
let t = &titles[0]; let t = &titles[0];
assert_eq!(t.extents.len(), 1); assert_eq!(t.extents.len(), 1);
// absolute start = vob_start(1000) + first_sector(10) = 1010. // absolute start = ifo_lba + vtstt_vobs(1000) + first_sector(10).
assert_eq!(t.extents[0].start_lba, 1010); // The IFO file sits at PART_START(3000) + data_lba(6000) = 9000, so
// 9000 + 1000 + 10 = 10010.
assert_eq!(t.extents[0].start_lba, 10010);
// inclusive: 109 - 10 + 1 = 100 sectors. // inclusive: 109 - 10 + 1 = 100 sectors.
assert_eq!(t.extents[0].sector_count, 100); assert_eq!(t.extents[0].sector_count, 100);
// DVD sector = 2048 bytes. // DVD sector = 2048 bytes.
@@ -607,12 +628,85 @@ mod tests {
assert_eq!(titles.len(), 1); assert_eq!(titles.len(), 1);
let t = &titles[0]; let t = &titles[0];
assert_eq!(t.extents.len(), 1); assert_eq!(t.extents.len(), 1);
// Title VOBS (3640) + cell first_sector (0) = 3640 — NOT the menu 44. // ifo_lba(9000) + vtstt_vobs(3640) + first_sector(0) = 12640 — built
// from the Title VOBS (0xC4), NOT the menu VOBS (0xC0). The IFO file is
// at PART_START(3000) + data_lba(6000) = 9000.
assert_eq!( assert_eq!(
t.extents[0].start_lba, 12640,
"extent must start at ifo_lba + vtstt_vobs (0xC4), not vtsm_vobs (0xC0)"
);
// Must not resolve from the menu VOBS (would be 9000 + 44 = 9044), nor
// use the raw IFO-relative vtstt_vobs (3640) without the absolute base.
assert_ne!(t.extents[0].start_lba, 9044, "must not use the menu VOBS");
assert_ne!(
t.extents[0].start_lba, 3640, t.extents[0].start_lba, 3640,
"extent must start at vtstt_vobs (0xC4), not vtsm_vobs (0xC0)" "must add the IFO's absolute disc LBA, not use the raw relative value"
);
}
/// ABSOLUTE-REBASE regression (THESILENCEOFTHELAMBS / Greenland fix):
/// `ifo::parse_vts` now sets `vob_start_sector = file_start_lba(IFO) +
/// vtstt_vobs`, so an extent's `start_lba` must equal the sum of THREE
/// independent terms — the IFO file's absolute on-disc LBA, the
/// IFO-relative `vtstt_vobs` (0xC4), and the cell's `first_sector` — none of
/// which may be dropped. The earlier code used the bare relative
/// `vtstt_vobs`, placing every extent `ifo_lba` sectors too early (the rip
/// opened in the VMGI/menu region before drifting into the movie). The
/// other tests fold two of the three terms together (zero cell offset, or
/// a single combined expectation); this one keeps all three distinct and
/// non-overlapping so a regression to ANY two-term combination is caught.
#[test]
fn scan_dvd_titles_extent_is_absolute_three_term_sum() {
let mut disc = MemDisc::new();
let vmg = build_vmg(&[(1, 1, 1)]);
// vtstt_vobs (Title VOBS, 0xC4) = 700; one cell first_sector = 33.
let vts = build_vts(700, 0x00, &[], &[], &[(33, 132)], false);
// IFO data at data_lba 6000 → absolute ifo_lba = PART_START(3000) + 6000.
let ifo_lba = PART_START + 6000; // 9000
let vtstt_vobs = 700u32;
let first_sector = 33u32;
let udf = build_video_ts_fs(
&mut disc,
&[
FileSpec {
name: "VIDEO_TS.IFO".into(),
icb_lba: 60,
data_lba: 5000,
contents: vmg,
},
FileSpec {
name: "VTS_01_0.IFO".into(),
icb_lba: 62,
data_lba: 6000,
contents: vts,
},
],
);
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
assert_eq!(t.extents.len(), 1);
let got = t.extents[0].start_lba;
// The one correct answer: all three terms summed (9000 + 700 + 33).
assert_eq!(
got,
ifo_lba + vtstt_vobs + first_sector,
"extent start must be file_start_lba(IFO) + vtstt_vobs + cell.first_sector"
);
// Each wrong two-term combination must be rejected:
assert_ne!(
got,
vtstt_vobs + first_sector,
"must not use the bare relative vtstt_vobs (missing the IFO's absolute LBA)"
);
assert_ne!(
got,
ifo_lba + first_sector,
"must not drop vtstt_vobs (the Title VOBS pointer)"
);
assert_ne!(
got,
ifo_lba + vtstt_vobs,
"must not drop the cell's first_sector offset"
); );
assert_ne!(t.extents[0].start_lba, 44, "must not use the menu VOBS");
} }
/// Multi-cell title: extents preserve cell order and each maps to its /// Multi-cell title: extents preserve cell order and each maps to its
@@ -648,9 +742,9 @@ mod tests {
); );
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0]; let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
assert_eq!(t.extents.len(), 2); assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 500); // 500 + 0 assert_eq!(t.extents[0].start_lba, 9500); // ifo_lba(9000) + 500 + 0
assert_eq!(t.extents[0].sector_count, 100); assert_eq!(t.extents[0].sector_count, 100);
assert_eq!(t.extents[1].start_lba, 700); // 500 + 200 assert_eq!(t.extents[1].start_lba, 9700); // ifo_lba(9000) + 500 + 200
assert_eq!(t.extents[1].sector_count, 100); assert_eq!(t.extents[1].sector_count, 100);
assert_eq!(t.size_bytes, 200 * 2048); assert_eq!(t.size_bytes, 200 * 2048);
} }
@@ -1051,8 +1145,9 @@ mod tests {
assert_eq!(titles[0].playlist, "VTS_01_1.VOB"); assert_eq!(titles[0].playlist, "VTS_01_1.VOB");
assert_eq!(titles[1].playlist, "VTS_02_2.VOB"); assert_eq!(titles[1].playlist, "VTS_02_2.VOB");
// Distinct vob_start → distinct extents. // Distinct vob_start → distinct extents.
assert_eq!(titles[0].extents[0].start_lba, 100); // VTS_01 IFO @ PART_START(3000)+6000=9000; VTS_02 IFO @ 3000+7000=10000.
assert_eq!(titles[1].extents[0].start_lba, 200); assert_eq!(titles[0].extents[0].start_lba, 9100); // 9000 + 100
assert_eq!(titles[1].extents[0].start_lba, 10200); // 10000 + 200
} }
/// chapter_times from the IFO become Chapter entries with ordinal /// chapter_times from the IFO become Chapter entries with ordinal
@@ -1168,8 +1263,8 @@ mod tests {
// The leading 0x90 cell is dropped: 2 feature extents, not 3. // The leading 0x90 cell is dropped: 2 feature extents, not 3.
assert_eq!(t.extents.len(), 2, "leading angle sub-block cell dropped"); assert_eq!(t.extents.len(), 2, "leading angle sub-block cell dropped");
// First extent starts at the feature cell (vob 1000 + 100), not at 1000+0. // First extent starts at the feature cell (vob 1000 + 100), not at 1000+0.
assert_eq!(t.extents[0].start_lba, 1000 + 100); assert_eq!(t.extents[0].start_lba, 9000 + 1000 + 100); // ifo_lba + vtstt + first
assert_eq!(t.extents[1].start_lba, 1000 + 300); assert_eq!(t.extents[1].start_lba, 9000 + 1000 + 300);
// Chapter times shift earlier by the dropped 5s. Program 0 was at the // Chapter times shift earlier by the dropped 5s. Program 0 was at the
// dropped head (clamped to 0); program 1 was at cell 3 = // dropped head (clamped to 0); program 1 was at cell 3 =
// dur(cell0)+dur(cell1) = 5 + 59 = 64s, now 59s after the 5s shift. // dur(cell0)+dur(cell1) = 5 + 59 = 64s, now 59s after the 5s shift.
@@ -1219,8 +1314,8 @@ mod tests {
let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0]; let t = &Disc::scan_dvd_titles(&mut disc, &udf)[0];
// Nothing dropped: both cells become extents, starting at the very head. // Nothing dropped: both cells become extents, starting at the very head.
assert_eq!(t.extents.len(), 2); assert_eq!(t.extents.len(), 2);
assert_eq!(t.extents[0].start_lba, 1000); // 1000 + 0, head intact assert_eq!(t.extents[0].start_lba, 9000 + 1000); // ifo_lba + vtstt + 0, head intact
assert_eq!(t.extents[1].start_lba, 1200); assert_eq!(t.extents[1].start_lba, 9000 + 1200);
// Chapter 0 stays at 0.0 (no shift). // Chapter 0 stays at 0.0 (no shift).
assert!((t.chapters[0].time_secs - 0.0).abs() < 0.01); assert!((t.chapters[0].time_secs - 0.0).abs() < 0.01);
} }
+9 -6
View File
@@ -24,7 +24,7 @@ use std::io::Write;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::sync::atomic::Ordering; use std::sync::atomic::Ordering;
use crate::consts::SECTOR_BYTES; use crate::consts::{SECTOR_BYTES, SECTOR_BYTES_U64};
/// AACS aligned unit = 3 sectors / 6144 bytes. Content reads are issued in /// AACS aligned unit = 3 sectors / 6144 bytes. Content reads are issued in
/// multiples of this so the decrypt step always sees whole units. /// multiples of this so the decrypt step always sees whole units.
const AACS_UNIT_SECTORS: u32 = 3; const AACS_UNIT_SECTORS: u32 = 3;
@@ -234,9 +234,12 @@ impl Disc {
// Acquire (rather than Relaxed) on these per-file delta loads: // Acquire (rather than Relaxed) on these per-file delta loads:
// `extract_tree` drives `dec` single-threaded so there is no race // `extract_tree` drives `dec` single-threaded so there is no race
// today, but Acquire costs nothing on x86 and gives a happens- // today, and Acquire costs nothing on x86. Note this is only half
// before edge if file extraction is ever parallelised, so the // the synchronisation: the paired counter store
// delta can never read a torn/stale counter across iterations. // (sector/decrypting.rs `fetch_add`) is Relaxed, so an Acquire
// load alone does NOT yet establish a happens-before edge. Before
// file extraction is parallelised, upgrade that store to Release
// (or stronger) so the delta cannot read a stale counter.
let before_loss = decrypt_loss.load(Ordering::Acquire); let before_loss = decrypt_loss.load(Ordering::Acquire);
let (mut fr, halted) = let (mut fr, halted) =
extract_one_file(&mut dec, dest, pf, total_bytes, &mut done_bytes, opts)?; extract_one_file(&mut dec, dest, pf, total_bytes, &mut done_bytes, opts)?;
@@ -282,7 +285,7 @@ impl Disc {
for &(abs_lba, byte_len) in &pf.extents { for &(abs_lba, byte_len) in &pf.extents {
extents.push(crate::disc::Extent { extents.push(crate::disc::Extent {
start_lba: abs_lba, start_lba: abs_lba,
sector_count: (byte_len as u64).div_ceil(SECTOR_BYTES as u64) as u32, sector_count: (byte_len as u64).div_ceil(SECTOR_BYTES_U64) as u32,
}); });
} }
} }
@@ -467,7 +470,7 @@ fn extract_one_file<S: SectorSource>(
// the per-extent re-anchoring in the mux read paths // the per-extent re-anchoring in the mux read paths
// (`mux/disc.rs`, `sector/prefetched.rs`). No-op for CSS / None. // (`mux/disc.rs`, `sector/prefetched.rs`). No-op for CSS / None.
dec.set_unit_base(abs_lba); dec.set_unit_base(abs_lba);
let sectors = (byte_len as u64).div_ceil(SECTOR_BYTES as u64) as u32; let sectors = (byte_len as u64).div_ceil(SECTOR_BYTES_U64) as u32;
let mut sector_off: u32 = 0; let mut sector_off: u32 = 0;
while sector_off < sectors { while sector_off < sectors {
let mut batch = (sectors - sector_off).min(READ_BATCH_SECTORS); let mut batch = (sectors - sector_off).min(READ_BATCH_SECTORS);
+24 -23
View File
@@ -627,28 +627,29 @@ impl Codec {
]; ];
pub(crate) fn from_coding_type(ct: u8) -> Self { pub(crate) fn from_coding_type(ct: u8) -> Self {
use crate::consts::coding_type as c;
match ct { match ct {
0x24 => Codec::Hevc, c::HEVC => Codec::Hevc,
0x1B => Codec::H264, c::H264 => Codec::H264,
0xEA => Codec::Vc1, c::VC1 => Codec::Vc1,
0x02 => Codec::Mpeg2, c::MPEG2_VIDEO => Codec::Mpeg2,
0x83 => Codec::TrueHd, c::TRUEHD => Codec::TrueHd,
0x86 => Codec::DtsHdMa, c::DTS_HD_MA => Codec::DtsHdMa,
0x85 => Codec::DtsHdHr, c::DTS_HD_HR => Codec::DtsHdHr,
0x82 => Codec::Dts, c::DTS => Codec::Dts,
0x81 => Codec::Ac3, c::AC3 => Codec::Ac3,
0x84 | 0xA1 => Codec::Ac3Plus, c::AC3_PLUS | c::AC3_PLUS_SECONDARY => Codec::Ac3Plus,
0x80 => Codec::Lpcm, c::LPCM => Codec::Lpcm,
// 0x86 (primary) / 0xA2 (secondary) are the DTS-HD MA // DTS_HD_MA (primary 0x86) / DTS_HD_SECONDARY (0xA2) are the
// lossless pair, parallel to 0x81/0xA1 for AC-3. 0xA2 is // DTS-HD MA lossless pair, parallel to AC3/AC3_PLUS_SECONDARY for
// lossless MA, not lossy HR. // AC-3. The secondary code is lossless MA, not lossy HR.
0xA2 => Codec::DtsHdMa, c::DTS_HD_SECONDARY => Codec::DtsHdMa,
// 0x90 = Presentation Graphics (PG / subtitles). 0x91 = Interactive // PG (0x90) = Presentation Graphics (subtitles). IG (0x91, menus)
// Graphics (IG / menus) and 0x92 = Text subtitles are distinct HDMV // and TEXT_SUBTITLE (0x92) are distinct HDMV coding types and are
// coding types and are NOT PG subtitle streams; only 0x90 maps to // NOT PG subtitle streams; only PG maps to Pgs. IG falls through to
// Pgs. IG (0x91) falls through to Unknown so the PMT/STN walker drops // Unknown so the PMT/STN walker drops it rather than surfacing a
// it rather than surfacing a bogus PGS subtitle track for a menu ES. // bogus PGS subtitle track for a menu ES.
0x90 => Codec::Pgs, c::PG => Codec::Pgs,
ct => Codec::Unknown(ct), ct => Codec::Unknown(ct),
} }
} }
@@ -5068,8 +5069,8 @@ mod tests {
let mf = Mapfile::load(&disc.mapfile_for(&iso_path)).expect("load mapfile"); let mf = Mapfile::load(&disc.mapfile_for(&iso_path)).expect("load mapfile");
let good = mf.ranges_with(&[SectorStatus::Finished]); let good = mf.ranges_with(&[SectorStatus::Finished]);
let bad_ranges = mf.ranges_with(&[SectorStatus::NonTrimmed]); let bad_ranges = mf.ranges_with(&[SectorStatus::NonTrimmed]);
let disc_bytes = sectors as u64 * 2048; const SEC: u64 = crate::consts::SECTOR_BYTES_U64;
const SEC: u64 = crate::consts::SECTOR_BYTES as u64; let disc_bytes = sectors as u64 * SEC;
// The first failing batch starts at LBA 320; everything before it read // The first failing batch starts at LBA 320; everything before it read
// cleanly and must be Finished. // cleanly and must be Finished.
+12 -12
View File
@@ -825,12 +825,12 @@ pub(super) fn handle_read_success<R: SectorSource + ?Sized>(
state.stall_start = (state.now)(); state.stall_start = (state.now)();
state.bytes_good_last = bytes_good_now; state.bytes_good_last = bytes_good_now;
} }
if (state.now)().duration_since(state.stall_start) > std::time::Duration::from_secs(STALL_SECS) let stall_elapsed = (state.now)().duration_since(state.stall_start);
{ if stall_elapsed > std::time::Duration::from_secs(STALL_SECS) {
tracing::warn!( tracing::warn!(
target: "freemkv::disc", target: "freemkv::disc",
phase = "patch_stall", phase = "patch_stall",
elapsed_secs = (state.now)().duration_since(state.stall_start).as_secs(), elapsed_secs = stall_elapsed.as_secs(),
bytes_good = bytes_good_now, bytes_good = bytes_good_now,
bytes_good_start = state.bytes_good_start, bytes_good_start = state.bytes_good_start,
"Patch stalled - no recovery for {}s, exiting pass", "Patch stalled - no recovery for {}s, exiting pass",
@@ -1107,13 +1107,12 @@ pub(super) fn handle_read_failure<R: SectorSource + ?Sized>(
state.stall_start = (state.now)(); state.stall_start = (state.now)();
state.bytes_good_last = bytes_good_now; state.bytes_good_last = bytes_good_now;
} }
if (state.now)().duration_since(state.stall_start) let stall_elapsed = (state.now)().duration_since(state.stall_start);
> std::time::Duration::from_secs(STALL_SECS) if stall_elapsed > std::time::Duration::from_secs(STALL_SECS) {
{
tracing::warn!( tracing::warn!(
target: "freemkv::disc", target: "freemkv::disc",
phase = "patch_stall", phase = "patch_stall",
elapsed_secs = (state.now)().duration_since(state.stall_start).as_secs(), elapsed_secs = stall_elapsed.as_secs(),
bytes_good = bytes_good_now, bytes_good = bytes_good_now,
bytes_good_start = state.bytes_good_start, bytes_good_start = state.bytes_good_start,
"Patch stalled (NOT_READY path) - no recovery for {}s, exiting pass", "Patch stalled (NOT_READY path) - no recovery for {}s, exiting pass",
@@ -1193,12 +1192,12 @@ pub(super) fn handle_read_failure<R: SectorSource + ?Sized>(
state.stall_start = (state.now)(); state.stall_start = (state.now)();
state.bytes_good_last = bytes_good_now; state.bytes_good_last = bytes_good_now;
} }
if (state.now)().duration_since(state.stall_start) > std::time::Duration::from_secs(STALL_SECS) let stall_elapsed = (state.now)().duration_since(state.stall_start);
{ if stall_elapsed > std::time::Duration::from_secs(STALL_SECS) {
tracing::warn!( tracing::warn!(
target: "freemkv::disc", target: "freemkv::disc",
phase = "patch_stall", phase = "patch_stall",
elapsed_secs = (state.now)().duration_since(state.stall_start).as_secs(), elapsed_secs = stall_elapsed.as_secs(),
consecutive_failures = state.consecutive_failures, consecutive_failures = state.consecutive_failures,
bytes_good = bytes_good_now, bytes_good = bytes_good_now,
bytes_good_start = state.bytes_good_start, bytes_good_start = state.bytes_good_start,
@@ -1472,13 +1471,14 @@ pub(super) fn check_range_watchdog(
state.range_bytes_good = bytes_good_now; state.range_bytes_good = bytes_good_now;
state.range_start = (state.now)(); state.range_start = (state.now)();
} }
if (state.now)().duration_since(state.range_start).as_secs() >= frame.range_budget_secs { let range_elapsed = (state.now)().duration_since(state.range_start);
if range_elapsed.as_secs() >= frame.range_budget_secs {
tracing::warn!( tracing::warn!(
target: "freemkv::disc", target: "freemkv::disc",
phase = "patch_range_stall", phase = "patch_range_stall",
range_lba = frame.range_pos / 2048, range_lba = frame.range_pos / 2048,
range_sectors = frame.range_sectors, range_sectors = frame.range_sectors,
elapsed_secs = (state.now)().duration_since(state.range_start).as_secs(), elapsed_secs = range_elapsed.as_secs(),
budget_secs = frame.range_budget_secs, budget_secs = frame.range_budget_secs,
bytes_recovered = state.range_bytes_good.saturating_sub(state.bytes_good_before), bytes_recovered = state.range_bytes_good.saturating_sub(state.bytes_good_before),
"Range stalled - moving to next range" "Range stalled - moving to next range"
+5 -1
View File
@@ -433,8 +433,12 @@ impl Drive {
} }
Ok(()) Ok(())
} }
// No unlocker matched: not an error — fall through to OEM route. // No unlocker matched, or one matched but only hit a capability
// failure (not firmware-unlockable / no OEM VID): not an error —
// fall through to the OEM host-cert route.
Ok(None) => Ok(()), Ok(None) => Ok(()),
// A genuine transport fault during unlock (UnlockError::Scsi)
// propagates here and aborts init — the bus is dead.
Err(e) => Err(e), Err(e) => Err(e),
}; };
tracing::info!( tracing::info!(
+49
View File
@@ -0,0 +1,49 @@
//! DVD-Video navigation — read-only resolver for the **main-feature start
//! point** (issue #40). Mirrors what a DVD player's nav VM resolves: First-Play
//! → menu "Play" → title dispatch → the first cell of the feature, so the rip
//! starts at the movie rather than at raw cell 0 (e.g. skipping a leading
//! logo/warning segment when the disc's own navigation does).
//!
//! Byte layout follows the DVD-Video specification (VMGI/VTSI headers,
//! PGC/cell tables, PCI/HLI button packets); the VM command decoder is
//! verified against libdvdnav's decoder.
//!
//! Current contents: [`vmcmd`] — the VM command decoder (proven against the
//! SOTL/Greenland test discs). The IFO/PCI parsing and the navigation executor
//! that resolves the start cell build on top of this.
pub mod vmcmd;
use crate::sector::SectorSource;
/// Resolve the feature title's **true start cell** (0-based index into the
/// title PGC's cell list) by following the disc's own navigation — First-Play →
/// menu "Play" → title dispatch — the way a player reaches the movie. This is
/// what lets the rip begin at the feature instead of at raw cell 0 when the
/// disc's nav enters the title past a leading logo/warning segment (e.g. a
/// disc whose "Play" resolves to a later cell than cell 0).
///
/// Returns `None` when navigation cannot be resolved, so the caller falls back
/// to the structural leading-cell filter (today's behaviour, ≈ cell 0 / 0:00).
///
/// TODO(#40): the IFO/PCI parsing + nav executor (built on [`vmcmd`]) land
/// incrementally. Until the executor is complete this returns `None`, so wiring
/// it in is behaviour-neutral; improvements to the resolver take effect here
/// without touching the call site.
pub fn resolve_feature_start(
reader: &mut dyn SectorSource,
udf: &crate::udf::UdfFs,
vtsn: u16,
vts_ttn: u16,
) -> Option<usize> {
// `reader`/`udf` are the seam inputs the nav executor will consume to read
// VIDEO_TS.IFO + the VTS IFOs/menu VOBs. Reserved until that lands.
let _ = (reader, udf);
tracing::trace!(
target: "freemkv::dvdnav",
vtsn,
vts_ttn,
"nav start-cell resolver: unresolved — caller falls back to leading-cell filter"
);
None
}
+408
View File
@@ -0,0 +1,408 @@
//! DVD-Video VM command decoder.
//!
//! An 8-byte navigation command as found in PGC command tables (pre/post/cell)
//! and PCI button info. Decoded per the DVD-Video VM instruction set and
//! verified against libdvdnav's command decoder.
//!
//! Bit model: the 8 bytes are a big-endian 64-bit word. `byte0` bits 7-5 are the
//! command **type**; for type 1, `byte0` bit 4 selects Link (0) vs Jump (1), and
//! `byte1` bits 3-0 are the sub-command. Compare predicates live in `byte1`
//! bits 6-4 with the operands in bytes 2-5.
//!
//! This module is pure decode + a register model — no I/O, no English (numeric
//! semantics only), matching libfreemkv conventions. The navigation *executor*
//! and IFO/PCI parsing build on top of this.
/// A decoded navigation instruction. Only the variants freemkv's start-point
/// resolver needs are modelled explicitly; everything else is [`Instr::Other`].
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Instr {
Nop,
/// Stop executing the current command list (resume cell playback).
Break,
/// Goto command line within the same list (1-based).
Goto {
line: u8,
},
/// Leave the current domain.
Exit,
/// Jump to a VMG title (1-based TT_SRPT index).
JumpTt {
ttn: u8,
},
/// Jump to a title within the current VTS (1-based VTS title index).
JumpVtsTt {
ttn: u8,
},
/// Jump to a part-of-title (chapter) within a VTS title.
JumpVtsPtt {
ttn: u8,
pttn: u16,
},
/// Jump to the First-Play PGC.
JumpSsFp,
/// Jump to a Video-Manager menu (`menu` = menu id).
JumpSsVmgm {
menu: u8,
},
/// Jump to a Video-Title-Set menu.
JumpSsVtsm {
vts: u8,
ttn: u8,
menu: u8,
},
/// Jump to a specific VMGM menu PGC.
JumpSsVmgmPgc {
pgcn: u16,
},
/// Call a sub-domain (raw retained; resume handled by the executor).
CallSs {
sub: u8,
},
/// Link to a PGC number within the current domain.
LinkPgcn {
pgcn: u16,
},
/// Link to a part-of-title within the current PGC's title.
LinkPttn {
pttn: u16,
},
/// Link to a program number within the current PGC (1-based).
LinkPgn {
pgn: u8,
},
/// Link to a cell number within the current PGC (1-based).
LinkCn {
cn: u8,
},
/// A link "subset" op (LinkTopCell/NextPG/RSM/…); `sub` is the raw code.
LinkSub {
sub: u8,
},
/// Set a GPRM. `op` is the set-op code (1=mov, 3=add, …); value is immediate
/// (`imm`) when `immediate`, else the contents of register `src`.
SetGprm {
reg: u8,
op: u8,
immediate: bool,
imm: u16,
src: u8,
},
/// Set a system parameter / unmodelled set — executor may ignore.
SetSystem,
/// Anything not individually modelled (kept as raw bytes).
Other([u8; 8]),
}
/// A compare predicate carried by a command (`byte1` bits 6-4). `None` = always.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Compare {
/// Compare op: 1=&,2===,3=!=,4=>=,5=>,6=<=,7=<.
pub op: u8,
/// Left register index (GPRM 0-15, SPRM 128+).
pub lhs_reg: u8,
/// Right side: immediate when `immediate`, else register `rhs_reg`.
pub immediate: bool,
pub imm: u16,
pub rhs_reg: u8,
}
/// A fully decoded command: its predicate (if any) and the instruction.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Command {
pub compare: Option<Compare>,
pub instr: Instr,
}
// Command types — `byte0` bits 7-5.
const TYPE_SPECIAL: u8 = 0;
const TYPE_LINK_JUMP: u8 = 1;
const TYPE_SET_SYSTEM: u8 = 2;
const TYPE_SET_GPRM: u8 = 3;
// Special (type 0) sub-commands — `byte1` bits 3-0.
const SP_GOTO: u8 = 1;
const SP_BREAK: u8 = 2;
// Jump/Call (type 1, direct=1) sub-commands.
const JP_EXIT: u8 = 1;
const JP_JUMP_TT: u8 = 2;
const JP_JUMP_VTS_TT: u8 = 3;
const JP_JUMP_VTS_PTT: u8 = 5;
const JP_JUMP_SS: u8 = 6;
const JP_CALL_SS: u8 = 8;
// Link (type 1, direct=0) sub-commands. NOTE: sub-op 0 is NOP/no-link and 1 is
// the LinkSub form (libdvdnav `decoder.c` `eval_link_instruction`).
const LK_SUB: u8 = 1;
const LK_PGCN: u8 = 4;
const LK_PTTN: u8 = 5;
const LK_PGN: u8 = 6;
const LK_CN: u8 = 7;
// JumpSS sub-domain selector — `byte5` bits 7-6.
const SS_FP: u8 = 0;
const SS_VMGM_MENU: u8 = 1;
const SS_VTSM: u8 = 2;
// Operand field widths (spec-defined bit counts).
const MASK_TTN: u8 = 0x7F; // 7-bit title number
const MASK_PGN: u8 = 0x7F; // 7-bit program number
const MASK_LINKOP: u8 = 0x1F; // 5-bit link sub-op
const MASK_REG: u8 = 0x0F; // 4-bit GPRM index
const MASK_MENU: u8 = 0x0F; // 4-bit menu id
const MASK_PTTN: u16 = 0x03FF; // 10-bit part-of-title
const MASK_PGCN: u16 = 0x7FFF; // 15-bit PGC number
#[inline]
fn be16(b: &[u8; 8], o: usize) -> u16 {
((b[o] as u16) << 8) | b[o + 1] as u16
}
// Compare-operand layouts ("if_version"s) per libdvdnav `decoder.c`. The op
// nibble is always `byte1` bits 6-4; the immediate flag is `byte1` bit 7. The
// operand *offsets* differ by command family.
//
// v1 (special + link): lhs reg = b[3]; rhs imm = bytes4-5 / rhs reg = b[4].
// v2 (jump + system-set): lhs reg = b[6]; rhs reg = b[7] (registers only).
// v3 (set-GPRM): lhs reg = b[2]; rhs imm = bytes6-7 / rhs reg = b[6].
fn if_v1(b: &[u8; 8]) -> Option<Compare> {
let op = (b[1] >> 4) & 7;
(op != 0).then(|| Compare {
op,
lhs_reg: b[3],
immediate: b[1] >> 7 != 0,
imm: be16(b, 4),
rhs_reg: b[4],
})
}
fn if_v2(b: &[u8; 8]) -> Option<Compare> {
let op = (b[1] >> 4) & 7;
(op != 0).then(|| Compare {
op,
lhs_reg: b[6],
immediate: false,
imm: 0,
rhs_reg: b[7],
})
}
fn if_v3(b: &[u8; 8]) -> Option<Compare> {
let op = (b[1] >> 4) & 7;
(op != 0).then(|| Compare {
op,
lhs_reg: b[2],
immediate: b[1] >> 7 != 0,
imm: be16(b, 6),
rhs_reg: b[6],
})
}
/// Decode an 8-byte VM command.
pub fn decode(b: &[u8; 8]) -> Command {
let typ = b[0] >> 5;
let direct = (b[0] >> 4) & 1;
let setop = b[0] & 0x0F;
let cmd = b[1] & 0x0F;
// Compare predicate, with the operand layout for this command family
// (libdvdnav `decoder.c` `vm_eval_command` type dispatch).
let compare = match (typ, direct) {
(TYPE_SPECIAL, _) => if_v1(b),
(TYPE_LINK_JUMP, 1) => if_v2(b), // jump
(TYPE_LINK_JUMP, 0) => if_v1(b), // link
(TYPE_SET_SYSTEM, _) => if_v2(b),
(TYPE_SET_GPRM, _) => if_v3(b),
_ => None, // 4/5/6 compound — not needed by the resolver
};
// JumpSS sub-domain selector lives in byte5 bits 7-6.
let ss_sel = b[5] >> 6;
let instr = match typ {
TYPE_LINK_JUMP if direct == 1 => match cmd {
JP_EXIT => Instr::Exit,
JP_JUMP_TT => Instr::JumpTt {
ttn: b[5] & MASK_TTN,
},
JP_JUMP_VTS_TT => Instr::JumpVtsTt {
ttn: b[5] & MASK_TTN,
},
JP_JUMP_VTS_PTT => Instr::JumpVtsPtt {
ttn: b[5] & MASK_TTN,
pttn: be16(b, 2) & MASK_PTTN,
},
JP_JUMP_SS => match ss_sel {
SS_FP => Instr::JumpSsFp,
SS_VMGM_MENU => Instr::JumpSsVmgm {
menu: b[5] & MASK_MENU,
},
SS_VTSM => Instr::JumpSsVtsm {
vts: b[4],
ttn: b[3],
menu: b[5] & MASK_MENU,
},
_ => Instr::JumpSsVmgmPgc {
pgcn: be16(b, 2) & MASK_PGCN,
},
},
JP_CALL_SS => Instr::CallSs { sub: ss_sel },
_ => Instr::Nop,
},
TYPE_LINK_JUMP => match cmd {
// direct == 0 (link). sub-op 0 = NOP/no-link.
LK_SUB => Instr::LinkSub {
sub: b[7] & MASK_LINKOP,
},
LK_PGCN => Instr::LinkPgcn {
pgcn: be16(b, 6) & MASK_PGCN,
},
LK_PTTN => Instr::LinkPttn {
pttn: be16(b, 6) & MASK_PTTN,
},
LK_PGN => Instr::LinkPgn {
pgn: b[7] & MASK_PGN,
},
LK_CN => Instr::LinkCn { cn: b[7] },
_ => Instr::Nop,
},
TYPE_SPECIAL => match cmd {
SP_GOTO => Instr::Goto { line: b[7] },
SP_BREAK => Instr::Break,
_ => Instr::Nop,
},
TYPE_SET_GPRM => Instr::SetGprm {
reg: b[3] & MASK_REG,
op: setop,
immediate: direct != 0,
imm: be16(b, 4),
src: b[5],
},
TYPE_SET_SYSTEM => Instr::SetSystem,
_ => Instr::Other(*b),
};
Command { compare, instr }
}
#[cfg(test)]
mod tests {
use super::*;
fn h(s: &str) -> [u8; 8] {
let v: Vec<u8> = (0..8)
.map(|i| u8::from_str_radix(&s[i * 2..i * 2 + 2], 16).unwrap())
.collect();
v.try_into().unwrap()
}
// KATs taken from the real SOTL / Greenland discs (decoded in the PoC).
#[test]
fn greenland_first_play_is_jumptt_1() {
let c = decode(&h("3002000000010000"));
assert_eq!(c.instr, Instr::JumpTt { ttn: 1 });
assert!(c.compare.is_none());
}
#[test]
fn sotl_first_play_is_jumpss_vtsm_root() {
// 30 06 ... byte5=0x83 -> sub 2 (VTSM), vts=byte4=1, menu=byte5&0xF=3 (root)
let c = decode(&h("3006000101830000"));
assert_eq!(
c.instr,
Instr::JumpSsVtsm {
vts: 1,
ttn: 1,
menu: 3
}
);
}
#[test]
fn sotl_title_dispatch_is_conditional_linkpgn_2() {
// 20 a6 ... CmpLink: if GPRM0 == 2 -> LinkPGN 2 (cell 2 = the 5:02 start)
let c = decode(&h("20a6000000020002"));
assert_eq!(c.instr, Instr::LinkPgn { pgn: 2 });
let cmp = c.compare.expect("conditional");
assert_eq!(cmp.op, 2); // ==
assert_eq!(cmp.lhs_reg, 0); // GPRM0
assert!(cmp.immediate);
assert_eq!(cmp.imm, 2);
}
#[test]
fn sotl_root_button_is_linkpgcn_37() {
assert_eq!(
decode(&h("2004000000000025")).instr,
Instr::LinkPgcn { pgcn: 37 }
);
}
#[test]
fn greenland_scene_button_is_linkpgn() {
assert_eq!(
decode(&h("2006000000001401")).instr,
Instr::LinkPgn { pgn: 1 }
);
}
#[test]
fn jumpvts_ptt_decodes_ttn_and_pttn() {
// synthetic: 30 05 | ptt(bytes2-3)=0x0002 | ttn(byte5)=1
let c = decode(&h("3005000200010000"));
assert_eq!(c.instr, Instr::JumpVtsPtt { ttn: 1, pttn: 2 });
}
#[test]
fn setgprm_immediate_mov() {
// SOTL First-Play pre[0]: 71 00 | reg=byte3=6 | imm(bytes4-5)=0x03e8 -> g6 = 1000
match decode(&h("7100000603e80000")).instr {
Instr::SetGprm {
reg,
op,
immediate,
imm,
..
} => {
assert_eq!(reg, 6);
assert_eq!(op, 1); // mov
assert!(immediate);
assert_eq!(imm, 1000);
}
other => panic!("expected SetGprm, got {other:?}"),
}
}
// Regression for the libdvdnav cross-check: link sub-op 0 = NOP, 1 = LinkSub.
#[test]
fn link_subop_zero_is_nop_one_is_linksub() {
assert_eq!(decode(&h("2000000000000000")).instr, Instr::Nop);
assert_eq!(
decode(&h("2001000000000010")).instr,
Instr::LinkSub { sub: 0x10 }
);
}
// if_version_1 register compare: rhs register is byte4 (not byte5).
#[test]
fn link_register_compare_rhs_is_byte4() {
// 20 26: link, cmp=EQ(2), dircmp=0(register) ; cmd=6 LinkPGN
let c = decode(&h("2026000304000002"));
assert_eq!(c.instr, Instr::LinkPgn { pgn: 2 });
let cmp = c.compare.expect("conditional");
assert!(!cmp.immediate);
assert_eq!(cmp.lhs_reg, 3);
assert_eq!(cmp.rhs_reg, 4);
}
// if_version_2 jump compare: both operands are registers in byte6 / byte7.
#[test]
fn jump_compare_uses_bytes6_and_7() {
// 30 22: jump, cmp=EQ(2) ; cmd=2 JumpTT ttn=byte5=5
let c = decode(&h("3022000000050607"));
assert_eq!(c.instr, Instr::JumpTt { ttn: 5 });
let cmp = c.compare.expect("conditional");
assert!(!cmp.immediate);
assert_eq!(cmp.lhs_reg, 6);
assert_eq!(cmp.rhs_reg, 7);
}
}
+1
View File
@@ -1199,6 +1199,7 @@ mod tests {
E_NO_DISC_KEY, E_NO_DISC_KEY,
E_CSS_KEY_MISSING, E_CSS_KEY_MISSING,
E_AACS_NO_HOST_CERT, E_AACS_NO_HOST_CERT,
E_AACS_BUS_KEY_UNAVAILABLE,
E_KEYDB_CONNECT, E_KEYDB_CONNECT,
E_KEYDB_HTTP, E_KEYDB_HTTP,
E_KEYDB_INVALID, E_KEYDB_INVALID,
+13 -1
View File
@@ -440,7 +440,19 @@ fn parse_vts(
// to the feature and shifted every cell extent back by // to the feature and shifted every cell extent back by
// `vtstt_vobs - vtsm_vobs` sectors, so the rip opened on the parental // `vtstt_vobs - vtsm_vobs` sectors, so the rip opened on the parental
// prompt instead of the movie. The title content lives at `vtstt_vobs`. // prompt instead of the movie. The title content lives at `vtstt_vobs`.
let vob_start_sector = be_u32(&vts_data, VTSTT_VOBS_OFFSET)?; //
// `vtstt_vobs` is a sector address **relative to the start of this VTS_xx_0.IFO
// file**, not an absolute disc LBA. The cell `first_sector`/`last_sector`
// values are in turn relative to `vtstt_vobs`. To turn them into the absolute
// disc LBAs the reader needs, add the IFO file's own on-disc location (from
// the UDF FS). Without this rebase every extent started `ifo_lba` sectors too
// early — for THESILENCEOFTHELAMBS the feature began at LBA 126 (the VMGI /
// VIDEO_TS.VOB main-menu region) instead of 132886 (VTS_03_1.VOB), so the
// first ~4.5 min of muxed video was the disc's main menu before the stream
// drifted into the movie.
let vtstt_vobs = be_u32(&vts_data, VTSTT_VOBS_OFFSET)?;
let ifo_lba = udf.file_start_lba(reader, &path)?;
let vob_start_sector = ifo_lba.saturating_add(vtstt_vobs);
// Video attributes at offset 0x200 (2 bytes) // Video attributes at offset 0x200 (2 bytes)
let video = parse_video_attr(&vts_data)?; let video = parse_video_attr(&vts_data)?;
+4 -4
View File
@@ -72,7 +72,7 @@ use std::path::Path;
use crate::error::{Error, Result}; use crate::error::{Error, Result};
use crate::sector::SectorSource; use crate::sector::SectorSource;
use crate::consts::SECTOR_BYTES; use crate::consts::{SECTOR_BYTES, SECTOR_BYTES_U64};
/// Bytes-read threshold per `posix_fadvise(DONTNEED)` drop on the /// Bytes-read threshold per `posix_fadvise(DONTNEED)` drop on the
/// read side. Mirrors `WRITEBACK_CHUNK_BYTES` so the read-side page /// read side. Mirrors `WRITEBACK_CHUNK_BYTES` so the read-side page
@@ -134,7 +134,7 @@ impl FileSectorSource {
.metadata() .metadata()
.map_err(|e| Error::IoError { source: e })? .map_err(|e| Error::IoError { source: e })?
.len(); .len();
let sectors = len / SECTOR_BYTES as u64; let sectors = len / SECTOR_BYTES_U64;
if sectors > u32::MAX as u64 { if sectors > u32::MAX as u64 {
return Err(Error::IsoTooLarge { return Err(Error::IsoTooLarge {
path: path.to_string_lossy().into_owned(), path: path.to_string_lossy().into_owned(),
@@ -180,7 +180,7 @@ impl SectorSource for FileSectorSource {
if count == 0 { if count == 0 {
return Ok(0); return Ok(0);
} }
let offset = lba as u64 * SECTOR_BYTES as u64; let offset = lba as u64 * SECTOR_BYTES_U64;
self.file self.file
.seek(SeekFrom::Start(offset)) .seek(SeekFrom::Start(offset))
.map_err(|e| Error::IoError { source: e })?; .map_err(|e| Error::IoError { source: e })?;
@@ -494,7 +494,7 @@ mod tests {
#[test] #[test]
fn dontneed_eviction_does_not_affect_data() { fn dontneed_eviction_does_not_affect_data() {
// 32 MiB default chunk = 16384 sectors; read a bit past it. // 32 MiB default chunk = 16384 sectors; read a bit past it.
let total = (READ_DROP_CHUNK_BYTES_DEFAULT / SECTOR_BYTES as u64) as u32 + 64; let total = (READ_DROP_CHUNK_BYTES_DEFAULT / SECTOR_BYTES_U64) as u32 + 64;
let dir = tempdir().unwrap(); let dir = tempdir().unwrap();
let path = dir.path().join("drop.iso"); let path = dir.path().join("drop.iso");
make_iso(&path, total); make_iso(&path, total);
-869
View File
@@ -1,869 +0,0 @@
//! KEYDB.cfg updater — HTTP GET, unzip, verify, save.
//!
//! Zero external HTTP dependencies. Raw TCP for HTTP GET.
//! Uses `zip` and `flate2` (already in deps) for extraction.
use crate::error::{Error, Result};
use std::io::{Read, Write};
use std::net::{TcpStream, ToSocketAddrs};
use std::path::PathBuf;
use std::time::Duration;
/// Network operation timeout (connect / read / write). Keeps the daily
/// refresh thread from blocking indefinitely on an unresponsive mirror.
const NET_TIMEOUT: Duration = Duration::from_secs(10);
/// Read timeout — longer than connect/write since the keydb body can be
/// several MiB over a slow link.
const READ_TIMEOUT: Duration = Duration::from_secs(30);
/// Maximum redirects to follow before giving up.
const MAX_REDIRECTS: usize = 5;
/// Upper bound on decompressed keydb size. The published keydb is a few
/// MiB; 64 MiB is a generous ceiling that still caps a decompression
/// bomb (a tiny zip/gz can otherwise inflate to GiB and OOM the daily
/// refresh thread).
const MAX_KEYDB_BYTES: u64 = 64 * 1024 * 1024;
/// Read a decompressed stream into a String with a hard size ceiling.
/// Returns `Error::KeydbInvalid` if the input exceeds the cap, or
/// `Error::KeydbParse` if the bytes are not valid UTF-8.
fn read_capped_to_string<R: Read>(reader: R) -> Result<String> {
let mut buf = Vec::new();
// Read one byte past the cap so an exactly-at-cap stream is accepted
// but anything larger is rejected.
reader
.take(MAX_KEYDB_BYTES + 1)
.read_to_end(&mut buf)
.map_err(|_| Error::KeydbParse)?;
if buf.len() as u64 > MAX_KEYDB_BYTES {
return Err(Error::KeydbInvalid);
}
String::from_utf8(buf).map_err(|_| Error::KeydbParse)
}
/// Build the error returned when the executable's own directory can't be
/// determined (`std::env::current_exe()` fails or has no parent). This is an
/// *environment* failure — the process can't locate itself, which typically
/// signals a stripped container or CI configuration — not a corrupt or
/// unparseable keydb file. Map it to a `NotFound` I/O error so
/// display/remediation paths never claim a keydb parse failure for a file
/// that was never consulted.
fn no_home_dir() -> Error {
Error::IoError {
source: std::io::Error::from(std::io::ErrorKind::NotFound),
}
}
/// Standard keydb storage path — the canonical location to write the keydb to.
///
/// The keydb lives *next to the executable*: `<dir of current exe>/keydb.cfg`,
/// where the directory is `std::env::current_exe()`'s parent. This makes
/// freemkv a portable, standalone binary — drop the exe and its `keydb.cfg`
/// in the same folder and it works, with no OS-specific config dir
/// (`%APPDATA%`, `~/.config`, XDG) involved at all.
///
/// The CLI's read-side search lives in
/// `freemkv-keysources::keydb_search_paths`; it resolves the same exe-local
/// location, so the *write* default used by `save`/`update` and the read-side
/// search always agree.
///
/// Returns a `NotFound` I/O error (via [`no_home_dir`]) if the executable's
/// own directory can't be determined.
pub fn default_path() -> Result<PathBuf> {
std::env::current_exe()
.ok()
.and_then(|exe| exe.parent().map(|dir| dir.join("keydb.cfg")))
.ok_or_else(no_home_dir)
}
/// Download a KEYDB from a URL, verify, save to the standard path.
pub fn update(url: &str) -> Result<UpdateResult> {
let body = http_get(url)?;
save(&body)
}
/// Verify and save raw keydb bytes (plain text, .zip, or .gz).
pub fn save(data: &[u8]) -> Result<UpdateResult> {
let text = if data.starts_with(b"PK\x03\x04") {
extract_zip(data)?
} else if data.starts_with(&[0x1f, 0x8b]) {
read_capped_to_string(flate2::read::GzDecoder::new(data))?
} else {
// Plain-text body: route through the same capped reader as the gz/zip
// branches so an oversized uncompressed upload can't bypass
// MAX_KEYDB_BYTES.
read_capped_to_string(std::io::Cursor::new(data))?
};
let entries = text
.lines()
.filter(|l| {
let t = l.trim();
t.starts_with("0x")
|| t.starts_with("| DK")
|| t.starts_with("| PK")
|| t.starts_with("| HC")
})
.count();
if entries == 0 {
return Err(Error::KeydbInvalid);
}
let path = default_path()?;
write_atomic(&path, &text)?;
Ok(UpdateResult {
path,
entries,
bytes: text.len(),
})
}
/// Write `text` to `path` crash-safely (create parent dir, write a sibling
/// temp file, fsync, then atomic rename).
///
/// keydb.cfg is the single source of AACS truth, and `save`/`update` run
/// unattended (first-boot download + daily-refresh thread, with a container
/// restart on every release). A bare in-place `fs::write` truncates the file
/// before writing, so a SIGKILL (docker stop's grace window), OOM-kill, power
/// loss, or ENOSPC mid-write would leave the keydb half-written — the prior
/// good copy already gone. A truncated keydb doesn't error at write time; it
/// silently breaks key resolution on every later AACS rip. Writing to a temp
/// file then renaming (POSIX rename is atomic within a filesystem) means an
/// interrupted update leaves the previous keydb fully intact.
///
/// The fsync MUST succeed before the rename: a `sync_all` failure (ENOSPC,
/// ESTALE on the bind-mounted volume) means the kernel never guaranteed the
/// bytes reached stable storage, so publishing them via rename would defeat
/// crash-safety. The temp name is unique per call (pid + monotonic counter)
/// so a concurrent update can't share a fixed temp path and rename a mangled
/// file over the keydb.
fn write_atomic(path: &std::path::Path, text: &str) -> Result<()> {
let werr = || Error::KeydbWrite {
path: path.display().to_string(),
};
if let Some(dir) = path.parent() {
std::fs::create_dir_all(dir).map_err(|e| {
tracing::warn!(error = %e, path = %path.display(), "keydb dir create failed");
werr()
})?;
}
let tmp = {
use std::sync::atomic::{AtomicU64, Ordering};
static TMP_COUNTER: AtomicU64 = AtomicU64::new(0);
path.with_extension(format!(
"tmp.{}.{}",
std::process::id(),
TMP_COUNTER.fetch_add(1, Ordering::Relaxed)
))
};
let write_result = (|| -> std::io::Result<()> {
let mut f = std::fs::File::create(&tmp)?;
f.write_all(text.as_bytes())?;
f.sync_all()?;
Ok(())
})();
if let Err(e) = write_result {
let _ = std::fs::remove_file(&tmp);
tracing::warn!(error = %e, path = %path.display(), "keydb write/fsync failed; keydb unchanged");
return Err(werr());
}
if let Err(e) = std::fs::rename(&tmp, path) {
let _ = std::fs::remove_file(&tmp);
tracing::warn!(error = %e, path = %path.display(), "keydb rename failed; keydb unchanged");
return Err(werr());
}
// Durably commit the new dirent: on POSIX filesystems (ext2, some NFS) a
// crash right after the rename can lose the directory entry even though the
// rename returned. Best-effort (swallowed on failure); no-op on Windows.
if let Some(dir) = path.parent() {
crate::io::fsync::dir(dir);
}
Ok(())
}
/// Result of a KEYDB update -- path written, entry count, and byte size.
#[derive(Debug)]
pub struct UpdateResult {
pub path: PathBuf,
pub entries: usize,
pub bytes: usize,
}
fn http_get(url: &str) -> Result<Vec<u8>> {
let (mut host, mut port, mut path) = parse_url(url)?;
for _ in 0..MAX_REDIRECTS {
// Resolve to a concrete socket address so we can bound the connect
// with connect_timeout (plain connect() uses the OS default, which
// can be minutes).
let addr = (host.as_str(), port)
.to_socket_addrs()
.ok()
.and_then(|mut it| it.next())
.ok_or_else(|| Error::KeydbConnect { host: host.clone() })?;
let mut stream = TcpStream::connect_timeout(&addr, NET_TIMEOUT).map_err(|e| {
tracing::debug!(error = %e, host = %host, "keydb connect failed");
Error::KeydbConnect { host: host.clone() }
})?;
stream
.set_read_timeout(Some(READ_TIMEOUT))
.map_err(|_| Error::KeydbConnect { host: host.clone() })?;
stream
.set_write_timeout(Some(NET_TIMEOUT))
.map_err(|_| Error::KeydbConnect { host: host.clone() })?;
// HTTP/1.0 forces close-delimited framing: the server cannot reply
// with Transfer-Encoding: chunked, so the raw body is the keydb
// bytes with no chunk-size lines to de-frame.
let request = format!(
"GET {path} HTTP/1.0\r\nHost: {host}\r\nConnection: close\r\nAccept-Encoding: identity\r\n\r\n"
);
stream
.write_all(request.as_bytes())
.map_err(|_| Error::KeydbConnect { host: host.clone() })?;
// Read the header block incrementally up to the \r\n\r\n terminator,
// bounded to ~64 KiB, BEFORE pulling any body. This avoids buffering up
// to 100 MiB per redirect hop just to inspect the status / Location.
const MAX_HEADER_BYTES: usize = 64 * 1024;
let mut reader = std::io::BufReader::new(stream);
let mut header_buf: Vec<u8> = Vec::with_capacity(1024);
let mut byte = [0u8; 1];
loop {
let n = reader
.read(&mut byte)
.map_err(|_| Error::KeydbConnect { host: host.clone() })?;
if n == 0 {
// Server closed the connection before the header block
// completed: a connection/protocol-level fault, not malformed
// keydb content.
return Err(Error::KeydbConnect { host: host.clone() });
}
header_buf.push(byte[0]);
if header_buf.ends_with(b"\r\n\r\n") {
break;
}
if header_buf.len() >= MAX_HEADER_BYTES {
// Oversized header block from the server: a protocol-level
// fault, not a keydb content parse failure. `>=` caps the
// buffer at exactly MAX_HEADER_BYTES (the `>` form admitted one
// extra byte before tripping).
return Err(Error::KeydbConnect { host: host.clone() });
}
}
// header_buf includes the trailing \r\n\r\n.
let header_end = header_buf.len() - 4;
// Lossy: a stray non-UTF-8 byte in the header block must not blank
// out the whole status line / Location header (which would surface
// as an undiagnosable KeydbHttp{status:0}).
let headers = String::from_utf8_lossy(&header_buf[..header_end]).into_owned();
let headers = headers.as_str();
let status = parse_status(headers).ok_or(Error::KeydbParse)?;
// Only treat a Location header as a redirect when the status is
// actually 3xx; a 200 carrying a stray Location (some proxies) is
// not a redirect, and a 3xx without Location is a malformed redirect.
if (300..=399).contains(&status) {
let location =
extract_header(headers, "Location").ok_or(Error::KeydbHttp { status })?;
let (next_host, next_port, next_path) = resolve_redirect(&location, &host, port)?;
host = next_host;
port = next_port;
path = next_path;
continue;
}
if status != 200 {
return Err(Error::KeydbHttp { status });
}
// Now read the body, still bounded by the existing 100 MiB cap. The
// BufReader carries any bytes already buffered past the header.
let mut body = Vec::new();
reader
.take(100 * 1024 * 1024)
.read_to_end(&mut body)
.map_err(|_| Error::KeydbConnect { host: host.clone() })?;
return Ok(body);
}
Err(Error::KeydbTooManyRedirects)
}
/// Resolve a `Location` value against the current request target.
/// Handles absolute `http://` URLs, scheme-relative `//host/path`,
/// absolute paths `/path`, and rejects unsupported schemes (e.g.
/// `https://`, which this dependency-light client cannot fetch) with a
/// diagnosable error rather than a generic parse failure.
fn resolve_redirect(
location: &str,
cur_host: &str,
cur_port: u16,
) -> Result<(String, u16, String)> {
let loc = location.trim();
if let Some(rest) = loc.strip_prefix("//") {
// Scheme-relative: //host[:port]/path — inherit http.
return parse_url(&format!("http://{rest}"));
}
if loc.starts_with('/') {
// Absolute path on the same host/port.
return Ok((cur_host.to_string(), cur_port, loc.to_string()));
}
if let Some(scheme) = loc.split("://").next() {
if loc.contains("://") && !scheme.eq_ignore_ascii_case("http") {
return Err(Error::KeydbUnsupportedScheme {
scheme: scheme.to_string(),
});
}
}
parse_url(loc)
}
fn parse_url(url: &str) -> Result<(String, u16, String)> {
// Reject non-http(s) up front so the caller gets a scheme diagnostic
// rather than an opaque parse error.
if let Some(scheme) = url.split("://").next() {
if url.contains("://") && !scheme.eq_ignore_ascii_case("http") {
return Err(Error::KeydbUnsupportedScheme {
scheme: scheme.to_string(),
});
}
}
let url = url.strip_prefix("http://").ok_or(Error::KeydbParse)?;
let (host_port, path) = match url.find('/') {
Some(i) => (&url[..i], &url[i..]),
None => (url, "/"),
};
let (host, port) = match host_port.find(':') {
Some(i) => {
let port_str = &host_port[i + 1..];
// A non-empty-but-unparseable port is a malformed URL; only an
// omitted port defaults to 80.
let port = if port_str.is_empty() {
80
} else {
port_str.parse().map_err(|_| Error::KeydbParse)?
};
(&host_port[..i], port)
}
None => (host_port, 80u16),
};
Ok((host.to_string(), port, path.to_string()))
}
fn parse_status(headers: &str) -> Option<u16> {
headers
.lines()
.next()
.and_then(|l| l.split_whitespace().nth(1))
.and_then(|s| s.parse().ok())
}
/// Locate the end of the HTTP header block (the index of the `\r\n\r\n`).
/// Retained for the framing unit tests; the live path now reads headers
/// incrementally in `http_get` so the whole response is never buffered.
#[cfg_attr(not(test), allow(dead_code))]
fn find_header_end(data: &[u8]) -> Option<usize> {
data.windows(4).position(|w| w == b"\r\n\r\n")
}
fn extract_header(headers: &str, name: &str) -> Option<String> {
// Split on the first ':' rather than byte-indexing at name.len(),
// which would panic on a multibyte UTF-8 codepoint straddling that
// offset (headers are decoded from untrusted network bytes). Also
// accepts single-character values (e.g. "Location:x").
for line in headers.lines() {
if let Some((key, value)) = line.split_once(':') {
if key.trim().eq_ignore_ascii_case(name) {
return Some(value.trim().to_string());
}
}
}
None
}
fn extract_zip(data: &[u8]) -> Result<String> {
let cursor = std::io::Cursor::new(data);
let mut archive = zip::ZipArchive::new(cursor).map_err(|_| Error::KeydbParse)?;
for i in 0..archive.len() {
let file = archive.by_index(i).map_err(|_| Error::KeydbParse)?;
if file.name().ends_with(".cfg") || file.name().ends_with(".CFG") {
return read_capped_to_string(file);
}
}
Err(Error::KeydbInvalid)
}
#[cfg(test)]
mod tests {
use super::*;
// Per project convention, tests never touch /tmp (wiped on reboot).
// Anchor scratch under the crate's target/ (gitignored), not /tmp.
fn scratch(tag: &str) -> std::path::PathBuf {
use std::sync::atomic::{AtomicU64, Ordering};
static CTR: AtomicU64 = AtomicU64::new(0);
let n = CTR.fetch_add(1, Ordering::Relaxed);
let d = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("target/test-scratch")
.join(format!("keydb-test-{}-{}-{}", std::process::id(), tag, n));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
}
// Regression: a missing home directory (HOME/USERPROFILE unset) is an
// *environment* failure, not a corrupt keydb. It must NOT surface as
// E8004 (KeydbParse → "failed to parse the keydb file"), which would
// blame a file that was never consulted. It maps to a NotFound I/O
// error in the 5xxx (I/O) category instead.
#[test]
fn no_home_dir_is_io_not_found_not_keydb_parse() {
let e = no_home_dir();
match e {
Error::IoError { source } => {
assert_eq!(source.kind(), std::io::ErrorKind::NotFound);
}
other => panic!("expected IoError(NotFound), got {other:?}"),
}
// And explicitly: it is not the keydb-parse code.
assert_ne!(no_home_dir().code(), Error::KeydbParse.code());
}
// The write default is local to the executable: `<exe dir>/keydb.cfg`.
// Under `cargo test`, `current_exe()` is the test binary in `target/…`;
// assert against the same computation, not a hardcoded path.
#[test]
fn default_path_is_local_to_executable() {
let expected = std::env::current_exe()
.ok()
.and_then(|exe| exe.parent().map(|dir| dir.join("keydb.cfg")));
match expected {
Some(p) => assert_eq!(default_path().unwrap(), p),
None => {
// No exe parent available → must surface the env failure.
assert!(matches!(default_path(), Err(Error::IoError { .. })));
}
}
}
#[test]
fn write_atomic_replaces_existing_and_leaves_no_temp() {
let dir = scratch("atomic");
let path = dir.join("freemkv").join("keydb.cfg");
// First write creates the parent dir + file.
write_atomic(&path, "0xAAAA = old\n").unwrap();
assert_eq!(std::fs::read_to_string(&path).unwrap(), "0xAAAA = old\n");
// Second write replaces it in place.
write_atomic(&path, "0xBBBB = new\n").unwrap();
assert_eq!(std::fs::read_to_string(&path).unwrap(), "0xBBBB = new\n");
// No leftover *.tmp.* sibling — the temp file was renamed, not orphaned.
let leftovers: Vec<_> = std::fs::read_dir(path.parent().unwrap())
.unwrap()
.filter_map(|e| e.ok())
.map(|e| e.file_name().to_string_lossy().into_owned())
.filter(|n| n.contains(".tmp."))
.collect();
assert!(leftovers.is_empty(), "stray temp files: {leftovers:?}");
}
#[test]
fn write_atomic_failure_preserves_prior_keydb() {
// Simulate the crash window: a good keydb already on disk, then an
// update whose write target can't be created (parent path is a file,
// so create_dir_all under it fails — i.e. ENOTDIR). The rename never
// happens, so the existing keydb must survive untouched.
let dir = scratch("preserve");
let good = dir.join("keydb.cfg");
write_atomic(&good, "0xGOOD = keep\n").unwrap();
// `good` is a regular file; treating it as a directory parent fails.
let doomed = good.join("freemkv").join("keydb.cfg");
let err = write_atomic(&doomed, "0xBAD = partial\n");
assert!(matches!(err, Err(Error::KeydbWrite { .. })));
// Prior good copy is intact.
assert_eq!(std::fs::read_to_string(&good).unwrap(), "0xGOOD = keep\n");
}
#[test]
fn parse_url_defaults_and_paths() {
let (h, p, path) = parse_url("http://example.com/keydb.zip").unwrap();
assert_eq!(
(h.as_str(), p, path.as_str()),
("example.com", 80, "/keydb.zip")
);
let (h, p, path) = parse_url("http://example.com:8080").unwrap();
assert_eq!((h.as_str(), p, path.as_str()), ("example.com", 8080, "/"));
}
#[test]
fn parse_url_rejects_https_scheme() {
// TLS is unsupported by this client; surface a scheme diagnostic
// rather than a generic parse error.
assert!(matches!(
parse_url("https://example.com/k.zip"),
Err(Error::KeydbUnsupportedScheme { .. })
));
}
#[test]
fn parse_url_rejects_malformed_port() {
// Non-empty-but-unparseable port must error, not silently fall to 80.
assert!(matches!(
parse_url("http://example.com:abc/path"),
Err(Error::KeydbParse)
));
// An empty port still defaults to 80.
let (_, p, _) = parse_url("http://example.com:/path").unwrap();
assert_eq!(p, 80);
}
#[test]
fn redirect_to_https_is_unsupported_scheme_not_parse_error() {
// The bplaced-style mirror enabling TLS on a redirect must produce a
// diagnosable scheme error, not KeydbParse.
assert!(matches!(
resolve_redirect("https://mirror.example/keydb.zip", "old.host", 80),
Err(Error::KeydbUnsupportedScheme { .. })
));
}
#[test]
fn redirect_scheme_relative_and_absolute_path() {
// Scheme-relative //host/path inherits http.
let (h, p, path) = resolve_redirect("//mirror.example/a.zip", "old.host", 80).unwrap();
assert_eq!(
(h.as_str(), p, path.as_str()),
("mirror.example", 80, "/a.zip")
);
// Absolute path stays on the current host/port.
let (h, p, path) = resolve_redirect("/new/path.zip", "cur.host", 8080).unwrap();
assert_eq!(
(h.as_str(), p, path.as_str()),
("cur.host", 8080, "/new/path.zip")
);
// Absolute http URL is followed normally.
let (h, _, path) = resolve_redirect("http://other.host/x.zip", "cur.host", 80).unwrap();
assert_eq!((h.as_str(), path.as_str()), ("other.host", "/x.zip"));
}
#[test]
fn parse_status_extracts_code() {
assert_eq!(parse_status("HTTP/1.0 200 OK\r\nFoo: bar"), Some(200));
assert_eq!(parse_status("HTTP/1.1 301 Moved Permanently"), Some(301));
assert_eq!(parse_status("garbage"), None);
}
// ── New comprehensive tests ────────────────────────────────────────────────
/// find_header_end detects the \r\n\r\n separator (RFC 7230 §3 — HTTP header
/// terminator is CRLF CRLF). Returns the byte position of the first \r.
/// Mutation: searching for \n\n instead of \r\n\r\n misses the boundary.
#[test]
fn find_header_end_locates_crlfcrlf() {
let data = b"HTTP/1.0 200 OK\r\nContent-Length: 42\r\n\r\nbody starts here";
// The \r\n\r\n starts at byte 37 (after the Content-Length line).
let pos = find_header_end(data).expect("must find header end");
// body starts at pos + 4 (past the \r\n\r\n).
assert_eq!(
&data[pos + 4..],
b"body starts here",
"body must begin immediately after the \\r\\n\\r\\n boundary"
);
}
/// find_header_end returns None when there is no \r\n\r\n.
/// Mutation: returning Some(0) unconditionally makes this fail.
#[test]
fn find_header_end_returns_none_when_absent() {
let data = b"no separator here at all";
assert!(find_header_end(data).is_none());
}
/// extract_header is case-insensitive per RFC 7230 §3.2.
/// Mutation: using case-sensitive comparison misses "location" vs "Location".
#[test]
fn extract_header_case_insensitive() {
let headers = "HTTP/1.1 301 Moved\r\nlocation: http://new.host/path\r\n";
let val = extract_header(headers, "Location").expect("must find Location");
assert_eq!(val, "http://new.host/path");
}
/// extract_header with a missing header returns None.
/// Mutation: returning Some("") makes the caller proceed on a missing Location header.
#[test]
fn extract_header_missing_returns_none() {
let headers = "HTTP/1.0 200 OK\r\nContent-Type: text/plain\r\n";
assert!(extract_header(headers, "Location").is_none());
}
/// extract_header trims leading/trailing whitespace from the value.
/// RFC 7230 §3.2.6: optional whitespace around field value.
/// Mutation: not trimming the value keeps leading spaces in the URL.
#[test]
fn extract_header_trims_value_whitespace() {
let headers = "HTTP/1.1 301 Moved\r\nLocation: /new/path \r\n";
let val = extract_header(headers, "Location").unwrap();
assert_eq!(val, "/new/path", "value must be trimmed");
}
/// save() rejects data that is not a valid keydb (no recognisable entries).
/// Spec: entries are lines starting with "0x", "| DK", "| PK", or "| HC".
/// Mutation: dropping the entries==0 check lets an empty file be saved.
#[test]
fn save_rejects_empty_text() {
// Plain text with no valid keydb entries.
let garbage = b"this is not a keydb\njust random text\n";
assert!(
matches!(save(garbage), Err(Error::KeydbInvalid)),
"keydb without valid entries must be rejected"
);
}
/// save() accepts plain text with at least one "0x"-prefixed entry line.
/// Mutation: counting only "| DK" lines ignores the "0x" entry format.
#[test]
fn save_accepts_plaintext_with_0x_entries() {
// Minimal keydb-style file with a VUK entry (0x-prefixed).
let content = b"0xDEADBEEFCAFEBABE0102030405060708090A0B0C0D0E0F\n";
// We can't predict the HOME path in test environments without
// potentially writing to a real location. So only check that save()
// accepts this content as valid (may return KeydbWrite if dir exists
// but we lack permission — that still proves it passed the parse check).
let result = save(content);
// Accept either Ok (wrote successfully) or a write error (env issue),
// but NOT KeydbInvalid or KeydbParse.
match &result {
Ok(_) => {}
Err(Error::KeydbWrite { .. }) => {}
Err(e) => panic!("unexpected error for valid keydb content: {:?}", e),
}
}
/// save() accepts content with "| DK" entries (device-key table format).
/// Mutation: only accepting "0x" lines rejects DK-format keydb files.
#[test]
fn save_accepts_pipe_dk_entry_format() {
let content = b"| DK 0102030405060708 | 0102030405060708090a0b0c0d0e0f10 |\n";
let result = save(content);
match &result {
Ok(_) => {}
Err(Error::KeydbWrite { .. }) => {}
Err(e) => panic!("unexpected error for DK-format entry: {:?}", e),
}
}
/// save() accepts content with "| PK" entries (processing-key format).
/// Mutation: not including "| PK" in the filter rejects PK-format keydb files.
#[test]
fn save_accepts_pipe_pk_entry_format() {
let content = b"| PK 0102030405060708090a0b0c0d0e0f10 |\n";
let result = save(content);
match &result {
Ok(_) => {}
Err(Error::KeydbWrite { .. }) => {}
Err(e) => panic!("unexpected error for PK-format entry: {:?}", e),
}
}
/// save() accepts content with "| HC" entries (host certificate format).
/// Mutation: not including "| HC" in the filter rejects HC-format keydb files.
#[test]
fn save_accepts_pipe_hc_entry_format() {
let content = b"| HC 0102030405060708090a0b0c0d0e0f10 |\n";
let result = save(content);
match &result {
Ok(_) => {}
Err(Error::KeydbWrite { .. }) => {}
Err(e) => panic!("unexpected error for HC-format entry: {:?}", e),
}
}
/// save() recognises gzip-compressed input (magic bytes 0x1f 0x8b).
/// Spec: gzip format magic is 0x1F 0x8B (RFC 1952 §2.3.1).
/// Mutation: treating gzip magic as plain text fails to decompress.
#[test]
fn save_recognises_gzip_magic() {
// Truncated gzip (header only, no valid body) — must not be treated as
// plain text (no KeydbInvalid about entries), but as a parse error.
let bad_gz = [0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03];
let result = save(&bad_gz);
// A truncated gzip is either KeydbParse (decompression error) or
// KeydbInvalid (decompressed to empty). Must not be Ok.
assert!(result.is_err(), "truncated gzip must not be accepted");
// Crucially: must NOT be a plain-text UTF-8 error — gzip magic is not UTF-8.
match result.unwrap_err() {
Error::KeydbParse | Error::KeydbInvalid => {}
e => panic!("wrong error kind for truncated gzip: {:?}", e),
}
}
/// save() recognises ZIP magic bytes PK\x03\x04 and routes to extract_zip.
/// Spec: ZIP local file header signature is 0x50 0x4B 0x03 0x04 (PKZIP APPNOTE §4.3.6).
/// A truncated ZIP must error, but NOT as plain UTF-8 text.
/// Mutation: checking gzip magic before ZIP magic means ZIP files are
/// fed to the gzip decoder and produce the wrong error.
#[test]
fn save_recognises_zip_magic() {
// Valid ZIP magic followed by garbage — must be routed to extract_zip.
let bad_zip = b"PK\x03\x04garbage that is not a real zip";
let result = save(bad_zip);
assert!(result.is_err(), "invalid zip must be rejected");
// Must be a parse error, not a UTF-8 error.
match result.unwrap_err() {
Error::KeydbParse | Error::KeydbInvalid => {}
e => panic!("wrong error for bad zip: {:?}", e),
}
}
/// read_capped_to_string rejects data exceeding MAX_KEYDB_BYTES.
/// Spec: doc says "Returns Error::KeydbInvalid if the input exceeds the cap".
/// Mutation: removing the length check accepts decompression bombs.
#[test]
fn read_capped_to_string_rejects_oversized_input() {
// Build a reader that reports it has more data than the cap.
// We use a Cursor with MAX_KEYDB_BYTES + 1 bytes of content.
let too_big = vec![b'A'; (MAX_KEYDB_BYTES + 1) as usize];
let cursor = std::io::Cursor::new(too_big);
let result = read_capped_to_string(cursor);
assert!(
matches!(result, Err(Error::KeydbInvalid)),
"oversized input must yield KeydbInvalid, got: {:?}",
result
);
}
/// read_capped_to_string returns KeydbParse (not KeydbInvalid) for
/// non-UTF-8 input. Guards the doc/behavior contract: KeydbInvalid is
/// reserved for the size-cap violation, a decode failure is a parse error.
#[test]
fn read_capped_to_string_non_utf8_yields_parse() {
// 0xFF is never a valid UTF-8 byte.
let cursor = std::io::Cursor::new(vec![0xFFu8, 0xFE, 0xFD]);
let result = read_capped_to_string(cursor);
assert!(
matches!(result, Err(Error::KeydbParse)),
"non-UTF-8 input must yield KeydbParse, got: {:?}",
result
);
}
/// read_capped_to_string accepts exactly MAX_KEYDB_BYTES (at-cap is allowed).
/// Spec: doc says "Read one byte past the cap so an exactly-at-cap stream is accepted."
/// Mutation: using `>=` instead of `>` in the length check rejects valid at-cap files.
#[test]
fn read_capped_to_string_accepts_at_cap_size() {
let at_cap = vec![b'A'; MAX_KEYDB_BYTES as usize];
let cursor = std::io::Cursor::new(at_cap);
let result = read_capped_to_string(cursor);
assert!(result.is_ok(), "exactly MAX_KEYDB_BYTES must be accepted");
}
/// parse_status returns None for an empty/malformed status line (not a
/// meaningless 0). The call site maps None to Error::KeydbParse.
#[test]
fn parse_status_empty_input_returns_none() {
assert_eq!(parse_status(""), None);
assert_eq!(parse_status("\r\n"), None);
}
/// Regression: set_read_timeout / set_write_timeout failures must surface as
/// KeydbConnect, not be silently swallowed.
///
/// We can't easily synthesise a TcpStream whose set_*timeout syscall fails
/// without a platform-specific socket hack, so instead we verify that the
/// error-mapping expression itself is correct: if set_read_timeout were to
/// fail for a given host, the result must be Err(KeydbConnect { host }).
///
/// The test constructs the exact Err value the code would return and asserts
/// it is KeydbConnect (not, say, silently Ok or a different variant). This
/// pins the variant selection so a future refactor that changes the `.ok()`
/// pattern back would need to update this test as well.
#[test]
fn timeout_set_failure_maps_to_keydb_connect() {
// Simulate what the propagated error looks like.
let host = "hostile.example.com".to_string();
// The io::Error that set_read_timeout would return on failure.
let io_err = std::io::Error::from(std::io::ErrorKind::InvalidInput);
// Apply the same map_err the production code uses.
let result: Result<()> =
Err(io_err).map_err(|_| Error::KeydbConnect { host: host.clone() });
assert!(
matches!(result, Err(Error::KeydbConnect { host: ref h }) if h == "hostile.example.com"),
"set_timeout failure must map to KeydbConnect, got: {:?}",
result
);
}
/// Regression: http_get to an unreachable host returns KeydbConnect, not a hang.
/// This exercises the connect_timeout path (and thus confirms the overall
/// error-propagation chain is wired); the timeout-set propagation is exercised
/// by the unit test above.
///
/// Uses port 1 on localhost, which is reserved/unassigned and virtually never
/// listening. connect_timeout with NET_TIMEOUT will refuse or time out quickly.
/// We only assert the error variant, not the host field, since the OS may
/// resolve the address differently.
#[test]
fn http_get_unreachable_host_returns_keydb_connect() {
// Port 1 on loopback — almost always refused immediately.
let result = http_get("http://127.0.0.1:1/keydb.zip");
// Must be an Err; KeydbConnect is expected for a TCP-level failure.
// KeydbParse or KeydbHttp would indicate the wrong error path.
assert!(result.is_err(), "unreachable host must fail");
match result.unwrap_err() {
Error::KeydbConnect { .. } => {}
e => panic!("expected KeydbConnect for unreachable host, got: {:?}", e),
}
}
/// Regression: when the server accepts the connection but closes it before
/// sending complete HTTP headers (the `n == 0` byte-read path), that is a
/// connection/protocol-level fault. It must surface as KeydbConnect (E8000),
/// NOT KeydbParse (E8004) — the keydb content was never received, let alone
/// malformed.
#[test]
fn http_get_server_drops_before_headers_returns_keydb_connect() {
use std::io::Read as _;
use std::net::TcpListener;
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let addr = listener.local_addr().unwrap();
let server = std::thread::spawn(move || {
if let Ok((mut sock, _)) = listener.accept() {
// Drain the request so the client's write_all completes, then
// drop the socket without writing any response. The client's
// header read then returns n == 0.
let mut buf = [0u8; 512];
let _ = sock.read(&mut buf);
drop(sock);
}
});
let url = format!("http://127.0.0.1:{}/keydb.zip", addr.port());
let result = http_get(&url);
server.join().unwrap();
assert!(result.is_err(), "dropped connection must fail");
match result.unwrap_err() {
Error::KeydbConnect { .. } => {}
e => panic!("expected KeydbConnect for dropped connection, got: {:?}", e),
}
}
}
+6
View File
@@ -100,6 +100,12 @@ pub struct DiscInputsCtx<'a> {
impl<'a> DiscInputsCtx<'a> { impl<'a> DiscInputsCtx<'a> {
/// Build a context over `inputs`, parsing the encrypted title keys at the /// Build a context over `inputs`, parsing the encrypted title keys at the
/// stride for AACS major `version_u8` (1 = V10, else V20/V21). /// stride for AACS major `version_u8` (1 = V10, else V20/V21).
///
/// A present-but-malformed `unit_key_ro` (truncated / wrong magic / wrong
/// stride) parses to an empty key set, so a later [`Self::enc_title_keys`]
/// returns `Ok(&[])` indistinguishably from a disc that legitimately has no
/// title keys — the parse failure is swallowed here, not surfaced as an
/// error.
pub fn new(inputs: &'a DiscInputs, version_u8: u8) -> Self { pub fn new(inputs: &'a DiscInputs, version_u8: u8) -> Self {
use crate::aacs::{AacsVersion, parse_unit_key_ro}; use crate::aacs::{AacsVersion, parse_unit_key_ro};
let enc_keys = if inputs.unit_key_ro.is_empty() { let enc_keys = if inputs.unit_key_ro.is_empty() {
+10 -6
View File
@@ -634,6 +634,7 @@ fn append_clpi_orphans(
reader: &mut dyn SectorSource, reader: &mut dyn SectorSource,
udf: &UdfFs, udf: &UdfFs,
) -> usize { ) -> usize {
use crate::consts::coding_type as c;
// Index existing labels by PID — but StreamLabel doesn't carry // Index existing labels by PID — but StreamLabel doesn't carry
// PID. Index by (type, language, codec_hint) tuple instead; this // PID. Index by (type, language, codec_hint) tuple instead; this
// is fuzzier than PID matching but the only signal available // is fuzzier than PID matching but the only signal available
@@ -679,9 +680,11 @@ fn append_clpi_orphans(
// Interactive Graphics (BD-J menu overlay), NOT a user-facing // Interactive Graphics (BD-J menu overlay), NOT a user-facing
// subtitle — skip it, matching the MPLS path which drops IG. // subtitle — skip it, matching the MPLS path which drops IG.
let stype = match s.coding_type { let stype = match s.coding_type {
0x80..=0x86 | 0xA1 | 0xA2 => StreamLabelType::Audio, c::LPCM..=c::DTS_HD_MA | c::AC3_PLUS_SECONDARY | c::DTS_HD_SECONDARY => {
0x90 => StreamLabelType::Subtitle, StreamLabelType::Audio
_ => continue, // 0x91 IG / video / unknown — skip }
c::PG => StreamLabelType::Subtitle,
_ => continue, // IG / video / unknown — skip
}; };
// Same dedup logic as MPLS: normalize language, build codec // Same dedup logic as MPLS: normalize language, build codec
// hint, check against existing label set. // hint, check against existing label set.
@@ -1121,9 +1124,10 @@ mod registry_tests {
// and as a marker for "these parsers exist." // and as a marker for "these parsers exist."
let _ = (name, detect, parse); let _ = (name, detect, parse);
} }
// The loop above touches every registry entry; iterating a non-empty // The loop above touches every registry entry. The non-empty
// fixed-size array is the assertion (a `.is_empty()` check would be // invariant is covered separately by `parsers_registry_order_locked`,
// const-folded). The test fails to compile if the tuple shape changes. // whose assert_eq! on the expected order fails if PARSERS is empty.
// This test fails to compile if the tuple shape changes.
} }
} }
+15 -14
View File
@@ -227,21 +227,22 @@ pub(crate) fn language_display_name(iso: &str) -> String {
/// bytes (the table covers everything the spec defines, but unknown /// bytes (the table covers everything the spec defines, but unknown
/// values are still possible on malformed discs). /// values are still possible on malformed discs).
pub(crate) fn codec_name(coding_type: u8) -> &'static str { pub(crate) fn codec_name(coding_type: u8) -> &'static str {
use crate::consts::coding_type as c;
match coding_type { match coding_type {
0x02 => "MPEG-2", c::MPEG2_VIDEO => "MPEG-2",
0x1B => "H.264", c::H264 => "H.264",
0x24 => "HEVC", c::HEVC => "HEVC",
0x80 => "LPCM", c::LPCM => "LPCM",
0x81 => "AC-3", c::AC3 => "AC-3",
0x82 => "DTS", c::DTS => "DTS",
0x83 => "TrueHD", c::TRUEHD => "TrueHD",
0x84 => "AC-3+", c::AC3_PLUS => "AC-3+",
0x85 => "DTS-HD HR", // BD-ROM Part 3-1: 0x85 = DTS-HD High Resolution c::DTS_HD_HR => "DTS-HD HR", // BD-ROM Part 3-1: 0x85 = DTS-HD High Resolution
0x86 => "DTS-HD MA", c::DTS_HD_MA => "DTS-HD MA",
0x90 => "PG", c::PG => "PG",
0x91 => "IG", c::IG => "IG",
0xA1 => "AC-3+ Secondary", c::AC3_PLUS_SECONDARY => "AC-3+ Secondary",
0xA2 => "DTS-HD Secondary", c::DTS_HD_SECONDARY => "DTS-HD Secondary",
_ => "", _ => "",
} }
} }
+15 -1
View File
@@ -85,6 +85,20 @@
//! | E8xxx | Keydb errors (fetch, parse, load) | //! | E8xxx | Keydb errors (fetch, parse, load) |
//! | E9xxx | Stream / mux errors (URL, PES, pipeline) | //! | E9xxx | Stream / mux errors (URL, PES, pipeline) |
/// Single source of truth for every freemkv version surface.
///
/// `FREEMKV_VERSION` is the package version, overridable at build time via the
/// `FREEMKV_BUILD_LABEL` env (see `build.rs`); `GIT_SUFFIX` is the git short
/// hash. The CLI's `--version`, the MKV muxing/writing-application field, and
/// the FVI generator tag all derive from these two consts, so a binary reports
/// the exact same label it stamps into the files it produces — no split-brain
/// where an MKV claims one version and the binary another.
pub const VERSION_LABEL: &str = concat!(env!("FREEMKV_VERSION"), env!("GIT_SUFFIX"));
/// The muxing/writing-application string written into MKV output
/// (`"freemkv <version> (g<hash>)"`).
pub const MUX_APP: &str = concat!("freemkv ", env!("FREEMKV_VERSION"), env!("GIT_SUFFIX"));
pub mod aacs; pub mod aacs;
pub(crate) mod clpi; pub(crate) mod clpi;
pub mod consts; pub mod consts;
@@ -93,13 +107,13 @@ pub mod decrypt;
pub mod diag; pub mod diag;
pub mod disc; pub mod disc;
pub mod drive; pub mod drive;
pub mod dvdnav;
pub mod error; pub mod error;
pub mod event; pub mod event;
pub mod halt; pub mod halt;
pub(crate) mod identity; pub(crate) mod identity;
pub(crate) mod ifo; pub(crate) mod ifo;
pub mod io; pub mod io;
pub mod keydb;
pub mod keysource; pub mod keysource;
pub mod labels; pub mod labels;
pub(crate) mod mpls; pub(crate) mod mpls;
+29 -34
View File
@@ -179,7 +179,9 @@ pub fn parse(data: &[u8]) -> Result<Playlist> {
} }
// PG subtitles // PG subtitles
for _ in 0..n_pg { for _ in 0..n_pg {
if let Some((entry, next)) = parse_stream_entry(item, spos, STREAM_CATEGORY_PG_SUBTITLE) { if let Some((entry, next)) =
parse_stream_entry(item, spos, STREAM_CATEGORY_PG_SUBTITLE)
{
streams.push(entry); streams.push(entry);
spos = next; spos = next;
} else { } else {
@@ -196,7 +198,9 @@ pub fn parse(data: &[u8]) -> Result<Playlist> {
} }
// Secondary audio // Secondary audio
for _ in 0..n_sec_audio { for _ in 0..n_sec_audio {
if let Some((mut entry, next)) = parse_stream_entry(item, spos, STREAM_CATEGORY_AUDIO) { if let Some((mut entry, next)) =
parse_stream_entry(item, spos, STREAM_CATEGORY_AUDIO)
{
entry.stream_type = 5; entry.stream_type = 5;
entry.secondary = true; entry.secondary = true;
streams.push(entry); streams.push(entry);
@@ -213,7 +217,9 @@ pub fn parse(data: &[u8]) -> Result<Playlist> {
} }
// Secondary video (PiP) // Secondary video (PiP)
for _ in 0..n_sec_video { for _ in 0..n_sec_video {
if let Some((mut entry, next)) = parse_stream_entry(item, spos, STREAM_CATEGORY_VIDEO) { if let Some((mut entry, next)) =
parse_stream_entry(item, spos, STREAM_CATEGORY_VIDEO)
{
entry.stream_type = 6; entry.stream_type = 6;
entry.secondary = true; entry.secondary = true;
streams.push(entry); streams.push(entry);
@@ -240,7 +246,9 @@ pub fn parse(data: &[u8]) -> Result<Playlist> {
} }
// Secondary PG (PiP subtitles) — must consume to keep spos aligned // Secondary PG (PiP subtitles) — must consume to keep spos aligned
for _ in 0..n_pip_pg { for _ in 0..n_pip_pg {
if let Some((mut entry, next)) = parse_stream_entry(item, spos, STREAM_CATEGORY_PG_SUBTITLE) { if let Some((mut entry, next)) =
parse_stream_entry(item, spos, STREAM_CATEGORY_PG_SUBTITLE)
{
entry.secondary = true; entry.secondary = true;
streams.push(entry); streams.push(entry);
// Skip reference data: num_refs(1) + reserved(1) + refs + padding // Skip reference data: num_refs(1) + reserved(1) + refs + padding
@@ -256,7 +264,9 @@ pub fn parse(data: &[u8]) -> Result<Playlist> {
} }
// Dolby Vision enhancement layer // Dolby Vision enhancement layer
for _ in 0..n_dv { for _ in 0..n_dv {
if let Some((mut entry, next)) = parse_stream_entry(item, spos, STREAM_CATEGORY_VIDEO) { if let Some((mut entry, next)) =
parse_stream_entry(item, spos, STREAM_CATEGORY_VIDEO)
{
entry.stream_type = 7; entry.stream_type = 7;
entry.secondary = true; entry.secondary = true;
streams.push(entry); streams.push(entry);
@@ -333,6 +343,7 @@ const STREAM_CATEGORY_PG_SUBTITLE: u8 = 3;
const STREAM_CATEGORY_IG: u8 = 4; const STREAM_CATEGORY_IG: u8 = 4;
fn parse_stream_entry(item: &[u8], pos: usize, stream_type: u8) -> Option<(StreamEntry, usize)> { fn parse_stream_entry(item: &[u8], pos: usize, stream_type: u8) -> Option<(StreamEntry, usize)> {
use crate::consts::coding_type as c;
if pos + 2 > item.len() { if pos + 2 > item.len() {
return None; return None;
} }
@@ -387,22 +398,27 @@ fn parse_stream_entry(item: &[u8], pos: usize, stream_type: u8) -> Option<(Strea
let mut color_space_val = 0u8; let mut color_space_val = 0u8;
let mut language = String::new(); let mut language = String::new();
// `stream_type` here is the STN category passed by the caller, which is
// only ever a primary category (VIDEO/AUDIO/PG_SUBTITLE/IG). Secondary
// audio/video and the DV enhancement layer are parsed through their
// matching primary category (identical attribute layout) and re-tagged by
// the caller after this returns, so there are no secondary arms here.
match stream_type { match stream_type {
1 => { STREAM_CATEGORY_VIDEO => {
// Video: coding_type(1) + format_rate(1) + [hdr_info(1) if HEVC] // Video: coding_type(1) + format_rate(1) + [hdr_info(1) if HEVC]
if sa.len() >= 2 { if sa.len() >= 2 {
video_format = (sa[1] >> 4) & 0x0F; video_format = (sa[1] >> 4) & 0x0F;
video_rate = sa[1] & 0x0F; video_rate = sa[1] & 0x0F;
} }
if coding_type == 0x24 && sa.len() > 2 { if coding_type == c::HEVC && sa.len() > 2 {
dynamic_range = (sa[2] >> 4) & 0x0F; dynamic_range = (sa[2] >> 4) & 0x0F;
color_space_val = sa[2] & 0x0F; color_space_val = sa[2] & 0x0F;
} }
} }
2 => { STREAM_CATEGORY_AUDIO => {
// Audio: coding_type(1) + format_rate(1) + language(3) // Audio: coding_type(1) + format_rate(1) + language(3)
// Exception: PGS (0x90/0x91) in audio slot uses PG layout: coding_type(1) + language(3) // Exception: PG/IG in an audio slot uses PG layout: coding_type(1) + language(3)
if coding_type == 0x90 || coding_type == 0x91 { if coding_type == c::PG || coding_type == c::IG {
if sa.len() >= 4 { if sa.len() >= 4 {
language = String::from_utf8_lossy(&sa[1..4]).to_string(); language = String::from_utf8_lossy(&sa[1..4]).to_string();
} }
@@ -416,35 +432,14 @@ fn parse_stream_entry(item: &[u8], pos: usize, stream_type: u8) -> Option<(Strea
} }
} }
} }
3 => { STREAM_CATEGORY_PG_SUBTITLE => {
// PG: coding_type(1) + language(3). // PG: coding_type(1) + language(3).
// IG (type 4) is parsed only to advance spos and is then // IG is parsed only to advance spos and is then discarded by the
// discarded by the caller, so it deliberately has no arm here. // caller, so it deliberately has no arm here.
if sa.len() >= 4 { if sa.len() >= 4 {
language = String::from_utf8_lossy(&sa[1..4]).to_string(); language = String::from_utf8_lossy(&sa[1..4]).to_string();
} }
} }
5 => {
// Secondary audio: same as primary audio
if sa.len() >= 2 {
audio_format = (sa[1] >> 4) & 0x0F;
audio_rate = sa[1] & 0x0F;
}
if sa.len() >= 5 {
language = String::from_utf8_lossy(&sa[2..5]).to_string();
}
}
6 | 7 => {
// Secondary video: same as primary video
if sa.len() >= 2 {
video_format = (sa[1] >> 4) & 0x0F;
video_rate = sa[1] & 0x0F;
}
if coding_type == 0x24 && sa.len() > 2 {
dynamic_range = (sa[2] >> 4) & 0x0F;
color_space_val = sa[2] & 0x0F;
}
}
_ => {} _ => {}
} }
+11 -4
View File
@@ -181,8 +181,13 @@ impl PictureInfo {
match self.detail { match self.detail {
CodingDetail::Mpeg2(m) => { CodingDetail::Mpeg2(m) => {
if !m.frame_picture { if !m.frame_picture {
// A single field picture is inherently interlaced; the // A single field picture is inherently interlaced. Which
// top_field_first bit names which field this picture is. // field it actually codes is given by picture_structure
// (top/bottom), not by top_field_first — §6.3.10 constrains
// top_field_first to 0 for field pictures, so it is not the
// spec source here. picture_structure is not retained on
// this carrier, so top_field_first is used only as the lone
// field hint available (best-effort, not spec-derived).
Some(if m.top_field_first { Some(if m.top_field_first {
FieldOrder::Tff FieldOrder::Tff
} else { } else {
@@ -203,8 +208,10 @@ impl PictureInfo {
/// Number of field-display periods this picture occupies — the basis for /// Number of field-display periods this picture occupies — the basis for
/// soft-telecine (2:3 pulldown) timing. MPEG-2 (ISO/IEC 13818-2 §6.3.10, /// soft-telecine (2:3 pulldown) timing. MPEG-2 (ISO/IEC 13818-2 §6.3.10,
/// ffmpeg `nb_fields = repeat_pict + 2`): a field picture occupies 1 field, /// ffmpeg `nb_fields = repeat_pict + 2`): a field picture occupies 1 field,
/// a normal frame 2, a `repeat_first_field` frame 3 (or 4/6 in a progressive /// a normal frame 2, a `repeat_first_field` progressive-frame 3 (or 4/6 in a
/// sequence). Codecs without pulldown signalling report the normal 2 fields. /// progressive sequence); an rff bit on a non-progressive interlaced frame is
/// spec-forbidden (§6.3.10) and is treated as 2. Codecs without pulldown
/// signalling report the normal 2 fields.
pub fn nb_fields(&self) -> u8 { pub fn nb_fields(&self) -> u8 {
match self.detail { match self.detail {
CodingDetail::Mpeg2(m) => { CodingDetail::Mpeg2(m) => {
+156 -11
View File
@@ -5,14 +5,14 @@
//! Buffers across PES boundaries so frames spanning two PES packets //! Buffers across PES boundaries so frames spanning two PES packets
//! are emitted complete. //! are emitted complete.
use super::startcode::BitReader;
use super::{CodecParser, Frame, PesPacket, pts_to_ns}; use super::{CodecParser, Frame, PesPacket, pts_to_ns};
const DTS_CORE_SYNC: [u8; 4] = [0x7F, 0xFE, 0x80, 0x01]; const DTS_CORE_SYNC: [u8; 4] = [0x7F, 0xFE, 0x80, 0x01];
/// DTS-HD extension substream syncword. The parser delimits an access unit by /// DTS-HD extension substream syncword. An access unit is delimited by the next
/// the next CORE sync (so every extension between two cores is captured), and /// CORE sync; the parser locates and exactly sizes each extension substream (via
/// never needs to locate or size the extension itself — so this is referenced /// `exss_frame_size`) so a false core sync inside the EXSS payload can't split
/// only by the tests that synthesize extension substreams. /// the AU and truncate the lossless extension.
#[cfg(test)]
const DTS_HD_EXT_SYNC: [u8; 4] = [0x64, 0x58, 0x20, 0x25]; const DTS_HD_EXT_SYNC: [u8; 4] = [0x64, 0x58, 0x20, 0x25];
/// DTS / DTS-HD elementary-stream parser. Buffers DTS across PES boundaries so /// DTS / DTS-HD elementary-stream parser. Buffers DTS across PES boundaries so
@@ -336,11 +336,84 @@ enum NextCore {
/// match the core syncword, so each candidate is validated by decoding its /// match the core syncword, so each candidate is validated by decoding its
/// core size: a match whose decoded size is implausible (< MIN_CORE_FRAME_BYTES /// core size: a match whose decoded size is implausible (< MIN_CORE_FRAME_BYTES
/// or > MAX_AU_BYTES) is a false sync and is skipped, continuing the search. /// or > MAX_AU_BYTES) is a false sync and is skipped, continuing the search.
/// Both DTS syncwords (core `0x7FFE8001`, EXSS `0x64582025`) are 32-bit words.
const SYNCWORD_BYTES: usize = DTS_CORE_SYNC.len();
/// DTS-HD extension-substream (EXSS) header field bit widths (ETSI TS 102 114,
/// ExtSS header). `bHeaderSizeType` selects the short form (`nuExtSSHeaderSize`
/// 8 bits, `nuExtSSFsize` 16 bits) or, for larger substreams, the long form
/// (12 / 20 bits).
const EXSS_USER_DEFINED_BITS: u32 = 8;
const EXSS_INDEX_BITS: u32 = 2;
const EXSS_HEADER_SIZE_TYPE_BITS: u32 = 1;
const EXSS_HDRSIZE_BITS_SHORT: u32 = 8;
const EXSS_FSIZE_BITS_SHORT: u32 = 16;
const EXSS_HDRSIZE_BITS_LONG: u32 = 12;
const EXSS_FSIZE_BITS_LONG: u32 = 20;
/// `bHeaderSizeType == 1` selects the long-form field widths.
const EXSS_HEADER_SIZE_TYPE_LONG: u32 = 1;
/// Bytes that must be buffered to read the EXSS size fields in the worst case
/// (long form): the 4-byte sync plus the bits up through `nuExtSSFsize`.
const EXSS_HEADER_MIN_BYTES: usize = SYNCWORD_BYTES
+ (EXSS_USER_DEFINED_BITS
+ EXSS_INDEX_BITS
+ EXSS_HEADER_SIZE_TYPE_BITS
+ EXSS_HDRSIZE_BITS_LONG
+ EXSS_FSIZE_BITS_LONG)
.div_ceil(u8::BITS) as usize;
/// DTS-HD extension substream (EXSS) total byte size — INCLUDING the
/// `0x64582025` syncword — read precisely from its header. `buf` must begin with
/// `DTS_HD_EXT_SYNC`. `None` when the size fields aren't fully buffered.
///
/// `nuExtSSFsize` is the total frame size in bytes minus one. Parsing it lets the
/// AU framer skip the extension by its exact length instead of scanning its
/// (arbitrary) payload for a core sync.
fn exss_frame_size(buf: &[u8]) -> Option<usize> {
if buf.len() < EXSS_HEADER_MIN_BYTES {
return None;
}
let mut r = BitReader::new(&buf[SYNCWORD_BYTES..]);
let _user = r.read_bits(EXSS_USER_DEFINED_BITS)?; // nUserDefinedBits
let _idx = r.read_bits(EXSS_INDEX_BITS)?; // nExtSSIndex
let large = r.read_bits(EXSS_HEADER_SIZE_TYPE_BITS)? == EXSS_HEADER_SIZE_TYPE_LONG;
let (hbits, fbits) = if large {
(EXSS_HDRSIZE_BITS_LONG, EXSS_FSIZE_BITS_LONG)
} else {
(EXSS_HDRSIZE_BITS_SHORT, EXSS_FSIZE_BITS_SHORT)
};
let _hdr = r.read_bits(hbits)?; // nuExtSSHeaderSize (not needed for framing)
let fsize_minus_one = r.read_bits(fbits)?; // nuExtSSFsize = total bytes - 1
Some(fsize_minus_one as usize + 1)
}
/// Offset where the current access unit ends (the start of the next core
/// frame). The AU is the core frame plus its trailing DTS-HD extension
/// substreams, which are skipped PRECISELY by their declared size — so a chance
/// core syncword inside the XLL lossless payload can never be mistaken for the
/// next AU boundary (the bug that truncated the extension and produced the
/// "Failed to decode block code(s)" class). Falls back to the heuristic core-sync
/// scan only when an extension can't be sized (malformed / truncated input).
fn next_core_boundary(buf: &[u8], core_size: usize) -> NextCore { fn next_core_boundary(buf: &[u8], core_size: usize) -> NextCore {
let mut from = core_size; let mut pos = core_size;
while let Some(rel) = find_sync(&buf[from..], &DTS_CORE_SYNC) { loop {
let pos = from + rel; if buf.len() < pos + SYNCWORD_BYTES {
// Need the candidate's core header to judge it. return NextCore::NeedMore; // need a syncword to identify the next chunk
}
if buf[pos..].starts_with(&DTS_HD_EXT_SYNC) {
match exss_frame_size(&buf[pos..]) {
Some(sz) if sz >= SYNCWORD_BYTES => {
if buf.len() < pos + sz {
return NextCore::NeedMore; // extension not fully buffered
}
pos += sz; // skip the whole extension substream precisely
}
// Couldn't size it (truncated/garbage header) — heuristic fallback.
_ => return scan_for_next_core(buf, pos),
}
} else if buf[pos..].starts_with(&DTS_CORE_SYNC) {
// The bytes right after the precisely-skipped extensions are the next
// core frame — the AU boundary.
if buf.len() - pos < CORE_HEADER_MIN_BYTES { if buf.len() - pos < CORE_HEADER_MIN_BYTES {
return NextCore::NeedMore; return NextCore::NeedMore;
} }
@@ -348,8 +421,31 @@ fn next_core_boundary(buf: &[u8], core_size: usize) -> NextCore {
if (MIN_CORE_FRAME_BYTES..=MAX_AU_BYTES).contains(&sz) { if (MIN_CORE_FRAME_BYTES..=MAX_AU_BYTES).contains(&sz) {
return NextCore::Found(pos); return NextCore::Found(pos);
} }
// False sync inside extension payload — skip it and keep searching. return scan_for_next_core(buf, pos); // implausible core here — fall back
from = pos + 4; } else {
// Neither a known extension nor a core sync at the precise boundary
// (padding / junk) — fall back to the heuristic scan.
return scan_for_next_core(buf, pos);
}
}
}
/// Heuristic fallback (the pre-fix behaviour): scan forward for the next core
/// syncword whose decoded size is plausible. Used only when precise extension
/// skipping can't proceed; a chance core syncword in extension payload usually
/// decodes to an implausible size and is skipped.
fn scan_for_next_core(buf: &[u8], from: usize) -> NextCore {
let mut from = from;
while let Some(rel) = find_sync(&buf[from..], &DTS_CORE_SYNC) {
let pos = from + rel;
if buf.len() - pos < CORE_HEADER_MIN_BYTES {
return NextCore::NeedMore;
}
let sz = dts_core_frame_size(&buf[pos..]);
if (MIN_CORE_FRAME_BYTES..=MAX_AU_BYTES).contains(&sz) {
return NextCore::Found(pos);
}
from = pos + SYNCWORD_BYTES;
} }
NextCore::None NextCore::None
} }
@@ -399,6 +495,55 @@ mod tests {
data data
} }
/// A real DTS-HD EXSS substream of `total` bytes (short header form), with an
/// optional false DTS core syncword embedded in its payload (decoding to a
/// plausible core size) — to prove precise sizing, not a payload scan, bounds
/// the extension.
fn make_exss(total: usize, false_core_at: Option<usize>) -> Vec<u8> {
let mut d = vec![0u8; total];
d[0..4].copy_from_slice(&DTS_HD_EXT_SYNC);
// Short form: all header fields 0 except nuExtSSFsize = total - 1, laid
// out at bit 19 after the sync (byte 6 low 5 bits, byte 7, byte 8 top 3).
let fsize = (total - 1) as u32;
d[6] = ((fsize >> 11) & 0x1F) as u8;
d[7] = ((fsize >> 3) & 0xFF) as u8;
d[8] = ((fsize & 0x07) << 5) as u8;
if let Some(at) = false_core_at {
d[at..at + 4].copy_from_slice(&DTS_CORE_SYNC);
let fcs = 512u32 - 1; // decode to a plausible core size — fools the heuristic
d[at + 5] = (d[at + 5] & 0xFC) | ((fcs >> 12) & 0x03) as u8;
d[at + 6] = ((fcs >> 4) & 0xFF) as u8;
d[at + 7] = (d[at + 7] & 0x0F) | (((fcs & 0x0F) << 4) as u8);
}
d
}
#[test]
fn plausible_false_core_sync_inside_real_exss_does_not_split_au() {
// EXSS size parse round-trips.
assert_eq!(exss_frame_size(&make_exss(600, None)), Some(600));
// AU = core(512) + a REAL EXSS substream whose XLL payload embeds a DTS
// core syncword decoding to a plausible size (512). The heuristic-only
// framer would split here and truncate the lossless extension (the
// Dunkirk `dca` "Failed to decode block code(s)" class). Precise EXSS
// sizing spans the whole extension to the REAL next core.
let core = make_dts_core(512);
let exss = make_exss(600, Some(40));
let next = make_dts_core(512);
let mut buf = core.clone();
buf.extend_from_slice(&exss);
buf.extend_from_slice(&next);
assert!(
matches!(
next_core_boundary(&buf, core.len()),
NextCore::Found(end) if end == core.len() + exss.len()
),
"AU must end at the REAL next core (after the full EXSS), not the false sync inside it"
);
}
#[test] #[test]
fn parse_empty_pes() { fn parse_empty_pes() {
let mut parser = DtsParser::new(); let mut parser = DtsParser::new();
+2 -1
View File
@@ -325,7 +325,8 @@ impl CodecParser for H264Parser {
} }
/// Parse `(chroma_format_idc, bit_depth_luma_minus8, bit_depth_chroma_minus8)` from /// Parse `(chroma_format_idc, bit_depth_luma_minus8, bit_depth_chroma_minus8)` from
/// a High-Profile SPS NAL (profile_idc ∈ {100, 110, 122, 144}). /// a High-Profile SPS NAL (profile_idc ∈ `HIGH_PROFILES` — the 14 chroma/bit-depth
/// extended profiles `codec_private` invokes this for).
/// ///
/// SPS RBSP layout (ITU-T H.264 §7.3.2.1.1) up to the fields we need: /// SPS RBSP layout (ITU-T H.264 §7.3.2.1.1) up to the fields we need:
/// byte 0 NAL header (already known to be type 7) /// byte 0 NAL header (already known to be type 7)
+1 -4
View File
@@ -56,10 +56,7 @@ fn hevc_num_extra_slice_header_bits(pps_nal: &[u8]) -> Option<u32> {
br.read_ue()?; // pps_pic_parameter_set_id br.read_ue()?; // pps_pic_parameter_set_id
br.read_ue()?; // pps_seq_parameter_set_id br.read_ue()?; // pps_seq_parameter_set_id
br.skip_bits(2)?; // dependent_slice_segments_enabled_flag, output_flag_present_flag br.skip_bits(2)?; // dependent_slice_segments_enabled_flag, output_flag_present_flag
let mut n = 0u32; let n = br.read_bits(3)?;
for _ in 0..3 {
n = (n << 1) | br.read_bit()?;
}
Some(n) Some(n)
} }
+42 -1
View File
@@ -138,7 +138,18 @@ impl CodecParser for TrueHdParser {
// the next PES legitimately begins a new AU and seeds the base. // the next PES legitimately begins a new AU and seeds the base.
if self.buf.is_empty() { if self.buf.is_empty() {
if let Some(pts) = pes.pts { if let Some(pts) = pes.pts {
self.next_pts_ns = pts_to_ns(pts); // Resync to the authoritative PES PTS, but NEVER snap backward.
// TrueHD AUs are a fixed sample count (40 @ 48 kHz), so the
// per-AU `+AU_DURATION_NS` cadence is sample-accurate — more so
// than the disc's per-PES PTS, which carries the source muxer's
// own rounding jitter. When the buffer empties exactly on a PES
// boundary and that PES's PTS lands a few ticks *below* the
// running cadence, an unconditional reset would set the next
// AU's timestamp below the AU just emitted, producing the
// non-monotonic block timestamps a muxer rejects. Clamp to the
// running position so output stays strictly monotonic; a
// genuine forward gap/discontinuity is still adopted.
self.next_pts_ns = self.next_pts_ns.max(pts_to_ns(pts));
} }
} }
@@ -449,6 +460,36 @@ mod tests {
assert_eq!(frames[1].pts_ns - frames[0].pts_ns, AU_DURATION_NS); assert_eq!(frames[1].pts_ns - frames[0].pts_ns, AU_DURATION_NS);
} }
#[test]
fn pes_pts_lagging_the_au_cadence_never_emits_backward() {
// Regression: the per-AU cadence is sample-accurate, but a PES boundary
// can carry a PTS that lags it slightly (source-muxer rounding jitter).
// When the buffer empties exactly on that boundary, an unconditional
// reset to the PES PTS snapped the next AU's timestamp BELOW the AU just
// emitted — the non-monotonic block timestamps a muxer rejects (the
// Top Gun / Dune: Part Two case). The reset must clamp forward-only.
let mut parser = TrueHdParser::new();
let au = make_truehd_unit(100);
// PES1: three complete AUs at pts 90000 — buffer empties, cadence runs
// ahead to 90000_ns + 3*AU_DURATION_NS.
let mut d1 = au.clone();
d1.extend_from_slice(&au);
d1.extend_from_slice(&au);
let f1 = parser.parse(&make_pes(d1, Some(90000)));
assert_eq!(f1.len(), 3);
let last1 = f1.last().unwrap().pts_ns;
// PES2's PTS (90001) maps to fewer ns than the running cadence — pre-fix
// this snapped backward.
let f2 = parser.parse(&make_pes(au.clone(), Some(90001)));
assert_eq!(f2.len(), 1);
assert!(
f2[0].pts_ns >= last1,
"AU pts must not go backward when PES PTS lags the cadence: got {} after {}",
f2[0].pts_ns,
last1
);
}
#[test] #[test]
fn skip_interleaved_ac3() { fn skip_interleaved_ac3() {
let mut parser = TrueHdParser::new(); let mut parser = TrueHdParser::new();
+3 -3
View File
@@ -273,8 +273,8 @@ const SUP_HEADER_LEN: usize = SUP_MAGIC.len() + 4 + 4;
const SEG_PCS: u8 = 0x16; const SEG_PCS: u8 = 0x16;
/// PGS segment type: END of display set. /// PGS segment type: END of display set.
const SEG_END: u8 = 0x80; const SEG_END: u8 = 0x80;
/// PCS `composition_state` value: Epoch Start (a fresh display). /// PCS `composition_state` value: Normal (an update to the current epoch).
const PCS_COMPOSITION_STATE_EPOCH_START: u8 = 0x80; const PCS_COMPOSITION_STATE_NORMAL: u8 = 0x00;
/// PGS segment header on the wire (inside `frame.data`): type(1) + size(2 BE). /// PGS segment header on the wire (inside `frame.data`): type(1) + size(2 BE).
const PGS_SEG_HEADER_LEN: usize = 3; const PGS_SEG_HEADER_LEN: usize = 3;
/// Byte offset of `width`/`height` within a PCS segment (after type+size). /// Byte offset of `width`/`height` within a PCS segment (after type+size).
@@ -346,7 +346,7 @@ impl PgsSupWriter {
PCS_FRAME_RATE, PCS_FRAME_RATE,
0x00, 0x00,
0x00, // composition_number 0x00, // composition_number
PCS_COMPOSITION_STATE_EPOCH_START, PCS_COMPOSITION_STATE_NORMAL,
0x00, // palette_update_flag 0x00, // palette_update_flag
0x00, // palette_id 0x00, // palette_id
PCS_NO_OBJECTS, PCS_NO_OBJECTS,
+13 -3
View File
@@ -82,8 +82,16 @@ fn write_fvi_record(w: &mut dyn Write, r: &PictureRecord) -> io::Result<()> {
// `src` is REQUIRED by the record schema (Appendix A); when provenance is // `src` is REQUIRED by the record schema (Appendix A); when provenance is
// absent the member is still emitted as null — a reader treats null as // absent the member is still emitted as null — a reader treats null as
// "position unknown". // "position unknown".
//
// Per `docs/FVI_FORMAT.md` §9, `src.byte` is the offset of the AU's first
// byte WITHIN its `sector` (not the absolute source offset). `SourcePos.byte`
// is the absolute offset, so reduce it modulo the sector size; `sector`
// already carries the whole-sector count.
let src = match r.source { let src = match r.source {
Some(s) => serde_json::json!({ "sector": s.sector, "byte": s.byte }), Some(s) => serde_json::json!({
"sector": s.sector,
"byte": s.byte % u64::from(FVI_SECTOR_SIZE),
}),
None => serde_json::Value::Null, None => serde_json::Value::Null,
}; };
@@ -355,8 +363,9 @@ mod tests {
let dir = tempdir(); let dir = tempdir();
let path = dir.join("movie.fvi"); let path = dir.join("movie.fvi");
let mut sink = FviSink::create(&path, &mpeg2_title(), "iso://m.iso".into(), 1).unwrap(); let mut sink = FviSink::create(&path, &mpeg2_title(), "iso://m.iso".into(), 1).unwrap();
// Video frame on track 0 → indexed. // Video frame on track 0 → indexed. Offset 2148 = sector 1, byte 100
sink.write(&vframe(0, Some(i_pic()), Some(SourcePos::at_byte(2048)))) // within that sector (exercises the within-sector `src.byte`, §9).
sink.write(&vframe(0, Some(i_pic()), Some(SourcePos::at_byte(2148))))
.unwrap(); .unwrap();
// Audio frame on a non-video track → ignored. // Audio frame on a non-video track → ignored.
sink.write(&vframe(7, None, Some(SourcePos::at_byte(9999)))) sink.write(&vframe(7, None, Some(SourcePos::at_byte(9999))))
@@ -387,6 +396,7 @@ mod tests {
assert_eq!(rec["nb_fields"], 2); assert_eq!(rec["nb_fields"], 2);
assert_eq!(rec["pts"], 0); assert_eq!(rec["pts"], 0);
assert_eq!(rec["src"]["sector"], 1); assert_eq!(rec["src"]["sector"], 1);
assert_eq!(rec["src"]["byte"], 100); // 2148 % 2048 → within-sector (§9)
assert!(rec.get("dts").is_none(), "no DTS on a frame → omitted"); assert!(rec.get("dts").is_none(), "no DTS on a frame → omitted");
assert!( assert!(
rec.get("gop").is_none(), rec.get("gop").is_none(),
+22 -6
View File
@@ -76,15 +76,18 @@ const PCR_INTERVAL_PACKETS: u64 = 40;
/// the picture it timestamps. 200 ms in 90 kHz ticks. /// the picture it timestamps. 200 ms in 90 kHz ticks.
const PCR_LEAD_90KHZ: u64 = 90_000 / 5; const PCR_LEAD_90KHZ: u64 = 90_000 / 5;
// PMT stream-type codes. These are the same elementary-stream coding-type
// registry as the parse side; the single source of truth is `consts::coding_type`.
use crate::consts::coding_type;
/// HEVC stream-type code, ISO/IEC 13818-1 Table 2-34 (2015 amendment). /// HEVC stream-type code, ISO/IEC 13818-1 Table 2-34 (2015 amendment).
const STREAM_TYPE_HEVC: u8 = 0x24; const STREAM_TYPE_HEVC: u8 = coding_type::HEVC;
/// AC-3 / E-AC-3. Not an ISO assignment — sits in the user-private /// AC-3 / E-AC-3. Not an ISO assignment — sits in the user-private
/// 0x80-0xFF range and is the Blu-ray Disc Association / ATSC A/52 /// 0x80-0xFF range and is the Blu-ray Disc Association / ATSC A/52
/// convention. /// convention.
const STREAM_TYPE_AC3: u8 = 0x81; const STREAM_TYPE_AC3: u8 = coding_type::AC3;
/// Dolby TrueHD. Also a private/BD-conventional value in the /// Dolby TrueHD. Also a private/BD-conventional value in the
/// user-private 0x80-0xFF range, not an ISO assignment. /// user-private 0x80-0xFF range, not an ISO assignment.
const STREAM_TYPE_TRUEHD: u8 = 0x83; const STREAM_TYPE_TRUEHD: u8 = coding_type::TRUEHD;
/// Audio codec hint for [`M2tsMux::new`] / [`M2tsMux::set_audio`]. The /// Audio codec hint for [`M2tsMux::new`] / [`M2tsMux::set_audio`]. The
/// muxer needs to know the codec to pick the right PMT `stream_type` /// muxer needs to know the codec to pick the right PMT `stream_type`
@@ -251,7 +254,10 @@ impl<W: Write> M2tsMux<W> {
// step — e.g. a leading audio frame ahead of the first video keyframe), // step — e.g. a leading audio frame ahead of the first video keyframe),
// which still floors to 0 per the documented behavior. // which still floors to 0 per the documented behavior.
let delta = raw_90k.wrapping_sub(base) & 0x1_FFFF_FFFF; let delta = raw_90k.wrapping_sub(base) & 0x1_FFFF_FFFF;
if delta > (1 << 32) { 0 } else { delta } // Signed 33-bit: the sign bit is bit 32 (value 2^32), so the entire
// upper half [2^32, 2^33) is negative (frame before base) and floors
// to 0. delta == 2^32 is the most-negative value (-2^32), hence `>=`.
if delta >= (1 << 32) { 0 } else { delta }
} }
/// Emit one PES payload as a chain of TS packets on `pid`. If `pcr` /// Emit one PES payload as a chain of TS packets on `pid`. If `pcr`
@@ -420,7 +426,12 @@ impl<W: Write> M2tsMux<W> {
/// Build a PES packet for a video access unit. /// Build a PES packet for a video access unit.
fn build_video_pes(pts_90k: u64, es: &[u8]) -> Vec<u8> { fn build_video_pes(pts_90k: u64, es: &[u8]) -> Vec<u8> {
build_pes_packet(0xE0, pts_90k, es, /* length_in_header */ false) build_pes_packet(
crate::consts::pes_stream_id::VIDEO,
pts_90k,
es,
/* length_in_header */ false,
)
} }
/// Build a PES packet for an audio access unit. /// Build a PES packet for an audio access unit.
@@ -430,7 +441,12 @@ fn build_audio_pes(pts_90k: u64, es: &[u8]) -> Vec<u8> {
// start code. For an access unit larger than ~64 KiB (rare — e.g. a // start code. For an access unit larger than ~64 KiB (rare — e.g. a
// large TrueHD frame) the length field falls back to the unbounded // large TrueHD frame) the length field falls back to the unbounded
// (0x0000) form, which most demuxers tolerate for private_stream_1. // (0x0000) form, which most demuxers tolerate for private_stream_1.
build_pes_packet(0xBD, pts_90k, es, /* length_in_header */ true) build_pes_packet(
crate::consts::pes_stream_id::PRIVATE_STREAM_1,
pts_90k,
es,
/* length_in_header */ true,
)
} }
fn build_pes_packet(stream_id: u8, pts_90k: u64, es: &[u8], length_in_header: bool) -> Vec<u8> { fn build_pes_packet(stream_id: u8, pts_90k: u64, es: &[u8], length_in_header: bool) -> Vec<u8> {
+55 -4
View File
@@ -28,6 +28,9 @@ const CICP_PRIMARIES_BT470BG: u8 = 5;
const CICP_PRIMARIES_BT601_525: u8 = 6; const CICP_PRIMARIES_BT601_525: u8 = 6;
/// ColourPrimaries = 9 (BT.2020 / BT.2100) — ITU-T H.273 Table 2. /// ColourPrimaries = 9 (BT.2020 / BT.2100) — ITU-T H.273 Table 2.
const CICP_PRIMARIES_BT2020: u8 = 9; const CICP_PRIMARIES_BT2020: u8 = 9;
/// ColourPrimaries = 2 ("unspecified" — colorimetry unknown) — ITU-T H.273
/// Table 2.
const CICP_PRIMARIES_UNSPECIFIED: u8 = 2;
/// TransferCharacteristics = 1 (BT.709) — ITU-T H.273 Table 3. /// TransferCharacteristics = 1 (BT.709) — ITU-T H.273 Table 3.
const CICP_TRANSFER_BT709: u8 = 1; const CICP_TRANSFER_BT709: u8 = 1;
@@ -41,6 +44,9 @@ const CICP_TRANSFER_PQ: u8 = 16;
/// TransferCharacteristics = 18 (ARIB STD-B67 / Hybrid Log-Gamma) — ITU-T H.273 /// TransferCharacteristics = 18 (ARIB STD-B67 / Hybrid Log-Gamma) — ITU-T H.273
/// Table 3. /// Table 3.
const CICP_TRANSFER_HLG: u8 = 18; const CICP_TRANSFER_HLG: u8 = 18;
/// TransferCharacteristics = 2 ("unspecified" — transfer unknown) — ITU-T H.273
/// Table 3.
const CICP_TRANSFER_UNSPECIFIED: u8 = 2;
/// MatrixCoefficients = 1 (BT.709) — ITU-T H.273 Table 4. /// MatrixCoefficients = 1 (BT.709) — ITU-T H.273 Table 4.
const CICP_MATRIX_BT709: u8 = 1; const CICP_MATRIX_BT709: u8 = 1;
@@ -50,6 +56,8 @@ const CICP_MATRIX_BT470BG: u8 = 5;
const CICP_MATRIX_BT601_525: u8 = 6; const CICP_MATRIX_BT601_525: u8 = 6;
/// MatrixCoefficients = 9 (BT.2020 non-constant luminance) — ITU-T H.273 Table 4. /// MatrixCoefficients = 9 (BT.2020 non-constant luminance) — ITU-T H.273 Table 4.
const CICP_MATRIX_BT2020NC: u8 = 9; const CICP_MATRIX_BT2020NC: u8 = 9;
/// MatrixCoefficients = 2 ("unspecified" — matrix unknown) — ITU-T H.273 Table 4.
const CICP_MATRIX_UNSPECIFIED: u8 = 2;
/// Matroska Colour/Range = 1 (broadcast / studio-swing "limited" range). RFC /// Matroska Colour/Range = 1 (broadcast / studio-swing "limited" range). RFC
/// 9559 Range element. (0 = unspecified, 2 = full.) /// 9559 Range element. (0 = unspecified, 2 = full.)
@@ -104,7 +112,16 @@ pub(crate) fn cicp_for_video(v: &VideoStream) -> (u8, u8, u8, u8) {
CICP_PRIMARIES_BT601_525, CICP_PRIMARIES_BT601_525,
COLOUR_RANGE_LIMITED, COLOUR_RANGE_LIMITED,
), ),
ColorSpace::Unknown => (0, 0, 0, 0), // Unknown colorimetry → CICP "unspecified" (code point 2) for matrix,
// transfer, and primaries, with limited range (the disc norm). Both the
// MKV sink and the FVI sidecar emit 2 so the two sinks of one title
// agree (matches `Colour::from_color_space`'s Unknown mapping).
ColorSpace::Unknown => (
CICP_MATRIX_UNSPECIFIED,
CICP_TRANSFER_UNSPECIFIED,
CICP_PRIMARIES_UNSPECIFIED,
COLOUR_RANGE_LIMITED,
),
}; };
// Override the transfer for HDR signalled by the HdrFormat (the coarse enum // Override the transfer for HDR signalled by the HdrFormat (the coarse enum
// can't express PQ/HLG). Only applies on the enum fallback; a measured CICP // can't express PQ/HLG). Only applies on the enum fallback; a measured CICP
@@ -756,9 +773,8 @@ impl<W: Write + Seek> MkvMuxer<W> {
} }
// Stamp the freemkv version so any muxed file is traceable to the build // Stamp the freemkv version so any muxed file is traceable to the build
// that produced it (MediaInfo "Writing application"/"library"). // that produced it (MediaInfo "Writing application"/"library").
const FREEMKV_MUX_APP: &str = concat!("freemkv ", env!("CARGO_PKG_VERSION")); ebml::write_string(&mut writer, ebml::MUXING_APP, crate::MUX_APP)?;
ebml::write_string(&mut writer, ebml::MUXING_APP, FREEMKV_MUX_APP)?; ebml::write_string(&mut writer, ebml::WRITING_APP, crate::MUX_APP)?;
ebml::write_string(&mut writer, ebml::WRITING_APP, FREEMKV_MUX_APP)?;
if let Some(t) = title { if let Some(t) = title {
ebml::write_string(&mut writer, ebml::TITLE, t)?; ebml::write_string(&mut writer, ebml::TITLE, t)?;
} }
@@ -1549,6 +1565,41 @@ mod tests {
); );
} }
/// Unknown colorimetry with no measured CICP and no HDR must emit CICP
/// "unspecified" (code point 2) for matrix/transfer/primaries — never 0 — so
/// the MKV sink agrees with the FVI sidecar (`Colour::from_color_space`).
#[test]
fn unknown_color_space_emits_unspecified_cicp() {
let v = VideoStream {
pid: 0xE0,
codec: Codec::Hevc,
resolution: Resolution::R1080p,
frame_rate: crate::disc::FrameRate::F24,
hdr: HdrFormat::Sdr,
color_space: ColorSpace::Unknown,
display_aspect: None,
secondary: false,
label: String::new(),
measured_cicp: None,
};
let t = MkvTrack::video(&v);
assert_eq!(
(
t.colour_matrix,
t.colour_transfer,
t.colour_primaries,
t.colour_range
),
(
CICP_MATRIX_UNSPECIFIED,
CICP_TRANSFER_UNSPECIFIED,
CICP_PRIMARIES_UNSPECIFIED,
COLOUR_RANGE_LIMITED
),
"Unknown colorimetry must emit CICP 'unspecified' (2), not 0"
);
}
/// Helper: search for a 4-byte big-endian EBML ID in a byte slice. /// Helper: search for a 4-byte big-endian EBML ID in a byte slice.
fn find_id(data: &[u8], id: u32) -> Option<usize> { fn find_id(data: &[u8], id: u32) -> Option<usize> {
let bytes = id.to_be_bytes(); let bytes = id.to_be_bytes();
+68 -24
View File
@@ -101,8 +101,12 @@ enum WriteMode {
/// Header written; muxing live. Boxed (MkvMuxer is large) to keep the enum /// Header written; muxing live. Boxed (MkvMuxer is large) to keep the enum
/// small (clippy::large_enum_variant). /// small (clippy::large_enum_variant).
Active(Box<MkvMuxer<Box<dyn WriteSeek + Send>>>), Active(Box<MkvMuxer<Box<dyn WriteSeek + Send>>>),
/// Transient placeholder held only across the Pending → Active swap; never /// Sentinel held in `self.mode` while the muxer is being built (across the
/// observed by `read` / `write` / `finish`. /// Pending → Active swap). It is also the terminal state left behind after
/// `finish()` swaps the muxer out, and the degraded state left behind if
/// `activate()` fails partway (the first error still surfaces via `?`). In
/// that terminal state a subsequent `write()` no-ops (`Ok(())`) and `finish()`
/// does not re-finalize.
Building, Building,
} }
@@ -188,7 +192,11 @@ impl MkvStream {
/// available), then write the header and replay buffered frames. A no-op if /// available), then write the header and replay buffered frames. A no-op if
/// not pending. The muxer only ever muxes the track it is given — this routes /// not pending. The muxer only ever muxes the track it is given — this routes
/// the parser's measured value onto that track first. /// the parser's measured value onto that track first.
fn activate(&mut self, coding: Option<crate::mux::codec::PictureInfo>) -> io::Result<()> { fn activate(
&mut self,
coding: Option<crate::mux::codec::PictureInfo>,
video_picture_seen: bool,
) -> io::Result<()> {
let mut pending = match std::mem::replace(&mut self.mode, Mode::Write(WriteMode::Building)) let mut pending = match std::mem::replace(&mut self.mode, Mode::Write(WriteMode::Building))
{ {
Mode::Write(WriteMode::Pending(p)) => p, Mode::Write(WriteMode::Pending(p)) => p,
@@ -199,7 +207,7 @@ impl MkvStream {
} }
}; };
if let Some(vt) = pending.video_track { if let Some(vt) = pending.video_track {
apply_coding_to_track(&mut pending.tracks[vt], coding); apply_coding_to_track(&mut pending.tracks[vt], coding, video_picture_seen);
} }
// --log-level 3: dump the FINAL TrackEntry metadata (field order set). // --log-level 3: dump the FINAL TrackEntry metadata (field order set).
for (i, track) in pending.tracks.iter().enumerate() { for (i, track) in pending.tracks.iter().enumerate() {
@@ -240,12 +248,20 @@ impl MkvStream {
/// Set a video track's `FieldOrder` from the MEASURED coding of the first coded /// Set a video track's `FieldOrder` from the MEASURED coding of the first coded
/// picture — the parser's value, the first time, never a guess. /// picture — the parser's value, the first time, never a guess.
/// ///
/// A progressive track has no field order (left UNDETERMINED — expected). An /// A progressive track — or a progressive picture on an interlaced-flagged track
/// INTERLACED track that reaches here with no measured field order is a /// — has no field order (left UNDETERMINED — expected). An INTERLACED track that
/// reaches here WITH a video picture but no measured field order is a
/// parser/source gap (MPEG-2 carries `top_field_first` on every interlaced /// parser/source gap (MPEG-2 carries `top_field_first` on every interlaced
/// picture, so it should never be missing): LOG it loudly so the source can be /// picture, so it should never be missing): LOG it loudly so the source can be
/// debugged, and leave UNDETERMINED — a muxer never fabricates a source fact. /// debugged, and leave UNDETERMINED — a muxer never fabricates a source fact.
fn apply_coding_to_track(track: &mut MkvTrack, coding: Option<crate::mux::codec::PictureInfo>) { /// `video_picture_seen == false` (an empty title finalized with no frames, or a
/// cap-triggered build that never saw the video frame) is NOT a defect — the
/// missing coding is expected there, so log it quietly.
fn apply_coding_to_track(
track: &mut MkvTrack,
coding: Option<crate::mux::codec::PictureInfo>,
video_picture_seen: bool,
) {
// HDR10 static metadata measured from the bitstream (HEVC SEI). Applied for // HDR10 static metadata measured from the bitstream (HEVC SEI). Applied for
// ANY track type that carries it (independent of interlace): the first coded // ANY track type that carries it (independent of interlace): the first coded
// picture's PictureInfo holds it once both HDR10 SEI messages were seen. // picture's PictureInfo holds it once both HDR10 SEI messages were seen.
@@ -260,17 +276,32 @@ fn apply_coding_to_track(track: &mut MkvTrack, coding: Option<crate::mux::codec:
match coding.and_then(|c| c.field_order()) { match coding.and_then(|c| c.field_order()) {
Some(FieldOrder::Tff) => track.field_order = ebml::FIELD_ORDER_TFF, Some(FieldOrder::Tff) => track.field_order = ebml::FIELD_ORDER_TFF,
Some(FieldOrder::Bff) => track.field_order = ebml::FIELD_ORDER_BFF, Some(FieldOrder::Bff) => track.field_order = ebml::FIELD_ORDER_BFF,
other => { // A progressive picture on an interlaced-flagged track carries no field
// order. Leave UNDETERMINED (not a guess) — there is no parser gap here.
Some(FieldOrder::Progressive) => {
track.field_order = ebml::FIELD_ORDER_UNDETERMINED;
}
None if video_picture_seen => {
tracing::warn!( tracing::warn!(
target: "mux", target: "mux",
"interlaced video track reached the muxer with NO measured field order \ "interlaced video track had a video picture but NO usable field order \
(field_order={:?}, coding_present={}); writing FieldOrder=UNDETERMINED \ (coding_present={}); writing FieldOrder=UNDETERMINED NOT a guess. \
NOT a guess. Debug why the source/parser did not set top_field_first.", Debug why the source/parser did not set top_field_first.",
other,
coding.is_some(), coding.is_some(),
); );
track.field_order = ebml::FIELD_ORDER_UNDETERMINED; track.field_order = ebml::FIELD_ORDER_UNDETERMINED;
} }
None => {
// No video picture was ever measured (empty title finalized with no
// frames, or a cap-triggered build before the first video frame).
// Coding is legitimately absent, not a parser defect — log quietly.
tracing::debug!(
target: "mux",
"interlaced video track activated with no video picture \
(empty/buffered-only title); writing FieldOrder=UNDETERMINED.",
);
track.field_order = ebml::FIELD_ORDER_UNDETERMINED;
}
} }
} }
@@ -427,7 +458,7 @@ impl crate::pes::Stream for MkvStream {
// Pass the trigger frame's coding only when it IS the video frame; a // Pass the trigger frame's coding only when it IS the video frame; a
// cap-triggered build never saw the video frame, so nothing measured // cap-triggered build never saw the video frame, so nothing measured
// is passed (apply_coding_to_track then logs + leaves UNDETERMINED). // is passed (apply_coding_to_track then logs + leaves UNDETERMINED).
self.activate(if use_coding { frame.coding } else { None })?; self.activate(if use_coding { frame.coding } else { None }, use_coding)?;
if let Mode::Write(WriteMode::Active(m)) = &mut self.mode { if let Mode::Write(WriteMode::Active(m)) = &mut self.mode {
return m.write_frame( return m.write_frame(
frame.track, frame.track,
@@ -450,7 +481,10 @@ impl crate::pes::Stream for MkvStream {
// A title that produced no frames (or only buffered ones) is still // A title that produced no frames (or only buffered ones) is still
// finalized into a valid MKV: activate now with no measured coding. // finalized into a valid MKV: activate now with no measured coding.
if matches!(self.mode, Mode::Write(WriteMode::Pending(_))) { if matches!(self.mode, Mode::Write(WriteMode::Pending(_))) {
self.activate(None)?; // No video picture was ever measured for this title (it produced no
// frames, or only buffered non-video ones): coding is legitimately
// absent, not a parser defect — `video_picture_seen=false`.
self.activate(None, false)?;
} }
if let Mode::Write(WriteMode::Active(m)) = if let Mode::Write(WriteMode::Active(m)) =
std::mem::replace(&mut self.mode, Mode::Write(WriteMode::Building)) std::mem::replace(&mut self.mode, Mode::Write(WriteMode::Building))
@@ -923,7 +957,7 @@ mod tests {
// MEASURED bottom-field-first → BFF (6). The red-flag fix. // MEASURED bottom-field-first → BFF (6). The red-flag fix.
let mut t = interlaced_track(); let mut t = interlaced_track();
apply_coding_to_track(&mut t, Some(pic(false, false))); apply_coding_to_track(&mut t, Some(pic(false, false)), true);
assert_eq!( assert_eq!(
t.field_order, t.field_order,
ebml::FIELD_ORDER_BFF, ebml::FIELD_ORDER_BFF,
@@ -932,27 +966,37 @@ mod tests {
// MEASURED top-field-first → TFF (1). // MEASURED top-field-first → TFF (1).
let mut t = interlaced_track(); let mut t = interlaced_track();
apply_coding_to_track(&mut t, Some(pic(true, false))); apply_coding_to_track(&mut t, Some(pic(true, false)), true);
assert_eq!( assert_eq!(
t.field_order, t.field_order,
ebml::FIELD_ORDER_TFF, ebml::FIELD_ORDER_TFF,
"measured TFF → FieldOrder=1" "measured TFF → FieldOrder=1"
); );
// Interlaced track, NO measured coding → UNDETERMINED (logged loudly, // Interlaced track, a video picture but NO usable field order →
// never faked). // UNDETERMINED (logged loudly, never faked).
let mut t = interlaced_track(); let mut t = interlaced_track();
apply_coding_to_track(&mut t, None); apply_coding_to_track(&mut t, None, true);
assert_eq!( assert_eq!(
t.field_order, t.field_order,
ebml::FIELD_ORDER_UNDETERMINED, ebml::FIELD_ORDER_UNDETERMINED,
"no measured value → UNDETERMINED, never a guess" "no measured value → UNDETERMINED, never a guess"
); );
// Interlaced track activated with NO video picture (empty/buffered-only
// title) → UNDETERMINED, logged quietly (not a parser defect).
let mut t = interlaced_track();
apply_coding_to_track(&mut t, None, false);
assert_eq!(
t.field_order,
ebml::FIELD_ORDER_UNDETERMINED,
"empty title → UNDETERMINED, never a guess"
);
// Progressive picture on an interlaced-flagged track → UNDETERMINED (no // Progressive picture on an interlaced-flagged track → UNDETERMINED (no
// field order applies; not faked to TFF/BFF). // field order applies; not faked to TFF/BFF).
let mut t = interlaced_track(); let mut t = interlaced_track();
apply_coding_to_track(&mut t, Some(pic(true, true))); apply_coding_to_track(&mut t, Some(pic(true, true)), true);
assert_eq!(t.field_order, ebml::FIELD_ORDER_UNDETERMINED); assert_eq!(t.field_order, ebml::FIELD_ORDER_UNDETERMINED);
// A PROGRESSIVE track is never touched — field order stays UNDETERMINED. // A PROGRESSIVE track is never touched — field order stays UNDETERMINED.
@@ -969,7 +1013,7 @@ mod tests {
measured_cicp: None, measured_cicp: None,
}); });
assert!(!prog.interlaced); assert!(!prog.interlaced);
apply_coding_to_track(&mut prog, Some(pic(false, false))); apply_coding_to_track(&mut prog, Some(pic(false, false)), true);
assert_eq!(prog.field_order, ebml::FIELD_ORDER_UNDETERMINED); assert_eq!(prog.field_order, ebml::FIELD_ORDER_UNDETERMINED);
} }
@@ -1011,18 +1055,18 @@ mod tests {
let mut t = make(); let mut t = make();
assert!(t.hdr10.is_none(), "fresh track has no HDR10"); assert!(t.hdr10.is_none(), "fresh track has no HDR10");
let pic = PictureInfo::coding_type_only(CodingType::I).with_hdr10(Some(h)); let pic = PictureInfo::coding_type_only(CodingType::I).with_hdr10(Some(h));
apply_coding_to_track(&mut t, Some(pic)); apply_coding_to_track(&mut t, Some(pic), true);
assert_eq!(t.hdr10, Some(h), "measured HDR10 must reach the track"); assert_eq!(t.hdr10, Some(h), "measured HDR10 must reach the track");
// Picture without HDR10 → track stays None (never fabricated). // Picture without HDR10 → track stays None (never fabricated).
let mut t = make(); let mut t = make();
let pic = PictureInfo::coding_type_only(CodingType::I); let pic = PictureInfo::coding_type_only(CodingType::I);
apply_coding_to_track(&mut t, Some(pic)); apply_coding_to_track(&mut t, Some(pic), true);
assert!(t.hdr10.is_none(), "no measured HDR10 → track stays None"); assert!(t.hdr10.is_none(), "no measured HDR10 → track stays None");
// No coding at all → None. // No coding at all → None.
let mut t = make(); let mut t = make();
apply_coding_to_track(&mut t, None); apply_coding_to_track(&mut t, None, true);
assert!(t.hdr10.is_none()); assert!(t.hdr10.is_none());
} }
+8 -8
View File
@@ -23,10 +23,10 @@ const SYSTEM_HEADER_ID: u8 = 0xBB;
const PROGRAM_END_ID: u8 = 0xB9; const PROGRAM_END_ID: u8 = 0xB9;
/// Private stream 1 (AC3, DTS, LPCM, subtitles). /// Private stream 1 (AC3, DTS, LPCM, subtitles).
const PRIVATE_STREAM_1: u8 = 0xBD; const PRIVATE_STREAM_1: u8 = crate::consts::pes_stream_id::PRIVATE_STREAM_1;
/// Private stream 2 (0xBF) — DVD navigation (PCI/DSI). Carries no muxable /// Private stream 2 (0xBF) — DVD navigation (PCI/DSI). Carries no muxable
/// elementary stream; expected to be dropped on every disc. /// elementary stream; expected to be dropped on every disc.
const PRIVATE_STREAM_2: u8 = 0xBF; const PRIVATE_STREAM_2: u8 = crate::consts::pes_stream_id::PRIVATE_STREAM_2;
/// Hard cap on the demuxer's reassembly buffer. A length-0 (unbounded) video /// Hard cap on the demuxer's reassembly buffer. A length-0 (unbounded) video
/// PES is delimited by the next PS-layer boundary; if a corrupt stream declares /// PES is delimited by the next PS-layer boundary; if a corrupt stream declares
@@ -107,8 +107,8 @@ impl PsPacket {
/// mis-routing the packet. /// mis-routing the packet.
pub fn dvd_pid(&self) -> Option<u16> { pub fn dvd_pid(&self) -> Option<u16> {
match self.stream_id { match self.stream_id {
0xE0..=0xEF => Some(DVD_VIDEO_PID), crate::consts::pes_stream_id::VIDEO..=0xEF => Some(DVD_VIDEO_PID),
0xBD => { PRIVATE_STREAM_1 => {
let sub = self.sub_stream_id?; let sub = self.sub_stream_id?;
dvd_audio_pid(sub).or_else(|| dvd_subtitle_pid(sub)) dvd_audio_pid(sub).or_else(|| dvd_subtitle_pid(sub))
} }
@@ -348,8 +348,8 @@ fn find_ps_boundary(data: &[u8], from: usize) -> Option<usize> {
fn is_pes_stream_id(id: u8) -> bool { fn is_pes_stream_id(id: u8) -> bool {
// Video: 0xE0-0xEF, MPEG audio: 0xC0-0xDF, private stream 1: 0xBD, // Video: 0xE0-0xEF, MPEG audio: 0xC0-0xDF, private stream 1: 0xBD,
// private stream 2: 0xBF, padding: 0xBE, ECM/EMM etc. // private stream 2: 0xBF, padding: 0xBE, ECM/EMM etc.
// We parse anything in the PES range. // We parse anything in the payload-bearing PES range.
matches!(id, 0xBD..=0xEF) crate::consts::pes_stream_id::PAYLOAD_RANGE.contains(&id)
} }
/// Parse a single PES packet from a byte slice that starts at the start code. /// Parse a single PES packet from a byte slice that starts at the start code.
@@ -365,12 +365,12 @@ fn parse_pes_packet(data: &[u8]) -> Option<PsPacket> {
let stream_id = data[3]; let stream_id = data[3];
// Padding stream — skip entirely. // Padding stream — skip entirely.
if stream_id == 0xBE { if stream_id == crate::consts::pes_stream_id::PADDING_STREAM {
return None; return None;
} }
// Streams without standard PES header extension. // Streams without standard PES header extension.
if stream_id == 0xBF { if stream_id == PRIVATE_STREAM_2 {
let payload = if data.len() > 6 { &data[6..] } else { &[] }; let payload = if data.len() > 6 { &data[6..] } else { &[] };
return Some(PsPacket { return Some(PsPacket {
stream_id, stream_id,
+25
View File
@@ -12,6 +12,7 @@
//! | network:// | Yes (listen) | Yes (connect) | host:port (required) | //! | network:// | Yes (listen) | Yes (connect) | host:port (required) |
//! | stdio:// | Yes (stdin) | Yes (stdout) | empty | //! | stdio:// | Yes (stdin) | Yes (stdout) | empty |
//! | null:// | -- | Yes | empty | //! | null:// | -- | Yes | empty |
//! | demux:// | -- | Yes | directory path (required) — per-track ES demux |
//! | fvi:// | -- | Yes | file path (required) — per-picture video index | //! | fvi:// | -- | Yes | file path (required) — per-picture video index |
//! //!
//! Bare paths without a scheme are rejected. //! Bare paths without a scheme are rejected.
@@ -920,6 +921,30 @@ mod tests {
!parse_url("dir://x").is_disc_source(), !parse_url("dir://x").is_disc_source(),
"dir:// is a sink, never a disc source" "dir:// is a sink, never a disc source"
); );
// fvi:// parses to Fvi with the raw remainder as the path, and is a
// sink (never a disc source) — parallel to the demux:// coverage above.
match parse_url("fvi://out/movie.fvi") {
StreamUrl::Fvi { path } => {
assert_eq!(path, PathBuf::from("out/movie.fvi"));
}
other => panic!("fvi:// must parse to Fvi, got {other:?}"),
}
assert_eq!(parse_url("fvi://x").scheme(), "fvi");
assert_eq!(parse_url("fvi://x/y.fvi").path_str(), "x/y.fvi");
assert!(
!parse_url("fvi://x").is_disc_source(),
"fvi:// is a sink, never a disc source"
);
}
/// `fvi://` is output-only: `input()` rejects it with StreamWriteOnly
/// (E9001 → Unsupported), mirroring `null://` / `demux://`.
#[test]
fn input_fvi_url_is_write_only() {
assert_eq!(
input_err_kind("fvi://out/movie.fvi"),
std::io::ErrorKind::Unsupported
);
} }
/// `dir://` is output-only: `input()` rejects it (StreamWriteOnly → /// `dir://` is output-only: `input()` rejects it (StreamWriteOnly →
+88 -7
View File
@@ -229,6 +229,12 @@ impl TsDemuxer {
/// `data` in place. Zero-copy on the bulk path; one 192-byte copy /// `data` in place. Zero-copy on the bulk path; one 192-byte copy
/// on the boundary. /// on the boundary.
pub fn feed(&mut self, data: &[u8]) -> Vec<PesPacket> { pub fn feed(&mut self, data: &[u8]) -> Vec<PesPacket> {
// A plain feed carries no provenance. Reset any base a prior
// `feed_at` left behind so mixing the two entry points is safe:
// after this call no `SourcePos` is stamped, and the stale running
// base can't leak a wrong offset into the boundary packet.
self.feed_base = 0;
self.has_base = false;
self.feed_inner(data) self.feed_inner(data)
} }
@@ -263,15 +269,20 @@ impl TsDemuxer {
self.remainder.extend_from_slice(data); self.remainder.extend_from_slice(data);
return completed; return completed;
} }
// Capture the remainder length before clearing — it's how many of
// the boundary packet's bytes lived in the PREVIOUS feed buffer,
// and `feed_base` currently points at the FIRST byte of THIS buffer.
let rem_len = self.remainder.len();
let mut boundary = [0u8; BD_SOURCE_PACKET_BYTES]; let mut boundary = [0u8; BD_SOURCE_PACKET_BYTES];
boundary[..self.remainder.len()].copy_from_slice(&self.remainder); boundary[..rem_len].copy_from_slice(&self.remainder);
boundary[self.remainder.len()..].copy_from_slice(&data[..need]); boundary[rem_len..].copy_from_slice(&data[..need]);
self.remainder.clear(); self.remainder.clear();
// The boundary packet began in the PREVIOUS feed buffer; stamp it // The boundary packet's first byte sat `rem_len` bytes before the
// with the offset just before this buffer (its first bytes' base). // current feed_base (in the previous buffer). Stamp it there — not
let src = self // at `feed_base - 1`, which would be wrong by `rem_len - 1` bytes.
.has_base let src = self.has_base.then(|| {
.then(|| crate::pes::SourcePos::at_byte(self.feed_base.saturating_sub(1))); crate::pes::SourcePos::at_byte(self.feed_base.saturating_sub(rem_len as u64))
});
self.process_packet(&boundary, src, &mut completed); self.process_packet(&boundary, src, &mut completed);
offset = need; offset = need;
} }
@@ -931,6 +942,76 @@ mod tests {
assert!(result.is_empty()); assert!(result.is_empty());
} }
/// A boundary packet (one split across two feeds) must be stamped with the
/// source offset of its FIRST byte, which sat `remainder.len()` bytes before
/// the current feed's base — not at `feed_base - 1`. We feed two 192-byte
/// packets via `feed_at`, splitting mid-second-packet so the second packet
/// is reassembled at the boundary, and assert its provenance lands exactly
/// on its first byte.
#[test]
fn boundary_packet_source_is_first_byte_not_base_minus_one() {
let pid = 0x1011;
let base: u64 = 20480; // sector-aligned (10 × 2048)
let mut demux = TsDemuxer::new(&[pid]);
let pkt0 = ts_payload_packet(pid, true, 0, &pes_start(b"AAAA"));
let pkt1 = ts_payload_packet(pid, true, 1, &pes_start(b"BBBB"));
let mut full = pkt0;
full.extend_from_slice(&pkt1);
// Split mid-pkt1 → pkt1 is reassembled from a 100-byte remainder + the
// next feed's head. pkt1's first byte is at absolute offset base + 192.
let split = BD_SOURCE_PACKET_BYTES + 100;
let out1 = demux.feed_at(base, &full[..split]);
assert!(out1.is_empty(), "pkt0's PES is still open");
// Second feed carries the rest; data[0] is at absolute base + split.
let out2 = demux.feed_at(base + split as u64, &full[split..]);
// pkt1 (PUSI) flushes pkt0's "AAAA" PES, stamped at pkt0's first byte.
assert_eq!(out2.len(), 1, "pkt0's PES completes when pkt1 starts");
assert_eq!(
out2[0].source.map(|s| s.byte),
Some(base),
"AAAA PES provenance is pkt0's first byte"
);
// Flush emits pkt1's "BBBB" PES — its source is the boundary stamp.
let out3 = demux.flush();
assert_eq!(out3.len(), 1, "pkt1's PES flushes out");
assert_eq!(
out3[0].source.map(|s| s.byte),
Some(base + BD_SOURCE_PACKET_BYTES as u64),
"boundary packet provenance must be its first byte (base + 192), \
not feed_base - 1"
);
}
/// Owner decision #7: a plain `feed()` must reset/ignore any base a prior
/// `feed_at()` left behind, so mixing the two is safe. After a `feed_at`
/// primes a base, the next plain `feed` must stamp `None` on PES packets it
/// begins.
#[test]
fn plain_feed_resets_prior_feed_at_base() {
let pid = 0x1011;
let mut demux = TsDemuxer::new(&[pid]);
// Prime a base via feed_at; pkt0's PES stays open.
let out1 = demux.feed_at(20480, &ts_payload_packet(pid, true, 0, &pes_start(b"AAAA")));
assert!(out1.is_empty());
// Plain feed must clear the base. pkt1 (PUSI) flushes "AAAA" (which was
// stamped during feed_at) and starts "BBBB" with NO provenance.
let out2 = demux.feed(&ts_payload_packet(pid, true, 1, &pes_start(b"BBBB")));
assert_eq!(out2.len(), 1, "AAAA completes");
let out3 = demux.flush();
assert_eq!(out3.len(), 1, "BBBB flushes");
assert_eq!(
out3[0].source, None,
"PES begun by a plain feed must carry no source after a prior feed_at"
);
}
// ── scan_streams PMT parsing ────────────────────────────────────────── // ── scan_streams PMT parsing ──────────────────────────────────────────
/// Wrap a 188-byte TS packet body in a 192-byte BD-TS packet /// Wrap a 188-byte TS packet body in a 192-byte BD-TS packet
+4 -3
View File
@@ -312,11 +312,12 @@ impl<W: Write> TsMuxer<W> {
/// Build a PES packet header for a BD stream. /// Build a PES packet header for a BD stream.
fn build_pes_header(pid: u16, pts_90k: u64, data_len: usize) -> Vec<u8> { fn build_pes_header(pid: u16, pts_90k: u64, data_len: usize) -> Vec<u8> {
use crate::consts::pes_stream_id;
// Determine stream_id from PID range // Determine stream_id from PID range
let stream_id: u8 = if is_video_pid(pid) { let stream_id: u8 = if is_video_pid(pid) {
0xE0 // video pes_stream_id::VIDEO
} else { } else {
0xBD // audio, PGS subtitle, or default (private stream 1) pes_stream_id::PRIVATE_STREAM_1 // audio, PGS subtitle, or default
}; };
let pes_data_len = data_len + 8; // 3 header bytes + 5 PTS bytes + data let pes_data_len = data_len + 8; // 3 header bytes + 5 PTS bytes + data
@@ -332,7 +333,7 @@ fn build_pes_header(pid: u16, pts_90k: u64, data_len: usize) -> Vec<u8> {
// video; `write_frame` splits oversized 0xBD access units so a private // video; `write_frame` splits oversized 0xBD access units so a private
// stream always fits a bounded u16 length here. The `> 65535` arm // stream always fits a bounded u16 length here. The `> 65535` arm
// remains a defensive fallback for video only. // remains a defensive fallback for video only.
if stream_id == 0xE0 || pes_data_len > 65535 { if stream_id == pes_stream_id::VIDEO || pes_data_len > u16::MAX as usize {
header.push(0x00); header.push(0x00);
header.push(0x00); header.push(0x00);
} else { } else {
+16 -1
View File
@@ -37,7 +37,7 @@ pub const FVI_VERSION: u32 = 1;
/// Producing tool tag for the header `"generator"` member /// Producing tool tag for the header `"generator"` member
/// (`docs/FVI_FORMAT.md` §6). /// (`docs/FVI_FORMAT.md` §6).
pub const FVI_GENERATOR: &str = concat!("freemkv/", env!("CARGO_PKG_VERSION")); pub const FVI_GENERATOR: &str = concat!("freemkv/", env!("FREEMKV_VERSION"), env!("GIT_SUFFIX"));
/// Header `"timescale"` for all `pts`/`dts` ticks (`docs/FVI_FORMAT.md` §10). /// Header `"timescale"` for all `pts`/`dts` ticks (`docs/FVI_FORMAT.md` §10).
/// The highway carries presentation timestamps in nanoseconds, so the timescale /// The highway carries presentation timestamps in nanoseconds, so the timescale
@@ -551,6 +551,21 @@ mod tests {
}, },
"measured CICP must override the coarse color_space enum" "measured CICP must override the coarse color_space enum"
); );
// Unknown colorimetry, SDR, no measured CICP → all code points map to
// "unspecified" (2), matching `from_color_space(Unknown)`. Both sinks of
// one title must emit 2, never 0.
let c = Colour::from_video(&mk(HdrFormat::Sdr, ColorSpace::Unknown, None));
assert_eq!(
c,
Colour {
primaries: 2,
transfer: 2,
matrix: 2,
full_range: false,
},
"Unknown colorimetry must emit CICP 'unspecified' (2), not 0"
);
} }
#[test] #[test]
+6 -9
View File
@@ -107,21 +107,18 @@ impl PesFrame {
/// truncated `.pes` data would be accepted as a graceful end. /// truncated `.pes` data would be accepted as a graceful end.
pub fn deserialize(r: &mut dyn std::io::Read) -> std::io::Result<Option<Self>> { pub fn deserialize(r: &mut dyn std::io::Read) -> std::io::Result<Option<Self>> {
// Probe one byte first to distinguish clean EOF from a truncated // Probe one byte first to distinguish clean EOF from a truncated
// header. // header. Loop on EINTR so back-to-back signals don't fail a
// recoverable read — symmetric with read_exact's internal retry
// on the rest of the header and the data below.
let mut first = [0u8; 1]; let mut first = [0u8; 1];
loop {
match r.read(&mut first) { match r.read(&mut first) {
Ok(0) => return Ok(None), // clean EOF, no frame started Ok(0) => return Ok(None), // clean EOF, no frame started
Ok(_) => {} Ok(_) => break,
Err(e) if e.kind() == std::io::ErrorKind::Interrupted => { Err(e) if e.kind() == std::io::ErrorKind::Interrupted => continue,
// Retry-once on EINTR before committing to the header read.
match r.read(&mut first) {
Ok(0) => return Ok(None),
Ok(_) => {}
Err(e) => return Err(e), Err(e) => return Err(e),
} }
} }
Err(e) => return Err(e),
}
let mut header = [0u8; 22]; // 1 + 8 + 1 + 8 + 4 let mut header = [0u8; 22]; // 1 + 8 + 1 + 8 + 4
header[0] = first[0]; header[0] = first[0];
+70 -6
View File
@@ -104,11 +104,17 @@ pub fn register_unlocker(u: Box<dyn Unlocker>) {
/// * `Ok(Some((name, vid)))` — a registered unlocker matched, put the drive /// * `Ok(Some((name, vid)))` — a registered unlocker matched, put the drive
/// into extended mode, and returned the OEM Volume ID. The caller stashes /// into extended mode, and returned the OEM Volume ID. The caller stashes
/// the VID for the handshake phase and need not run the cert handshake. /// the VID for the handshake phase and need not run the cert handshake.
/// * `Ok(None)` — no unlocker matched, OR the matching unlocker failed /// * `Ok(None)` — no unlocker matched, OR the matching unlocker reported a
/// ([`UnlockError`], logged). Either way the drive is usable in stock mode /// *capability* failure ([`UnlockError::FirmwareNotUnlockable`],
/// and the caller falls through to the in-tree cert handshake. Folding an /// [`UnlockError::VidUnavailable`], or a cert-auth outcome — all logged).
/// unlock failure into `Ok(None)` keeps drive `init()` infallible — a drive /// Either way the drive is usable in stock mode and the caller falls
/// that simply isn't firmware-unlockable must not fail init. /// through to the in-tree cert handshake. Folding a capability failure into
/// `Ok(None)` keeps drive `init()` infallible — a drive that simply isn't
/// firmware-unlockable must not fail init.
/// * `Err(_)` — the matching unlocker hit a genuine SCSI/transport fault
/// ([`UnlockError::Scsi`]). The bus is broken, not merely unsupported, so
/// this propagates and aborts init rather than silently falling through to
/// a cert handshake that would also fail.
pub(crate) fn route_unlock( pub(crate) fn route_unlock(
scsi: &mut dyn ScsiTransport, scsi: &mut dyn ScsiTransport,
id: &DriveId, id: &DriveId,
@@ -124,10 +130,29 @@ pub(crate) fn route_unlock(
let name = u.name().to_string(); let name = u.name().to_string();
match u.unlock(scsi, id) { match u.unlock(scsi, id) {
Ok(vid) => return Ok(Some((name, vid))), Ok(vid) => return Ok(Some((name, vid))),
Err(e) => { // A genuine SCSI/transport fault is not "this drive can't be
// unlocked" — the bus is broken. Propagate so init() aborts
// instead of falling through to a cert handshake that will
// also fail on the same dead transport. The numeric code from
// the originating error is logged; the returned error is the
// canonical transport-error variant.
Err(UnlockError::Scsi(code)) => {
tracing::error!(
target: "freemkv::unlock",
unlocker = %name,
code,
"unlocker hit a transport fault during unlock; aborting init"
);
return Err(crate::error::Error::ScsiError {
opcode: 0,
status: 0,
sense: None,
});
}
// A firmware unlocker that can't unlock / has no OEM VID: // A firmware unlocker that can't unlock / has no OEM VID:
// fall through to the cert handshake. Debug-only structured // fall through to the cert handshake. Debug-only structured
// log (variant identifiers, no English prose). // log (variant identifiers, no English prose).
Err(e) => {
tracing::warn!( tracing::warn!(
target: "freemkv::unlock", target: "freemkv::unlock",
unlocker = %name, unlocker = %name,
@@ -228,6 +253,10 @@ mod tests {
/// `None` → `unlock` yields `Err(UnlockError::VidUnavailable)` so /// `None` → `unlock` yields `Err(UnlockError::VidUnavailable)` so
/// `route_unlock` falls through to the cert handshake. /// `route_unlock` falls through to the cert handshake.
vid: Option<[u8; 16]>, vid: Option<[u8; 16]>,
/// When `Some(code)`, `unlock` yields `Err(UnlockError::Scsi(code))`
/// (a transport fault) instead of consulting `vid`, so `route_unlock`
/// propagates an error and aborts init.
scsi_err: Option<u16>,
/// Records whether set_max_read_speed was invoked. /// Records whether set_max_read_speed was invoked.
speed_ran: Arc<AtomicBool>, speed_ran: Arc<AtomicBool>,
} }
@@ -238,6 +267,7 @@ mod tests {
ran, ran,
// Default: a successful unlock returning an all-zero VID. // Default: a successful unlock returning an all-zero VID.
vid: Some([0u8; 16]), vid: Some([0u8; 16]),
scsi_err: None,
speed_ran: Arc::new(AtomicBool::new(false)), speed_ran: Arc::new(AtomicBool::new(false)),
} }
} }
@@ -245,6 +275,10 @@ mod tests {
self.vid = vid; self.vid = vid;
self self
} }
fn with_scsi_err(mut self, code: u16) -> Self {
self.scsi_err = Some(code);
self
}
fn with_speed(mut self, speed_ran: Arc<AtomicBool>) -> Self { fn with_speed(mut self, speed_ran: Arc<AtomicBool>) -> Self {
self.speed_ran = speed_ran; self.speed_ran = speed_ran;
self self
@@ -263,6 +297,9 @@ mod tests {
_id: &DriveId, _id: &DriveId,
) -> std::result::Result<Vid, UnlockError> { ) -> std::result::Result<Vid, UnlockError> {
self.ran.store(true, Ordering::SeqCst); self.ran.store(true, Ordering::SeqCst);
if let Some(code) = self.scsi_err {
return Err(UnlockError::Scsi(code));
}
match self.vid { match self.vid {
Some(v) => Ok(Vid(v)), Some(v) => Ok(Vid(v)),
None => Err(UnlockError::VidUnavailable), None => Err(UnlockError::VidUnavailable),
@@ -359,6 +396,33 @@ mod tests {
assert!(got.is_none(), "no match → cert fallback"); assert!(got.is_none(), "no match → cert fallback");
} }
/// A matching unlocker that hits a genuine transport fault
/// (`UnlockError::Scsi`) makes `route_unlock` PROPAGATE an `Err` rather
/// than fold to `Ok(None)`: a dead bus must abort init, not silently fall
/// through to a cert handshake that would also fail. Capability failures
/// (`VidUnavailable` etc.) still fold to `Ok(None)` — proven by the sibling
/// routing tests; this one pins the transport-fault exception.
#[test]
fn route_unlock_propagates_scsi_transport_fault() {
let mut scsi = NoopTransport;
register_unlocker(Box::new(
FakeUnlocker::new("SCSIVNDR", Arc::new(AtomicBool::new(false)))
.with_scsi_err(crate::error::E_SCSI_ERROR),
));
let got = route_unlock(&mut scsi, &fake_id("SCSIVNDR"));
assert!(
got.is_err(),
"a transport fault during unlock aborts init (propagates Err)"
);
assert_eq!(
got.unwrap_err().code(),
crate::error::E_SCSI_ERROR,
"propagated error is the canonical transport-error code"
);
}
/// `unlocker_set_max_read_speed` consults the FIRST matching unlocker's /// `unlocker_set_max_read_speed` consults the FIRST matching unlocker's
/// `set_max_read_speed`. A matching unlocker is invoked; a non-match is a /// `set_max_read_speed`. A matching unlocker is invoked; a non-match is a
/// safe no-op (nothing invoked, `Ok(())`). /// safe no-op (nothing invoked, `Ok(())`).
+16 -8
View File
@@ -160,7 +160,7 @@ fn two_gop_image() -> Vec<u8> {
data[..a.len()].copy_from_slice(&a); data[..a.len()].copy_from_slice(&a);
let mut b = ps_pack_header(); let mut b = ps_pack_header();
b.extend_from_slice(&video_pes(&gop_es(), 3003)); // ~0.1s later b.extend_from_slice(&video_pes(&gop_es(), 3003)); // one frame (~33 ms) later at 29.97 fps (3003/90000 s)
let off = 3 * 2048; let off = 3 * 2048;
data[off..off + b.len()].copy_from_slice(&b); data[off..off + b.len()].copy_from_slice(&b);
@@ -276,13 +276,21 @@ fn fvi_sink_indexes_real_mpeg2_pipeline_output() {
vec![0, 3], vec![0, 3],
"stamped src sectors must reach the .fvi in arrival order; got {src_sectors:?}" "stamped src sectors must reach the .fvi in arrival order; got {src_sectors:?}"
); );
// For each stamped record, sector == byte / 2048 (SourcePos::at_byte). The // Per FVI_FORMAT.md §9, `src.byte` is the offset of the AU's first byte
// byte offset is exact (here 14 into each region, just past the pack // WITHIN its 2048-byte `src.sector`, so it is always < 2048. The two
// header), so it is NOT sector-aligned — provenance is byte-exact. // stamped sources sit 14 bytes into their sector (just past the pack
// header), so the within-sector byte is exact (14) — provenance is
// byte-exact, carried unchanged from demux.
for r in &records { for r in &records {
if let (Some(sector), Some(byte)) = (r["src"]["sector"].as_u64(), r["src"]["byte"].as_u64()) if let Some(byte) = r["src"]["byte"].as_u64() {
{ assert!(
assert_eq!(sector, byte / 2048, "src.sector must equal src.byte / 2048"); byte < 2048,
"src.byte is a within-sector offset (§9), must be < 2048; got {byte}"
);
assert_eq!(
byte, 14,
"stamped src.byte is 14 (just past the pack header)"
);
} }
} }
@@ -338,7 +346,7 @@ fn fvi_sink_indexes_non_mpeg2_frames_codec_agnostically() {
assert_eq!(recs[0]["type"], "I"); assert_eq!(recs[0]["type"], "I");
assert_eq!(recs[0]["pts"], 1234); assert_eq!(recs[0]["pts"], 1234);
assert_eq!(recs[0]["src"]["sector"], 4); // 8192 / 2048 assert_eq!(recs[0]["src"]["sector"], 4); // 8192 / 2048
assert_eq!(recs[0]["src"]["byte"], 8192); assert_eq!(recs[0]["src"]["byte"], 0); // 8192 is sector-aligned → within-sector offset 0 (§9)
assert!( assert!(
recs[0].get("field_order").is_none() && recs[0].get("nb_fields").is_none(), recs[0].get("field_order").is_none() && recs[0].get("nb_fields").is_none(),
"coding-absent record omits field_order/nb_fields" "coding-absent record omits field_order/nb_fields"