pq/libfreemkv
Watch
1
Fork
You've already forked libfreemkv
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Compare commits

base: pq:v1.6.5
Branches Tags
pq:dev
pq:main
pq:qa
..
compare: pq:v0.20.2
Branches Tags
pq:dev
pq:main
pq:qa
227 changed files with 22274 additions and 154617 deletions
Download patch file Download diff file Expand all files Collapse all files

63
.github/dependabot.yml vendored
View file

@ -1,63 +0,0 @@
version: 2
# Dependency updates land on `dev`, never on `main`.
#
# `main` here is a RELEASE POINTER that release.sh moves to each tag. A bot
# commit on it would put work there that no tag contains, which is exactly the
# state that aborted the 1.6.2 cascade at the last step -- so pointing
# Dependabot at main would recreate that failure on a schedule.
updates:
- package-ecosystem: cargo
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
# One PR per week for the routine bumps instead of one per crate. Eight
# repos times a handful of crates is a volume nobody reads, and an
# unread PR queue is indistinguishable from no updates at all.
groups:
minor-and-patch:
update-types:
- minor
- patch
ignore:
# The freemkv crates depend on each other by GIT TAG, re-pinned by
# release.sh as part of the release commit. Dependabot cannot see that
# cascade, so a PR bumping one of these would fight the release process
# and could pin a version whose tag does not exist yet.
- dependency-name: freemkv-unlock
- dependency-name: libfreemkv
- dependency-name: freemkv-keysources
- dependency-name: freemkv-i18n
- dependency-name: freemkv-engine
# The workflows are now real infrastructure -- the release cascade, the
# cross-platform hash matrix, the disc gate -- so their actions need the same
# attention as the crates.
- package-ecosystem: github-actions
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
actions:
update-types:
- minor
- patch
ignore:
# NOT a dependency: `dtolnay/rust-toolchain` is versioned by the RUST
# release it installs, and the tag we pin is the toolchain CI is pinned
# to on purpose -- precommit.sh runs the same one locally so a lint that
# passes on a developer's newer default cannot pass CI by accident.
#
# Dependabot reads those tags as semver and proposed 1.97.0 -> 1.100.0,
# a Rust version that does not exist. Every such PR 404s on toolchain
# download across all eight repos, and they regenerate weekly -- eight
# permanently-red PRs that promote.yml then has to special-case when it
# decides whether dev is green.
#
# Bumping the toolchain is a deliberate, all-eight-repos change, made by
# hand together with precommit.sh. There is nothing here for a bot.
- dependency-name: dtolnay/rust-toolchain

209
.github/workflows/ci.yml vendored
View file

@ -2,228 +2,41 @@ name: CI
on:
push:
# dev -> qa -> main. `dev` is where work lands and is meant to be pushed
# to often: these are the FAST checks, so a mistake surfaces in minutes.
# `qa` is the release candidate — it runs these too, plus the expensive
# suite in qa.yml. `main` only ever moves at release time, to a tagged
# commit that was already green on qa.
branches: [main, dev, qa]
branches: [main]
pull_request:
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
with:
components: clippy, rustfmt
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo fmt --check
working-directory: libfreemkv
# libfreemkv is a library — Cargo.lock is gitignored. --locked
# would always fail on a fresh runner because there's no committed
# lockfile to lock against. The binary crates (freemkv, autorip,
# bdemu) track Cargo.lock and DO use --locked.
# --all-targets so TEST code is linted too. Without it this crate — the
# reference implementation for the other seven — was the only one whose
# tests had never been linted at all, and it was hiding 74 findings.
- run: cargo clippy --all-targets -- -D warnings
working-directory: libfreemkv
- run: cargo clippy -- -D warnings
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- run: cargo test --tests
working-directory: libfreemkv
check-macos:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: macos-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- run: cargo check
working-directory: libfreemkv
check-windows:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: windows-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
# Build the tests (not just `cargo check`): catches errors in test
# code and forces full codegen of the Windows-only SPTI transport
# (src/scsi/windows.rs), which never compiles on the Linux/macOS dev
# hosts. We don't `cargo test` here — the suite needs no drive but the
# extra build is the value; running tests is covered by the Linux job.
- run: cargo build --tests
working-directory: libfreemkv
# ── Did this change break anything downstream? ──────────────────────────────
#
# Every job above proves libfreemkv builds. None proved its DEPENDENTS do,
# and that gap is real: an engine signature change broke autorip today and
# went unnoticed because consumer CI only fires on a push to that consumer.
# libfreemkv sits below all five of them, so a break here is worth strictly
# more than a break anywhere else in the project.
#
# `cargo check --all-targets` only — each dependent owns its own behaviour
# and has its own suite. The question here is just "does everything built on
# me still compile against this commit".
consumers:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with: { path: libfreemkv }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-unlock, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-unlock }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-keysources, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-keysources }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-engine, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-engine }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-i18n, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-i18n }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv }
- uses: actions/checkout@v7
with: { repository: freemkv/autorip, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: autorip }
- uses: actions/checkout@v7
with: { repository: freemkv/bdemu, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: bdemu }
- name: Point every dependent at THIS libfreemkv commit
shell: bash
run: |
for c in freemkv-keysources freemkv-engine freemkv autorip bdemu; do
mkdir -p "$c/.cargo"
cat > "$c/.cargo/config.toml" <<'EOF'
[patch.crates-io]
libfreemkv = { path = "../libfreemkv" }
freemkv-keysources = { path = "../freemkv-keysources" }
freemkv-engine = { path = "../freemkv-engine" }
freemkv-i18n = { path = "../freemkv-i18n" }
EOF
done
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: |
freemkv-keysources
freemkv-engine
freemkv
autorip
bdemu
# `cargo check` alone only proves the dependents still COMPILE against
# this commit. It cannot see a behavioural change — the library keeps its
# signatures and a dependent's tests start failing. That is the shape of
# every defect worth catching here, so run their suites too.
- run: cargo test --tests
working-directory: freemkv-keysources
- run: cargo test --tests
working-directory: freemkv-engine
- run: cargo test --tests
working-directory: freemkv
- run: cargo test --tests
working-directory: autorip
- run: cargo test --tests
working-directory: bdemu
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- run: cargo check

35
.github/workflows/leak-guard.yml vendored
View file

@ -1,35 +0,0 @@
name: leak-guard
# Self-contained public-repo leak gate. Public CI cannot reach the private
# tooling, so this encodes only the generic net: internal-infra references,
# tracked CLAUDE.md/.claude paths, and AI-attribution in commit messages.
# No project-specific reverse-engineering vocabulary lives here.
on: [push, pull_request]
jobs:
leak-guard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Compute commit range
id: range
run: |
if [ "${{ github.event_name }}" = "pull_request" ]; then
base="${{ github.event.pull_request.base.sha }}"
head="${{ github.event.pull_request.head.sha }}"
echo "range=$base..$head" >> "$GITHUB_OUTPUT"
else
before="${{ github.event.before }}"
after="${{ github.sha }}"
# New branch / first push: github.event.before is all-zeros.
if [ -z "$before" ] || [ "$before" = "0000000000000000000000000000000000000000" ]; then
echo "range=$after" >> "$GITHUB_OUTPUT"
else
echo "range=$before..$after" >> "$GITHUB_OUTPUT"
fi
fi
- name: Run leak-guard
run: bash ci/leak-guard.sh "${{ steps.range.outputs.range }}"

133
.github/workflows/qa.yml vendored
View file

@ -1,133 +0,0 @@
name: qa
# ── The qa gate: "is this production worth?" ────────────────────────────────
#
# dev -> qa -> main.
#
# `dev` is for committing often. ci.yml answers "is it green" in minutes with
# fmt, clippy and the unit suite, so a mistake surfaces while it is still cheap
# to fix. `qa` is the release-candidate branch, and THIS workflow is the claim
# that a commit is production worth: everything expensive that can run without
# physical media. `main` only ever receives a qa that went green here.
#
# Sibling repos are checked out at `qa`, NOT `dev`. A qa run that resolved its
# dependencies from dev tips would be validating a combination that is not the
# one being released, which is the exact failure this branch exists to prevent.
#
# What this gate CANNOT cover: `disc://` and real `iso://` need physical media,
# and no hosted runner has an optical drive or the image hoard. Those run on a
# self-hosted runner (see the media job at the end) and are the one leg that
# stays on hardware.
on:
push:
branches: [qa]
workflow_dispatch:
jobs:
# ── Name the candidate ────────────────────────────────────────────────
#
# Every push to `qa` is a release candidate, so every push gets a tag:
# v<version>-rc<N>, N incrementing. That is the answer to "which build is on
# qa right now, and is it the one I tested?" — a question that otherwise gets
# answered from memory.
#
# This runs FIRST and does not depend on the gates, deliberately. A red
# candidate needs a name more than a green one does: "rc3 failed
# release-tests on windows" is a sentence you can act on; "qa is red" is not.
# Red on qa is a working gate, not an incident — it is the branch saying this
# is not production worth yet. Fix on dev, get dev green, push qa again.
#
# release.yml excludes v*-rc* so a candidate never publishes a release.
rc-tag:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Stamp the next rc
shell: bash
run: |
set -euo pipefail
v=$(sed -n 's/^version = "\(.*\)"/\1/p' Cargo.toml | head -1)
[ -n "$v" ] || { echo "no version in Cargo.toml" >&2; exit 1; }
# Numeric sort on the rc ordinal: -rc10 must beat -rc9, and a plain
# lexical sort gets that backwards from the tenth candidate on.
n=$(git tag -l "v$v-rc*" | sed "s|^v$v-rc||" | sort -n | tail -1)
tag="v$v-rc$(( ${n:-0} + 1 ))"
git tag "$tag"
git push origin "$tag"
echo "### Candidate \`$tag\`" >> "$GITHUB_STEP_SUMMARY"
# The debug suite runs on every dev push. Release is a DIFFERENT build:
# overflow checks are off, debug_assert! is compiled out, and inlining
# changes what the optimiser can prove. A test that only passes in debug is
# a test that never guarded the binary anyone actually ships.
release-tests:
strategy:
fail-fast: false
matrix:
os: ['ubuntu-latest', 'macos-latest']
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --release --tests
working-directory: libfreemkv
# clippy's output is target-dependent: cfg-gated code only gets linted on
# the target it compiles for. Linting solely on the dev machine's host
# target is how a lint that CI rejects reaches a push.
cross-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
with:
components: clippy
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: rustup target add x86_64-unknown-linux-gnu
- run: cargo clippy --all-targets --target x86_64-unknown-linux-gnu -- -D warnings
working-directory: libfreemkv
# Windows compiles the tests but does not run them, matching the policy
# ci.yml already set. The value here is codegen: the #[cfg(windows)] halves
# of the SCSI transport and platform layers compile on no other runner, so
# without this they are first built at release time.
windows-build:
runs-on: windows-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo build --release --tests
working-directory: libfreemkv

45
.github/workflows/release.yml vendored
View file

@ -4,11 +4,6 @@ on:
push:
tags:
- 'v*'
# NOT the release-candidate tags. Every push to `qa` stamps a
# v<version>-rc<N> so a run can be named, and 'v*' matches those too —
# which would have this workflow build and PUBLISH a GitHub release for
# every candidate, including the red ones.
- '!v*-rc*'
permissions:
contents: write
@ -17,7 +12,7 @@ jobs:
verify:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
- name: Verify Cargo.toml version matches tag
run: |
CARGO_VER="v$(grep '^version' Cargo.toml | head -1 | sed 's/.*"\(.*\)"/\1/')"
@ -27,37 +22,33 @@ jobs:
fi
echo "Version match: $CARGO_VER"
# Tests run as a PARALLEL TRIPWIRE: they fail the run if they fail, but the
# publish/release jobs do NOT `needs:` this job. The tag decision was already
# gated by the local precommit (same Rust 1.97, same commit). Binary consumers
# (freemkv/autorip/bdemu) git-tag-pin libfreemkv and therefore start building
# the instant this tag exists — so this test job and the crates.io publish
# below must NOT sit on their critical path.
test:
needs: verify
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@1.97.0
- uses: Swatinem/rust-cache@v2
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
# libfreemkv is a library — Cargo.lock isn't tracked, so --locked
# would always fail (no lockfile to lock against on a fresh runner).
- run: cargo test
# NOTE: there is no crates.io publish job. libfreemkv is git-tag-only
# (`package.publish = false` — it git-deps the firmware crate freemkv-unlock,
# which never ships to crates.io). Every consumer git-tag-pins libfreemkv via
# a committed [patch.crates-io]; the git tag itself IS the release artifact.
# A `cargo publish` here fails hard on `publish = false`, so it was removed.
release:
# Only needs `verify`; the GitHub Release can be cut as soon as the version
# check passes, in parallel with test + publish.
needs: verify
publish:
needs: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- name: Publish to crates.io
run: cargo publish
env:
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
release:
needs: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- name: Create GitHub Release
uses: softprops/action-gh-release@v3
uses: softprops/action-gh-release@v2
with:
generate_release_notes: true

36
.github/workflows/update-readme.yml vendored Normal file
View file

@ -0,0 +1,36 @@
name: Update README version
on:
release:
types: [published]
permissions:
contents: write
jobs:
update:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with:
ref: main
token: ${{ secrets.ORG_DISPATCH_TOKEN }}
- name: Update version in README
run: |
VERSION="${{ github.event.release.tag_name }}"
FILE="README.md"
# Update cargo dependency version (e.g. "0.2" -> "0.3")
MAJOR_MINOR="${VERSION#v}"
MAJOR_MINOR="${MAJOR_MINOR%.*}"
sed -i "s|libfreemkv = \"[0-9]*\.[0-9]*\"|libfreemkv = \"${MAJOR_MINOR}\"|" "$FILE"
- name: Commit and push
run: |
VERSION="${{ github.event.release.tag_name }}"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add README.md
git diff --cached --quiet || git commit -m "Update to libfreemkv ${VERSION}"
git push

19
.gitignore vendored
View file

@ -4,22 +4,3 @@ Cargo.lock
*.swo
.DS_Store
.cargo/
# session scratch — never track (may contain RE breadcrumbs)
scratch/
# stray local build artifact
/rust_out
# internal agent context — never publish (path AND dir; leak-guard blocks both)
CLAUDE.md
.claude/
# Nightly harness output. Written into the repo it audits, and it embeds
# absolute paths from the machine that ran it — which must never reach a public
# repo. Ignored rather than relocated so a run from any working copy is safe.
.nightly/
# cargo-mutants working output: large, machine-specific, never committed
mutants.out/
mutants.out.old/

2705
CHANGELOG.md
View file

File diff suppressed because it is too large Load diff

83
CODE_OF_CONDUCT.md
View file

@ -1,83 +0,0 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
* Focusing on what is best not just for us as individuals, but for the overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.
Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported privately to the project maintainer via GitHub at https://github.com/MattJackson. All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series of actions.
**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.1, available at [https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
Community Impact Guidelines were inspired by [Mozilla's code of conduct enforcement ladder][Mozilla CoC].
For answers to common questions about this code of conduct, see the FAQ at [https://www.contributor-covenant.org/faq][FAQ]. Translations are available at [https://www.contributor-covenant.org/translations][translations].
[homepage]: https://www.contributor-covenant.org
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
[Mozilla CoC]: https://github.com/mozilla/diversity
[FAQ]: https://www.contributor-covenant.org/faq
[translations]: https://www.contributor-covenant.org/translations

2
CONTRIBUTING.md
View file

@ -24,4 +24,4 @@ cargo test
## License
By contributing, you agree your code will be licensed under MIT.
By contributing, you agree your code will be licensed under AGPL-3.0.

61
Cargo.toml
View file

@ -1,62 +1,33 @@
[package]
name = "libfreemkv"
version = "1.6.5"
version = "0.20.2"
edition = "2024"
rust-version = "1.97"
license = "MIT"
rust-version = "1.86"
license = "AGPL-3.0-only"
description = "Open source raw disc access library for optical drives"
repository = "https://github.com/freemkv/libfreemkv"
keywords = ["bluray", "uhd", "optical", "scsi", "disc"]
categories = ["hardware-support", "multimedia"]
# Keep internal AI-instruction / private notes out of the published crate.
exclude = ["CLAUDE.md"]
# OFF crates.io: libfreemkv git-deps freemkv-unlock (firmware, never published),
# so libfreemkv itself can only be consumed by git tag. Clients git-tag-pin it.
publish = false
[profile.release]
lto = "thin"
codegen-units = 1
[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"
sha1 = "0.10"
aes = "0.9"
# Interim path dep for local cross-repo dev; the release script re-pins this to
# `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` before tagging libfreemkv (so
# the released tag resolves freemkv-unlock from git, not a sibling path).
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.6.5" }
rand = "0.10"
zip = { version = "8", default-features = false, features = ["deflate"] }
base64 = "0.23"
# Read-only XML DOM parser (pure Rust, forbid(unsafe_code), entity-expansion
# bounded). Parses the HD-DVD Advanced-Content playlist `ADV_OBJ/VPLST000.XPL`
# — untrusted disc bytes — into authoritative titles/clips/chapters. A real
# parser, not a hand-rolled scanner: the XPL is genuine XML (comments, varied
# attribute order, self-closing tags).
roxmltree = "0.20"
# Trace-level instrumentation for the read/transport path (SgIoTransport::execute
# and friends). Permitted
# under CLAUDE.md ("Acceptable strings: debug/trace logging"). Consumers (autorip)
sha2 = "0.10"
aes = "0.8"
cbc = "0.1"
flate2 = "1"
num-bigint = "0.4"
num-traits = "0.2"
num-integer = "0.1"
rand = "0.8"
cmac = "0.7"
zip = { version = "2", default-features = false, features = ["deflate"] }
base64 = "0.22.1"
# Trace-level instrumentation for Disc::copy + SgIoTransport::execute. Permitted
# under project docs ("Acceptable strings: debug/trace logging"). Consumers (autorip)
# wire a tracing subscriber and pipe events into the JSONL debug log.
tracing = "0.1"
# Bounded MPSC channel with kernel-wakeup send_timeout. Used by `io::pipeline`
# so the halt-aware send/finish loops can BLOCK on consumer drain instead of
# polling — the 50 ms poll cadence of the previous mpsc-based impl capped mux
# throughput at ~1 MB/s (0.21.7).
crossbeam-channel = "0.5"
# Persistent work-stealing thread pool for parallel AACS unit
# decryption. Per-call std::thread::scope spawned fresh OS threads
# and that overhead dominated for typical batch sizes (60 units).
# rayon's global pool initialises once on first use.
rayon = "1"
# SIMD-accelerated bytestring search. Drives the HEVC/H.264 start-code
# scan in `mux::codec::h264::find_start_code` — naive byte-by-byte
# walk is ~500 MB/s single-thread on x86_64; memchr's vectorised
# `memmem::find` for the 3-byte `00 00 01` needle hits ~5 GB/s on
# AVX2-capable hosts.
memchr = "2"
[target.'cfg(target_os = "linux")'.dependencies]
libc = "0.2"

29
LICENSE
View file

@ -1,21 +1,16 @@
MIT License
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (c) 2026 Matthew Jackson & Contributors
Copyright (C) 2026 FreeMKV Contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation, version 3 of the License.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.

3
Makefile
View file

@ -1,5 +1,6 @@
# libfreemkv — local dev helper.
# Mirrors the workspace-wide CI checks but scoped to this single crate.
# Mirrors the cross-crate scripts in (internal)/scripts/test-all.sh
# but scoped to this single crate.
.PHONY: test build check ci clean

98
README.md
View file

@ -1,26 +1,24 @@
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
[![Crates.io](https://img.shields.io/crates/v/libfreemkv)](https://crates.io/crates/libfreemkv)
[![docs.rs](https://img.shields.io/docsrs/libfreemkv)](https://docs.rs/libfreemkv)
[![License: AGPL-3.0](https://img.shields.io/badge/license-AGPL--3.0-blue)](LICENSE)
# libfreemkv
Rust library for 4K UHD / Blu-ray / DVD optical drives. Drive access, disc scanning, stream labels, AACS decryption, CSS decryption, KEYDB updates, and content reading in one crate. Drive-level unlocking is handled internally; consumers work with disc access and decryption only.
Rust library for 4K UHD / Blu-ray / DVD optical drives. Drive access, disc scanning, stream labels, AACS decryption, CSS decryption, KEYDB updates, and content reading in one crate. Bundled drive profiles — no external files needed.
DVDs (CSS) decrypt out of the box. Blu-ray and UHD (AACS) require a `keydb.cfg` (default `~/.config/freemkv/keydb.cfg`) supplying disc-specific volume unique keys; no AACS key material is compiled in.
**12+ MB/s** sustained read speeds on BD. Drive prep (`init()`) handles unlocking internally via the `freemkv-unlock` crate — clients never see it; when no drive unlock applies, the library rips via the host-certificate AACS handshake.
**12+ MB/s** sustained read speeds on BD. Full init: unlock, firmware upload, speed calibration — all from pure Rust.
Multi-lingual by design — the library outputs structured data and numeric error codes, never English text. Build any UI or localization on top.
**[Source & API](https://github.com/freemkv/libfreemkv)** · **[Technical Docs](docs/)**
**[API Documentation](https://docs.rs/libfreemkv)** · **[Technical Docs](docs/)**
Part of the [freemkv](https://github.com/freemkv) project.
## Install
Consumed by git tag (not published to crates.io):
```toml
[dependencies]
libfreemkv = { git = "https://github.com/freemkv/libfreemkv", tag = "vX.Y.Z" }
libfreemkv = "0.18"
```
## Quick Start
@ -29,10 +27,10 @@ libfreemkv = { git = "https://github.com/freemkv/libfreemkv", tag = "vX.Y.Z" }
use libfreemkv::{Drive, Disc, ScanOptions};
use std::path::Path;
// Open drive — identified via INQUIRY
// Open drive — profiles are bundled, auto-identified
let mut drive = Drive::open(Path::new("/dev/sg4"))?;
drive.wait_ready()?; // wait for disc
drive.init()?; // unlock + prep (handled internally)
drive.init()?; // unlock + firmware upload
drive.probe_disc()?; // probe disc surface for optimal speeds
// Scan disc — UDF, playlists, streams, AACS (all automatic)
@ -55,19 +53,53 @@ output.finish()?;
### Multi-pass recovery rip
Recovery moved OUT of this crate in 1.6.0. The sweep/patch strategy, the
ddrescue mapfile, damage classification and the multipass loop now live in the
`freemkv-engine` crate as `freemkv_engine::recovery::{copy, sweep, patch}`.
For damaged discs the library exposes two flat verbs — `Disc::sweep` for the
forward Pass 1 and `Disc::patch` for retrying bad ranges. The library never
loops; the multipass policy is the caller's job. See
[`docs/rip-recovery.md`](docs/rip-recovery.md) and the design notes in
`(internal)/memory/0_18_redesign.md`.
libfreemkv keeps the layers underneath: the raw single-shot read
(`Drive::read`) and the SCSI-fact translation (`SenseFamily`) that the engine's
strategy is built on. The dependency runs engine → libfreemkv, so this crate
cannot call into it; front-ends get recovery from the engine directly. See
[`docs/rip-recovery.md`](docs/rip-recovery.md) for what stayed here.
```rust
use libfreemkv::{SweepOptions, PatchOptions};
use libfreemkv::disc::{mapfile, mapfile_path_for};
use std::path::Path;
let iso = Path::new("disc.iso");
// Pass 1: disc → ISO. Skip-on-error, zero-fill, write the sidecar mapfile.
disc.sweep(&mut drive, iso, &SweepOptions {
decrypt: true,
resume: false,
batch_sectors: None,
skip_on_error: true,
progress: None,
halt: None,
})?;
// Pass 2..N: retry every non-finished range. Idempotent.
loop {
let map = mapfile::Mapfile::load(&mapfile_path_for(iso))?;
let stats = map.stats();
if stats.bytes_pending + stats.bytes_unreadable == 0 { break; }
let outcome = disc.patch(&mut drive, iso, &PatchOptions {
decrypt: true,
block_sectors: None,
full_recovery: true,
reverse: true,
wedged_threshold: 50,
progress: None,
halt: None,
})?;
if outcome.bytes_recovered_this_pass == 0 { break; }
}
// Mux from the ISO via the normal stream pipeline (no drive involvement).
```
## What It Does
- **Drive access** — open, identify, internal unlock + prep, speed control, eject
- **Drive access** — open, identify, unlock, firmware upload, speed calibration, eject
- **12+ MB/s reads** — auto-detects kernel transfer limits, sustained full speed
- **Disc scanning** — UDF 2.50 filesystem, MPLS playlists, CLPI clip info
- **Stream labels** — 5 BD-J format parsers (Paramount, Criterion, Pixelogic, CTRM, Deluxe)
@ -81,28 +113,24 @@ cannot call into it; front-ends get recovery from the engine directly. See
| Stream | Input | Output | Transport |
|--------|-------|--------|-----------|
| DiscStream | Yes | -- | Optical drive via SCSI |
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written by `freemkv_engine::recovery`) |
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written via `Disc::sweep()`) |
| MkvStream | Yes | Yes | Matroska container |
| M2tsStream | Yes | Yes | BD transport stream with FMKV metadata header |
| NetworkStream | Yes (listen) | Yes (connect) | TCP with FMKV metadata header |
| StdioStream | Yes (stdin) | Yes (stdout) | Raw byte pipe |
| NullStream | -- | Yes | Discard sink (byte counter for benchmarks) |
Streams implement a single unified `pes::Stream` trait (re-exported as `PesStream`) exposing `read()` and `write()` on one type. `input()` / `output()` resolve URL strings to PES stream instances. All URLs use the `scheme://path` format — bare paths are rejected.
Streams implement `FrameSource` (read) and/or `FrameSink` (write); direction is type-checked. `input()` / `output()` resolve URL strings to PES stream instances. All URLs use the `scheme://path` format — bare paths are rejected.
### Keys
DVDs (CSS) decrypt out of the box, with no external key file needed.
Blu-rays and UHD (AACS) require a `keydb.cfg` at `~/.config/freemkv/keydb.cfg` (or passed via `ScanOptions`). No AACS key material is compiled into the binary.
AACS decryption requires a KEYDB.cfg file. If available at `~/.config/aacs/KEYDB.cfg` or passed via `ScanOptions`, the library handles everything — handshake, key derivation, and per-sector decryption — without the application needing to know anything about encryption.
## Architecture
```text
Drive — open, identify, init, single-shot read
Drive — open, identify, init, unlock, single-shot read
├── ScsiTransport — SG_IO (Linux), IOKit (macOS), SPTI (Windows)
└── unlock_bridge — private seam to the freemkv-unlock crate
(firmware / AACS cert / CSS bus-auth unlockers)
├── DriveProfile — per-drive unlock parameters (bundled)
└── PlatformDriver — MediaTek (supported), Renesas (planned)
Disc — scan titles, streams, AACS/CSS state
├── UDF reader — Blu-ray UDF 2.50 with metadata partitions
@ -111,11 +139,12 @@ Disc — scan titles, streams, AACS/CSS state
├── IFO parser — DVD title sets, PGC chains, cell addresses
├── Labels — 5 BD-J format parsers (detect + parse)
├── AACS — key resolution + content decryption
├── CSS — DVD CSS (bus auth → player-key disc crack → known-plaintext title-key attack)
├── CSS — DVD CSS cipher (table-driven, no keys needed)
└── KEYDB — download + verify + save
Streams — unified PES pipeline
├── PesStream — pes::Stream: one trait, read()/write() PES frames
├── FrameSource — read() PES frames (direction-typed)
├── FrameSink — write() PES frames (direction-typed)
├── DiscStream — sectors → decrypt → TS demux → PES
├── IsoStream — ISO file → decrypt → TS demux → PES
├── MkvStream — MKV mux/demux
@ -141,7 +170,6 @@ All errors are structured with numeric codes. No user-facing English text — ap
| E6xxx | Disc format errors |
| E7xxx | AACS errors |
| E8xxx | KEYDB update errors |
| E9xxx | Stream / mux errors (URL, PES, ISO, pipeline, demux) |
## Platform Support
@ -153,8 +181,8 @@ All errors are structured with numeric codes. No user-facing English text — ap
## Contributing
Run `freemkv info disc:// --share` with the [freemkv CLI](https://github.com/freemkv/freemkv) to capture your drive's identity for contribution. Drive-unlock profiles are maintained in the [freemkv-unlock](https://github.com/freemkv/freemkv-unlock) repository.
Run `freemkv info disc:// --share` with the [freemkv CLI](https://github.com/freemkv/freemkv) to contribute your drive's profile.
## License
MIT
AGPL-3.0-only

22
SECURITY.md
View file

@ -1,22 +0,0 @@
# Security Policy
## Supported versions
| Version | Supported |
| ------- | --------- |
| 1.6.x | Yes |
| < 1.6 | No |
Only the current 1.6.x line receives security fixes.
## Reporting a vulnerability
Report vulnerabilities privately through GitHub Security Advisories:
https://github.com/freemkv/libfreemkv/security/advisories/new
Do not open a public issue for a security report. Include the affected
version, steps to reproduce, and the impact you believe the issue has.
## Response time
You will get an initial response within 7 days.

6
TROUBLESHOOTING.md
View file

@ -141,8 +141,8 @@ If your machine has a free SATA port, use it.
freemkv uses a three-layer recovery model. See [`docs/rip-recovery.md`](docs/rip-recovery.md) for full details.
- **Pass 1 (`freemkv_engine::recovery::sweep`):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
- **Pass 2+ (`freemkv_engine::recovery::patch`):** Targeted re-reads of bad ranges with a long 60-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
- **Pass 1 (Disc::copy):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
- **Pass 2+ (Disc::patch):** Targeted re-reads of bad ranges with a long 30-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
- **In-stream (DiscStream):** Adaptive batch halving -- reduces request size on failure to isolate bad sectors within a larger block.
This means a disc with some bad sectors will still produce a usable ISO. The damaged areas are zero-filled in pass 1 and retried in subsequent passes. Structure-protected sectors (deliberate unreadable regions from copy protection) will never yield, which is expected.
@ -302,7 +302,7 @@ Many external drive enclosures (Vantec NexStar, Sabrent, OWC, etc.) do not adver
When something goes wrong during a rip, gather this information before filing an issue:
1. **freemkv version:** `freemkv --version` or the crate version in `Cargo.toml`.
2. **Drive model:** from the drive label, or from `freemkv info disc://`.
2. **Drive model:** from the drive label, or from `freemkv info`.
3. **Connection type:** USB (with bridge chipset if known) or direct SATA.
4. **Operating system and kernel:** `uname -a`.
5. **Kernel messages during the failure:** `dmesg | tail -50` immediately after the crash.

76
build.rs
View file

@ -1,6 +1,4 @@
fn main() {
emit_git_suffix();
let target = std::env::var("CARGO_CFG_TARGET_OS").unwrap_or_default();
if target == "macos" {
println!("cargo:rustc-link-lib=framework=IOKit");
@ -10,22 +8,8 @@ fn main() {
let obj = format!("{out_dir}/macos_shim.o");
let lib = format!("{out_dir}/libmacos_scsi.a");
// Build the shim for the TARGET arch, not the host's. A bare `cc` on an
// Apple-Silicon CI runner defaults to arm64, so cross-building to
// x86_64-apple-darwin would link a host-arch object against x86_64 Rust
// code → "Undefined symbols for architecture x86_64". (Still raw `cc`,
// not the `cc` crate, which breaks IOKit exclusive access.)
let target_arch = std::env::var("CARGO_CFG_TARGET_ARCH").unwrap_or_default();
let clang_arch: &str = if target_arch == "aarch64" {
"arm64"
} else {
&target_arch // x86_64 → x86_64
};
let cc_status = std::process::Command::new("cc")
std::process::Command::new("cc")
.args([
"-arch",
clang_arch,
"-c",
"src/scsi/macos_shim.c",
"-o",
@ -38,67 +22,15 @@ fn main() {
"-O2",
])
.status()
.expect("failed to spawn cc for macos_shim.c");
// `.status()` succeeding only means the process RAN. A real compile error
// exits non-zero, and ignoring that left no object file, which surfaced
// much later as an unexplained link failure against a missing symbol. The
// shim is macOS-only and is neither linted nor compiled on the other two
// platforms, so a mistake in it has exactly one chance to be noticed.
assert!(cc_status.success(), "cc failed to compile macos_shim.c");
.expect("failed to compile macos_shim.c");
let ar_status = std::process::Command::new("ar")
std::process::Command::new("ar")
.args(["rcs", &lib, &obj])
.status()
.expect("failed to spawn ar");
assert!(ar_status.success(), "ar failed to create the static lib");
.expect("failed to create static lib");
println!("cargo:rustc-link-search=native={out_dir}");
println!("cargo:rustc-link-lib=static=macos_scsi");
println!("cargo:rerun-if-changed=src/scsi/macos_shim.c");
}
}
/// Bake the git short hash into the build as `GIT_SUFFIX` so any muxed MKV or
/// FVI index is traceable to the exact source revision (e.g. ` (g835cc99)`).
/// Empty when git or the repo is unavailable (e.g. a crates.io tarball build),
/// leaving just the package version. Always emitted so `env!("GIT_SUFFIX")`
/// resolves on every target.
fn emit_git_suffix() {
// Version label for the muxing-app / FVI generator tag. `FREEMKV_BUILD_LABEL`
// overrides the Cargo package version when set (non-empty) — used to stamp a
// pre-release/test build without bumping Cargo.toml and disturbing the
// tag-pinned [patch] version matching. Unset → the package version.
let version = std::env::var("FREEMKV_BUILD_LABEL")
.ok()
.filter(|s| !s.trim().is_empty())
.or_else(|| std::env::var("CARGO_PKG_VERSION").ok())
.unwrap_or_default();
println!("cargo:rustc-env=FREEMKV_VERSION={version}");
println!("cargo:rerun-if-env-changed=FREEMKV_BUILD_LABEL");
let suffix = git_short_hash()
.map(|h| format!(" (g{h})"))
.unwrap_or_default();
println!("cargo:rustc-env=GIT_SUFFIX={suffix}");
// Re-run when HEAD (or the branch it points at) moves so the stamp stays
// current without a clean rebuild.
println!("cargo:rerun-if-changed=.git/HEAD");
if let Ok(head) = std::fs::read_to_string(".git/HEAD")
&& let Some(ref_path) = head.strip_prefix("ref: ")
{
println!("cargo:rerun-if-changed=.git/{}", ref_path.trim());
}
}
fn git_short_hash() -> Option<String> {
let out = std::process::Command::new("git")
.args(["rev-parse", "--short=7", "HEAD"])
.output()
.ok()?;
if !out.status.success() {
return None;
}
let h = String::from_utf8(out.stdout).ok()?.trim().to_string();
if h.is_empty() { None } else { Some(h) }
}

120
ci/leak-guard.sh
View file

@ -1,120 +0,0 @@
#!/usr/bin/env bash
#
# leak-guard.sh — self-contained public-repo leak gate.
#
# This is the LAST line of defense in CI. It is intentionally self-contained:
# public CI cannot reach the private tooling, so this script encodes ONLY the
# generic net — internal infrastructure references, agent-context files, and
# AI-attribution in commit messages. It deliberately contains NO project-
# specific reverse-engineering vocabulary (those words would themselves be a
# leak). The richer private scanner stays private.
#
# Fails (exit 1) if any of the following appear in the repo:
# 1. a tracked CLAUDE.md or .claude/ path (agent context — never public),
# 2. tracked file content matching the internal-infra net,
# 3. a commit message (in the given range) with AI attribution.
#
# Usage:
# leak-guard.sh [<commit-range>]
# <commit-range> optional git rev-list range to scan commit messages
# (e.g. "abc..def"). If omitted, commit-message scan is
# skipped (path + content checks always run).
set -euo pipefail
# Absolute path to this script, resolved before any cd, so we can exclude it
# from the content scan (it necessarily contains the detection patterns).
SELF_ABS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/$(basename "${BASH_SOURCE[0]}")"
REPO="$(git rev-parse --show-toplevel)"
cd "$REPO"
fail=0
note() { printf ' ✗ %s\n' "$1"; fail=1; }
# Internal-infra net — GENERIC ONLY. This script ships in the public repo, so
# the patterns themselves must not name any org-specific identifier (doing so
# would itself leak the infra they guard). We catch the leak *class*:
# - RFC1918 private IPv4 ranges (10/8, 172.16/12, 192.168/16),
# - private/internal/non-routable TLDs (.internal/.local/.lan/.corp/.invalid),
# - docker.internal.
# The full org-specific net (literal hostnames, service names, repo paths,
# vendor tooling, …) lives ONLY in the private scanner and never ships here.
INFRA_RE='\b10\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}|\b172\.(1[6-9]|2[0-9]|3[01])\.[0-9]{1,3}\.[0-9]{1,3}|\b192\.168\.[0-9]{1,3}\.[0-9]{1,3}|\.internal\b|\.local\b|\.lan\b|\.corp\b|\.invalid\b|docker\.internal'
# Home-path net — GENERIC ONLY. Catches an absolute developer home path
# committed into a tracked file (a macOS /Users/<user>/… or Linux /home/<user>/…
# path). This names NO specific user — it matches the leak *class* (any home
# path), so the pattern itself reveals nothing org- or person-specific. A real
# leak (e.g. /Users/alice/Developer/x slipping into a public RELEASE.md) trips
# this regardless of whose machine it came from. The username segment is a
# literal-username class ([A-Za-z0-9._-]) so dynamic/templated paths that build
# the user at runtime — shell `/home/$USER/`, doc `/home/<rip>/`, Rust
# `/home/{user}/` — do NOT false-positive; only a baked-in literal home leaks.
HOMEPATH_RE='/Users/[A-Za-z0-9._-]+/|/home/[A-Za-z0-9._-]+/'
# AI-attribution net (case-insensitive). "claude" matches only as a standalone
# word — NOT preceded by a dot/slash/alnum and NOT followed by .md — so legit
# mentions of CLAUDE.md / .claude/ in a commit message don't false-positive.
ATTR_RE='co-authored-by|generated with|🤖|(?<![.\/A-Za-z0-9])claude(?!\.md)'
echo "── leak-guard: tracked agent-context paths ──"
while IFS= read -r f; do
case "$f" in
CLAUDE.md|*/CLAUDE.md|.claude|.claude/*|*/.claude|*/.claude/*)
note "tracked agent-context file: $f (CLAUDE.md/.claude must never be tracked in a public repo)" ;;
esac
done < <(git ls-files)
# Match a PCRE against a file, emitting "LINE: MATCH". The pattern is passed as
# an argument (not interpolated into a //) so metacharacters like the "/" in a
# path-style token can't break the regex. Reads raw bytes so non-UTF-8 blobs
# don't abort the scan.
pcre_matches() {
perl -e '
my ($file, $re) = @ARGV;
open(my $fh, "<:raw", $file) or exit 0;
my $rx; eval { $rx = qr/$re/i }; exit 0 if $@;
while (my $l = <$fh>) { if ($l =~ /$rx/) { print "$.: $&\n"; } }
' "$1" "$2" 2>/dev/null
}
# This script's own source necessarily contains the detection patterns (e.g.
# the regex tokens in INFRA_RE), so scanning it would always self-flag. Skip it.
SELF="$(git ls-files --full-name -- "$SELF_ABS" 2>/dev/null | head -1)"
echo "── leak-guard: internal-infra references in tracked files ──"
while IFS= read -r f; do
case "$f" in *.png|*.jpg|*.jpeg|*.ico|*.gif|*.bin|*.crate|*.gz|*.zip|*.pdf) continue ;; esac
[ -n "$SELF" ] && [ "$f" = "$SELF" ] && continue
[ -f "$f" ] || continue
while IFS= read -r hit; do
[ -z "$hit" ] && continue
note "internal-infra reference: $f:$hit"
done < <(pcre_matches "$f" "$INFRA_RE")
while IFS= read -r hit; do
[ -z "$hit" ] && continue
note "[HOME-PATH] absolute home path: $f:$hit (no local home path may be committed to a public repo)"
done < <(pcre_matches "$f" "$HOMEPATH_RE")
done < <(git ls-files)
RANGE="${1:-}"
if [ -n "$RANGE" ]; then
echo "── leak-guard: AI-attribution in commit messages ($RANGE) ──"
while IFS= read -r sha; do
[ -z "$sha" ] && continue
msg="$(git log -1 --format='%B' "$sha" 2>/dev/null || true)"
# Pass the pattern as an argument (not interpolated into a //) so the
# lookbehind char class and "/" don't break the regex.
hit="$(printf '%s' "$msg" | perl -e '
my $re = $ARGV[0]; my $rx = qr/$re/i;
while (my $l = <STDIN>) { if ($l =~ /($rx)/) { print "$1\n"; last; } }
' "$ATTR_RE" | head -1 || true)"
[ -n "$hit" ] && note "commit ${sha:0:12}: message contains \"$hit\" (owner rule: zero AI attribution, ever)"
done < <(git rev-list "$RANGE" 2>/dev/null || true)
fi
echo
if [ "$fail" -ne 0 ]; then
echo "✗ leak-guard: blocking finding(s) above — DO NOT MERGE/PUBLISH"
exit 1
fi
echo "✓ leak-guard: clean"

302
docs/FVI_FORMAT.md
View file

@ -1,302 +0,0 @@
# FVI — Freemkv Video Index Format
**Specification version:** 1.0 (DRAFT)\
**File extension:** `.fvi`\
**Media type:** `application/vnd.freemkv.fvi+jsonl`\
**Status:** Draft for review. This document is the normative reference for the FVI
format; implementations and downstream tools cite it by section.
---
## 1. Scope and purpose
FVI is an open, codec-agnostic, byte-exact **index of the coded pictures** in a
video bitstream, together with **provenance** back to the source medium.
An FVI document answers, for every picture in a stream, three questions:
1. **Where is it?** — the byte-exact offset of its first byte in the *source*
(the disc/ISO/file), so a reader can extract or seek to any picture without
re-parsing the whole bitstream.
2. **What is it?** — coding type, random-access capability, GOP boundary, and
(where the codec defines them) field/pulldown attributes.
3. **When is it?** — decode and presentation timestamps on a declared timescale.
FVI is **not** a container, a codec, or a copy of the bitstream. It indexes; it
never stores coded samples. It is the serialized form of an indexer's per-picture
truth — carried from the demuxer, **never reconstructed** (§9).
## 2. Conformance
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**,
**SHOULD**, **SHOULD NOT**, **MAY**, and **OPTIONAL** are to be interpreted as
described in BCP 14 (RFC 2119, RFC 8174) when, and only when, they appear in all
capitals.
A **conformant writer** MUST emit a document that satisfies §4–§10. A
**conformant reader** MUST accept any such document and MUST ignore unknown
object members (§11) so that forward-compatible extensions do not break it.
## 3. Terminology
- **Picture** — one coded video frame (or pair of fields coded as a frame). The
unit FVI indexes.
- **Access unit (AU)** — the set of bitstream bytes that decode to exactly one
picture (ISO/IEC 14496-10 §3; ISO/IEC 23008-2 §3).
- **Coded order** — the order pictures appear in the bitstream. FVI records are
emitted in coded order.
- **GOP / coded video sequence** — a self-contained run beginning at a
random-access point.
- **Provenance** — the mapping from an AU back to the exact bytes of the physical
source it was read from (§9).
- **Source position (`src`)** — `{ file, sector, byte }`, the provenance anchor of
an AU.
## 4. Encoding
An FVI document is a sequence of **UTF-8** text lines separated by a single LF
(`U+000A`). Each non-empty line is exactly one JSON value (RFC 8259), forming a
**JSON Lines / NDJSON** stream. A writer MUST NOT emit a UTF-8 BOM. A writer MUST
NOT pretty-print: each JSON value occupies exactly one line.
The first line MUST be the **Header** object (§6). Each subsequent line is one
**Picture record** (§7), in coded order.
Rationale: line-delimited JSON is streamable (a writer appends as it indexes; a
reader processes without loading the whole file), line-addressable (picture *n*
is near line *n+1*), append-safe, and parseable by every language without a
custom grammar — while remaining a precisely specified format, not an ad-hoc dump.
A document MAY be concatenated for multiple elementary streams: each stream is its
own header line followed by its records. Readers MUST treat a Header line as the
start of a new stream section.
## 5. Document structure
```
<header> line 1 (exactly one Header object)
<record> line 2 .. N (one Picture record per picture, coded order)
[<header> <record>…] (OPTIONAL further stream sections)
```
## 6. Header object
| Member | JSON type | Req | Semantics / reference |
|---|---|---|---|
| `format` | string | MUST | Constant `"freemkv/video-index"`. Signature: a document begins with these bytes. |
| `fvi_version` | integer | MUST | Document format version. This spec defines `1`. |
| `generator` | string | SHOULD | Producing tool + version, e.g. `"freemkv/1.0.0-rc.6"`. |
| `stream` | object | MUST | The indexed elementary stream (§6.1). |
| `source` | object | MUST | Provenance root (§6.2). |
| `timescale` | integer | MUST | Ticks per second for all `pts`/`dts` (§10). E.g. `90000`. |
| `picture_count` | integer | MAY | Total pictures, if known at header time; OMITTED when streaming. |
### 6.1 `stream` object
| Member | JSON type | Req | Semantics / reference |
|---|---|---|---|
| `codec` | string | MUST | Registered codec id (Appendix B), e.g. `"mpeg2video"`, `"hevc"`. |
| `width`,`height` | integer | MUST | Coded luma dimensions in pixels. |
| `dar` | `[int,int]` | SHOULD | Display aspect ratio as `[num,den]`. |
| `frame_rate` | `[int,int]` | SHOULD | Nominal rate as exact rational `[num,den]` (e.g. `[24000,1001]`). |
| `scan` | string | MUST | `"progressive"`<br>`"interlaced"`<br>`"mbaff"` |
| `colour` | object | SHOULD | CICP per ITU-T H.273: `primaries`, `transfer`, `matrix` (integer CICP codes or registered names)<br>`range`: `"limited"` \| `"full"`<br>HDR: `mastering_display`, `max_cll`, `max_fall` per ITU-T H.273 / SMPTE ST 2086. |
| `language` | string | MAY | BCP 47 tag, if known. |
### 6.2 `source` object
| Member | JSON type | Req | Semantics / reference |
|---|---|---|---|
| `medium` | string | MUST | `"disc"`<br>`"iso"`<br>`"file"`<br>`"stream"` |
| `path` | string | MAY | Source path/label. |
| `title` | integer | MAY | Title/program number. |
| `playlist` | string | MAY | Playlist/PGC identifier. |
| `volume_id` | string | MAY | Disc volume identifier, if read. |
| `sector_size` | integer | SHOULD | Bytes per `src.sector` unit (e.g. `2048`). Lets readers convert `src` to an absolute byte offset. |
## 7. Picture record
One JSON object per coded picture, in coded order.
| Member | JSON type | Req | Semantics / reference |
|---|---|---|---|
| `n` | integer | MUST | Coded-order index, 0-based, contiguous. |
| `src` | object | MUST | Provenance: `{ "file": int?, "sector": uint, "byte": uint }` — the offset of this AU's **first byte** in the source (§9). MUST be carried from demux, never reconstructed. |
| `type` | string | MUST | Coding type:<br>`"I"`<br>`"P"`<br>`"B"`<br>_ISO/IEC 13818-2 §6.3.9; H.264/H.265 slice types collapsed to frame type._ |
| `key` | boolean | MUST | `true` iff this picture is an intra (I) picture / parser-flagged decode-restart point (IDR / IRAP / I-picture).<br>_MPEG-2 open-GOP clean-RAP precision (`closed_gop`) is not currently distinguished — see note below._ |
| `gop` | boolean | SHOULD | `true` iff this picture begins a GOP / coded video sequence.<br>_Omitted when the implementation does not carry a distinct GOP-boundary signal._ |
| `pts` | integer\|null | SHOULD | Presentation timestamp in `timescale` ticks; `null` if unknown. |
| `dts` | integer\|null | MAY | Decode timestamp in `timescale` ticks. |
| `size` | integer | MAY | AU length in bytes; enables byte-range extraction with `src`. |
| `recovered` | boolean | MAY | `true` iff any byte of this AU came from a retried/marginal read (§9.1).<br>_Default `false`._ |
| codec ext | object | MAY | Codec-specific members under the codec's namespace (§8). |
The `type` and `key` members are **codec-agnostic** and MUST be populated for
every codec. `type` is the I/P/B coding type the parser decoded (collapsing
H.264/H.265 slice types to a frame type); where no per-picture coding is carried
(audio / synthetic frames), `type` is `"I"` for a key picture else `"P"`. `key`
is the picture's random-access flag as the codec parser sets it (IDR / IRAP /
I-picture). A writer MUST NOT emit a degraded record (`type:"?"` or `src:null`)
merely because a codec lacks per-picture coding info — those fallbacks are
reserved for a field that is genuinely unavailable (e.g. provenance absent on a
synthetic source).
> **Limitation (honest random-access).** `key` is set from the picture's
> intra / decode-restart flag. The per-picture coding model this index carries
> does **not** distinguish MPEG-2 open-GOP clean random-access points
> (`closed_gop`) from any other I-picture, so `key` is the parser-flagged
> decode-restart point, not a verified clean-RAP claim. A future revision MAY
> tighten `key` for codecs/profiles that carry that signal; readers MUST NOT
> assume present `key` precision beyond "intra / decode-restart point".
### 7.1 Interlace / pulldown fields
Codec-agnostic interlace/pulldown attributes, derived through the indexer's
per-picture coding accessors (MPEG-2: ISO/IEC 13818-2 §6.3.10). Emitted as
top-level members of the record, and ONLY when the codec actually measured the
signal — an OPTIONAL member that is omitted (not defaulted) when unknown:
| Member | JSON type | Req | Semantics / reference |
|---|---|---|---|
| `field_order` | string | MAY | Display field order:<br>`"tff"` — top field first<br>`"bff"` — bottom field first<br>`"progressive"` — no field order applies<br>_Omitted when the codec did not signal it._ |
| `progressive` | boolean | MAY | `true` iff the picture is progressive.<br>_Omitted when the codec did not signal it._ |
| `nb_fields` | integer | MAY | Number of displayed field periods this picture occupies (the soft-telecine / 2:3 pulldown basis):<br>`1` for a single field picture<br>`2` for a normal frame<br>`3`/`4`/`6` for `repeat_first_field` pulldown per §6.3.10 |
Codecs that carry only a coding type (e.g. H.264 / HEVC / VC-1 through this
pipeline) omit `field_order` and `progressive` rather than guessing a default.
## 8. Codec model and extensibility
Core record members (§7) are codec-agnostic and present for every codec.
Codec-specific data is either (a) promoted to top-level members for a small,
registered set per codec profile (e.g. MPEG-2 §7.1), or (b) placed under an
`ext` object keyed by codec id for richer/optional data:
```json
{
"n": 42,
"type": "P",
"key": false,
"src": {
"sector": 17,
"byte": 924
},
"ext": {
"hevc": {
"temporal_id": 0,
"nal_type": 1
}
}
}
```
New codecs and members are added through Appendix B (codec registry) without a
breaking version bump, provided readers continue to ignore unknown members (§11).
## 9. Provenance and recovery semantics
`src` is **byte-exact** to the source as read. `src.sector` counts in
`source.sector_size`-byte units; `src.byte` is the offset within that sector of
the AU's first byte. For multi-file sources, `src.file` indexes a writer-declared
file list. Provenance MUST be the value observed at demux time; an implementation
MUST NOT recompute `src` by re-parsing — the point of FVI is to *carry* the truth.
### 9.1 Recovery
Because FVI is provenance-native, it can record reliability. A record with
`"recovered":true` indicates the AU's source bytes required retry/marginal-read
recovery. This lets downstream tools surface or quarantine pictures whose bytes
are not byte-identical to a clean read — a capability legacy index formats lack.
## 10. Time model
All `pts`/`dts` are integers in units of `1/timescale` seconds. `pts` is
presentation (display) time; `dts` is decode time. Records are in **coded**
(decode) order, so `pts` is not necessarily monotonic across records (B-pictures
reorder); `dts` is non-decreasing. Readers needing display order sort by `pts`.
## 11. Versioning and forward compatibility
- `fvi_version` is the document version; this spec defines `1`.
- **Additive** changes (new OPTIONAL members, new registered codecs) do NOT bump
`fvi_version`. Readers MUST ignore members they do not recognize.
- A change that alters the meaning of an existing member or makes a new member
REQUIRED bumps `fvi_version`.
- A reader encountering a higher `fvi_version` than it implements SHOULD process
the members it understands and MUST NOT reject the document solely for the
version being higher, unless a member it relies on is absent.
## 12. Conformance requirements (summary)
A conformant **writer** MUST: emit a Header first; emit records in coded order
with contiguous `n`; populate `src` from demux; use named/registered codec ids;
encode one JSON value per UTF-8 LF-terminated line.
A conformant **reader** MUST: accept any §4–§10 document; ignore unknown members;
not assume `picture_count`, `pts`, or `size` are present unless required above.
---
## Appendix A — JSON Schema (informative)
Header:
```json
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"required": ["format", "fvi_version", "stream", "source", "timescale"],
"properties": {
"format": { "const": "freemkv/video-index" },
"fvi_version": { "type": "integer", "minimum": 1 },
"timescale": { "type": "integer", "minimum": 1 },
"stream": { "type": "object", "required": ["codec", "width", "height", "scan"] },
"source": { "type": "object", "required": ["medium"] }
}
}
```
Record:
```json
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"required": ["n", "src", "type", "key"],
"properties": {
"n": { "type": "integer", "minimum": 0 },
"type": { "enum": ["I", "P", "B"] },
"key": { "type": "boolean" },
"src": {
"type": "object",
"required": ["sector", "byte"],
"properties": {
"file": { "type": "integer" },
"sector": { "type": "integer", "minimum": 0 },
"byte": { "type": "integer", "minimum": 0 }
}
}
}
}
```
## Appendix B — Registered codec identifiers
| `codec` | Bitstream | Field profile |
|---|---|---|
| `mpeg2video` | ISO/IEC 13818-2 | §7.1 (field_order/progressive/nb_fields) |
| `mpeg1video` | ISO/IEC 11172-2 | §7.1 |
| `h264` | ISO/IEC 14496-10 | core + `ext.h264` |
| `hevc` | ISO/IEC 23008-2 | core + `ext.hevc` |
| `vc1` | SMPTE 421M | core |
## Appendix C — Normative references
- RFC 2119, RFC 8174 — Requirement keywords (BCP 14).
- RFC 8259 — JSON.
- ISO/IEC 13818-2 — MPEG-2 video (picture coding, §6.3.9–6.3.10).
- ISO/IEC 14496-10 — H.264/AVC. ISO/IEC 23008-2 — H.265/HEVC.
- ITU-T H.273 — Coding-independent code points (colour primaries/transfer/matrix).
- SMPTE ST 2086 — Mastering display colour volume (HDR).
- BCP 47 — Language tags.
- RFC 9559 — Matroska (alignment of colour/field-order semantics).

2
docs/README.md
View file

@ -12,7 +12,7 @@ Technical documentation for [libfreemkv](https://github.com/freemkv/libfreemkv),
|----------|---------------|
| [Architecture](architecture.md) | Module map, design principles, error codes, platform support |
| [Drive Access](drive-access.md) | Drive, SCSI transport, profiles, unlock, why raw mode is needed |
| [Rip Recovery](rip-recovery.md) | What this crate owns of the recovery model: single-shot Drive::read, SenseFamily, DiscStream batch halving (the strategy itself moved to freemkv-engine in 1.6.0) |
| [Rip Recovery](rip-recovery.md) | Three-layer recovery model: Disc::patch, single-shot Drive::read, DiscStream batch halving |
| [AACS Encryption](aacs.md) | Key resolution (4 paths), content decryption, bus encryption, SCSI handshake |
| [UDF Filesystem](udf.md) | UDF 2.50 with metadata partitions, pointer chain, how files are read from disc |
| [MPLS Playlists](mpls.md) | Playlist format, play items, STN stream table, coding types |

Some files were not shown because too many files have changed in this diff Show more