Skip to content

build: use patched ffmpeg-kit for animated AVIF - #99

Closed
bee-san wants to merge 1 commit into
sohilsayed:mainfrom
bee-san:agent/ffmpeg-kit-avif-fixes
Closed

build: use patched ffmpeg-kit for animated AVIF#99
bee-san wants to merge 1 commit into
sohilsayed:mainfrom
bee-san:agent/ffmpeg-kit-avif-fixes

Conversation

@bee-san

@bee-san bee-san commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Important

Ready for code review. This is the native dependency required by the animated-AVIF capture work in #101. The artifact and native regressions have been verified; the combined path still needs real-device capture testing before merge.

The problem

Chimahon's animated scene capture asks Android's MediaCodec AV1 encoder to feed FFmpeg's AVIF muxer. The stock com.github.jmir1:ffmpeg-kit:1.17 artifact is based on FFmpeg n7.1, but that native path has several failure modes which cannot be fixed reliably from Kotlin:

  1. A malformed or truncated AV1 sequence header can hang while FFmpeg constructs the av1C box.
    libavformat/av1.c:uvlc() used while (get_bits_left(gb)). If an earlier skip moved the reader past the end, get_bits_left() became negative—and therefore truthy—so the loop did not terminate. This is reached through parse_sequence_header()ff_isom_write_av1c(), exactly the path used when muxing av1_mediacodec output into AVIF.

  2. MediaCodec's codec-configuration data is not always presented in the form expected by the AVIF muxer.
    Some devices expose an AV1CodecConfigurationRecord; the muxer needs the contained AV1 configuration OBUs, not the wrapper bytes.

  3. Generating extradata with a dummy frame and then calling flush() is not a safe encoder reset.
    Android does not define encoder behavior after MediaCodec.flush(). A later packet can retain GOP/reference state tied to the discarded dummy frame, producing an animation whose first visible frame depends on data that was never written.

  4. The muxer could accept a non-key first video packet.
    The resulting AVIF may be structurally present yet fail frame-zero decoding. A missing stss table is ambiguous—it can mean “all samples are sync samples” or that no useful keyframe evidence was written—so app-side box validation alone cannot repair a bad first packet.

These are native parser, encoder-state, and muxer-invariant problems. Moving the command to another process (#100), adding Kotlin retries, or validating the finished file (#101) can contain or detect failure, but none of those makes the native output correct.

What we tried first

Keep stock FFmpegKit and repair the bitstream in Kotlin

The first implementation normalized MediaCodec AV1 OBUs into an intermediate file and then ran a second FFmpeg remux pass. That approach was discarded because it:

  • duplicated AV1/container logic above FFmpeg;
  • created an extra file and an extra native session;
  • made cancellation and file ownership substantially harder;
  • still could not safely reset MediaCodec's internal reference state;
  • treated symptoms after encoding instead of enforcing the invariant at the muxer boundary.

Use process isolation as the fix

An earlier rationale said FFmpegKit had to run outside the player because aniyomi-mpv-lib and FFmpegKit shipped conflicting libav*.so files. Inspection disproved that: aniyomi-mpv-lib does not ship a second set of those libraries; libmpv.so depends on the FFmpeg libraries supplied by FFmpegKit. A separate process may still be useful for defensive global-state/crash containment (#100), but it does not fix these AV1 bugs.

Iterate through narrower native builds

This coordinate is not the first fork build:

  1. 1.17.1 backported only FFmpeg e44d76f61f, which bounded the uvlc() loop. It fixed the hang but not MediaCodec configuration, encoder restart, or frame-zero behavior.
  2. Later direct-mux builds—including 1.17.6.1—added MediaCodec configuration and restart corrections. They still could not distinguish a zero-key stream when stss was absent.
  3. The final build added the native first-packet keyframe guard and was published at a fresh immutable coordinate rather than rewriting an earlier tag.

The final stack also includes FFmpeg eff9ed7bff for MediaCodec bitstream-filter option initialization and 0ee18ff23d for extracting AV1 config OBUs from AV1CodecConfigurationRecord.

Track a mutable build

That was rejected for an upstream dependency. Reviewers need one reproducible coordinate with a fixed source lock, artifact digest, and narrowly scoped native delta.

Why this solution

This PR pins:

com.github.bee-san:ffmpeg-kit:1.17.8.1

The fork keeps the same FFmpeg n7.1 base, components, and Aniyomi SAF/custom-protocol patches as jmir1:ffmpeg-kit:1.17. Its release overlays only the affected libavcodec.so and libavformat.so entries in the verified upstream AAR.

The native patch set fixes the problem at the layer that owns each invariant:

  • bounds-check AV1 UVLC parsing instead of trying to time out a native infinite loop;
  • initialize the MediaCodec bitstream-filter path and extract configuration OBUs from AV1CodecConfigurationRecord;
  • replace the undefined post-extradata flush() sequence with stop → configure → start;
  • reject the first accepted AVIF video packet unless it is marked as a keyframe.

This is preferable to Kotlin OBU surgery because there is one encode/mux operation, one owner of AV1 syntax, and one authoritative first-packet check. #101 still validates the produced AVIF and falls back to the already-captured still image; this dependency does not remove the application-level safety net.

Provenance

  • JitPack coordinate/tag: 1.17.8.1
  • Native release: 1.17.8-native
  • AAR SHA-256: fe66560bd45e28f1a80449568d31b54986601d8836d0c4b3d2158cde68caf4b2
  • The native release includes SOURCE-LOCK.txt, BUILD-PROVENANCE.txt, an AAR entry manifest, and SHA256SUMS.
  • The JitPack-served AAR was verified byte-for-byte against the native release AAR.
  • Required transitive dependency com.arthenica:smart-exception-java:0.2.1 remains pinned separately in this repository.

The tags are annotated and the release artifacts are immutable/checksum-verifiable, but the tags are not cryptographically signed. That is a remaining provenance limitation rather than something this PR hides.

Scope

This PR intentionally changes only gradle/libs.versions.toml. It does not add the Kotlin capture pipeline, worker process, player lifecycle changes, or claim that all devices can encode AV1.

Related split PRs:

Verification

  • Native patch stack has regression coverage for the FFmpeg n7.1 base, including the MediaCodec restart path and rejection of a non-key first AVIF packet.
  • Release assets and hashes are published and immutable.
  • This branch is one dependency-only commit on upstream main; git diff --check passes.
  • Upstream PR CI currently stops at repository-wide Spotless failures in untouched baseline code before it reaches build/tests; this PR does not change the reported files.

Before merge

@bee-san

bee-san commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

My reason for this is that aniyomi is dead. We are using ffmpegkit from Aniyomi. We may miss important security updates, and I'd like 3 commits from upstream ffmpeg to fix avif issues lol

We could uprade to ffmpegnext (which is mobile ffmpeg 8) but our chatgippty reckons thats quite hard

Difficulty: 8/10

This is a native media-stack migration, not a Gradle dependency swap.

Chimahon currently pairs ffmpeg-kit:1.17 with aniyomi-mpv-lib:1.17.n in its version catalog. The mpv library is built against FFmpeg 7.1, excludes its own libav*.so files, and explicitly relies on the matching Aniyomi FFmpegKit package to supply them (README, build configuration).

FFmpegKitNext v8.1.1 uses FFmpeg 8.1.2. Swapping only the AAR would leave the existing libmpv.so linked against an incompatible FFmpeg ABI and likely cause native-load crashes.

@sohilsayed

Copy link
Copy Markdown
Owner

so to sum up my changes i decided that this is a bloated approach just to avoid the main problem of the app not publishing the needed codec so i decided to just ship that codec myself and just use a jni so this and the other ffmpeg prs will be closed however updating ffmpeg is indeed an idea im interested in
sorry for the trouble and kind of wasting your work

@sohilsayed sohilsayed closed this Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants