Skip to content

fix(player): fall back to MPV playable source when watching anime and classify sentence audio export failures - #96

Closed
Jolimunium wants to merge 9 commits into
sohilsayed:mainfrom
Jolimunium:fix/player-sentence-audio-export
Closed

fix(player): fall back to MPV playable source when watching anime and classify sentence audio export failures#96
Jolimunium wants to merge 9 commits into
sohilsayed:mainfrom
Jolimunium:fix/player-sentence-audio-export

Conversation

@Jolimunium

Copy link
Copy Markdown
Contributor

Problem

When watching anime in the player, exporting sentence audio during scene mining could fail silently or produce generic error toasts without actionable feedback. Typical failure causes included:

  • Invalid MPV track index mappings when MPV's frozen ff-index was missing or mismatched relative to FFmpeg streams.
  • Unreadable or unseekable original video sources (e.g., remote HTTP/stream read errors, DRM, or protected streams).
  • Restricted or unwhitelisted audio codecs.
  • Lack of fallback to MPV's playable video source when the original video source encountered audio extraction issues.
  • Generic Anki media warnings that did not specify the root cause of sentence audio generation failures.

Changes

1. Sentence Audio Fallback & Track Resolution (SceneSentenceAudioService.kt, SceneVideoInput.kt, SceneCaptureRequest.kt)

  • Added SceneVideoInputOrigin (ORIGINAL_VIDEO, PLAYABLE_VIDEO, EXTERNAL_AUDIO) to track source provenance.
  • Introduced ScenePlayableFallbackResolution and resolvePlayableFallback() to safely verify and resolve MPV's playable video source when the original video source fails.
  • Refactored SceneSentenceAudioService with multi-stage probing (Selected Track Probe → All Stream Probe → Unrestricted Discovery Probe) and automatic fallback to the playable source for retryable errors (isPlayableFallbackRetryable).
  • Implemented automatic track index recovery when exactly one readable audio stream exists, even if MPV's ff-index property is unpopulated.

2. Failure Classification & Safe Native Diagnostics (FfmpegKitSceneCommandExecutor.kt, SceneMediaProbe.kt, SentenceAudioDiagnosticLog.kt)

  • Added SceneFfmpegFailure enum and classifySceneFfmpegFailure() to categorize native FFmpeg/FFprobe failures (STREAM_MAPPING, SOURCE_READ, SEEK, OUTPUT_WRITE, UNKNOWN).
  • Enhanced SceneMediaProbe with inspectSelectedAudio() and audioStreams() to detect protection markers, codec types, and stream blocks.
  • Implemented SentenceAudioDiagnosticJournal and StorageFolderSentenceAudioDiagnosticLogger to record diagnostic events in storage while automatically redacting sensitive data (URLs, authorization tokens, headers, and local file paths).

3. Detailed UI Warnings & Translations (AnkiMedia.kt, AnkiCardCreator.kt, PlayerSceneMiningUi.kt, strings.xml, PlayerViewModel.kt)

  • Defined AnkiSentenceAudioFailure, AnkiSentenceAudioDiagnostic, and AnkiSentenceAudioPreparation in the Anki module.
  • Added AnkiMediaWarning.SentenceAudioGenerationFailed and SentenceAudioStorageFailed.
  • Updated PlayerSceneMiningUi.kt and added 27+ localized string resources in strings.xml to present detailed diagnostic toasts specifying whether issues arose from original video sources, MPV playable fallback sources, or external audio tracks.

4. Comprehensive Unit Tests

  • Added and updated tests in eu.kanade.tachiyomi.ui.player.scene:
    • SceneSentenceAudioServiceTest.kt: Playable fallback, track index recovery, probe failures, codec restrictions, external audio, and timeouts.
    • SceneCaptureRequestTest.kt: Subtitle capture request resolution and single-audio tracking.
    • SceneVideoInputTest.kt : Playable fallback resolution and discovery probe arguments.
    • SentenceAudioDiagnosticLogTest.kt : Log redaction (URL/token/header masking) and byte budget retention.
    • SceneFfmpegFailureClassifierTest.kt : FFmpeg failure regex classification.

Result

  • When exporting sentence audio while watching anime, if the original video source fails, the system automatically falls back to MPV's playable video source to successfully generate sentence audio.
  • Users receive precise, localized toast warnings detailing the exact failure reason instead of generic errors.
  • Sensitive credentials and stream URLs in native diagnostic logs are automatically redacted before storage.

- fall back only when the source has exactly one audio stream
- allow audio export from HDR video sources
- show clear warnings when sentence audio generation or Anki storage fails
- validate the selected stream before exporting sentence audio
- fall back only when FFprobe finds exactly one readable audio stream
- distinguish missing, non-audio, and protected stream failures
- distinguish original, MPV playable, and external audio inputs
- detect missing audio streams separately from restricted codecs
- show source-specific sentence audio export warnings
- retry sentence audio export from MPV's playable source only when the original has no audio stream
- preserve safe track mapping and source-specific failure diagnostics
@sohilsayed

Copy link
Copy Markdown
Owner

the animated mining will be reworked so if you can fix the conflicts that would be good

@Jolimunium

Copy link
Copy Markdown
Contributor Author

I've updated this in #106 instead.

@Jolimunium Jolimunium closed this Aug 4, 2026
@Jolimunium
Jolimunium deleted the fix/player-sentence-audio-export branch August 5, 2026 00:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants