Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/workflows/transcript-exchange.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: Transcript exchange

on:
push:
branches: [main, "agent/**"]
pull_request:

permissions:
contents: read

jobs:
test:
runs-on: macos-latest
strategy:
matrix:
python-version: ["3.9", "3.12"]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- run: python -m unittest -v test_transcript_exchange.py
55 changes: 51 additions & 4 deletions HANDOFF.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@
- Word/block editor: `ui_block_editor.py`
- Playback wrapper: `video_player.py`
- Timeline state: `timeline_manager.py`
- Headless transcript exporter: `transcribe_cli.py`
- Versioned transcript exchange: `transcript_exchange.py`

## Rules To Read First
- `CODEX.md`
Expand All @@ -33,6 +35,42 @@
- `VibeAnalyzer.spec` was recreated so the Windows build script has a real target again.
- Deployment notes are documented in `DEPLOYMENT.md`.

## Transcript Exchange Addition

Branch: `agent/transcript-exchange`

The new integration is deliberately isolated from GUI, playback, block highlighting, and core word-timing algorithms.

- `transcript_exchange.py`
- standard-library only;
- normalizes existing `TranscriptSegment(s,e,t,words)` and legacy dictionaries;
- preserves word-level timestamps in `vibe-video-analyzer/transcript` schema version 1;
- validates ordering and finite time values;
- writes JSON without replacing an existing file unless overwrite is explicit.
- `transcribe_cli.py`
- launches no GUI or VLC;
- lazily imports the existing transcription engine after argument parsing;
- always requests word timestamps;
- supports existing Faster-Whisper, VAD, denoise, and optional WhisperX settings;
- emits `<input>.vibe-transcript.json` by default.
- `test_transcript_exchange.py`
- runs without media/model dependencies;
- covers dataclass/dict normalization, word preservation, round-trip JSON, ordering rejection, protected writes, and dependency-free CLI help.
- `.github/workflows/transcript-exchange.yml`
- runs only the lightweight exchange test on Python 3.9 and 3.12.

The intended downstream consumer is Apple Pro Video MCP:

```text
Vibe transcript JSON
→ vibe_transcript_import
→ highlight_rank
→ edit_plan_build
→ FCPXML + SRT
```

Do not claim actual transcription runtime verification until the project environment and models are restored on the Mac.

## Current macOS Status
- Source-level macOS fixes were applied for VLC embedding and a few UI behaviors.
- Runtime is still blocked in this workspace because there is no `.venv` and no installed Python dependencies.
Expand Down Expand Up @@ -63,6 +101,7 @@ Checked on 2026-03-20:
- user requested full rollback.
- Result: those sync experiments were reverted.
- Current guidance: do not re-apply word timing changes directly in core paths without a clearly isolated experimental toggle.
- The transcript exchange addition does not alter those paths; it serializes their current outputs.

## Known Active Concerns
- Word-level highlight sync is still imperfect.
Expand All @@ -74,6 +113,7 @@ Checked on 2026-03-20:
- use an experimental on/off toggle,
- avoid changing default behavior first,
- test against real sample clips before keeping changes.
- The headless CLI runtime has not yet been exercised on the target Mac.

## User Preferences
- Functionality breakage is unacceptable.
Expand All @@ -85,24 +125,31 @@ Checked on 2026-03-20:
## Suggested Workflow For Next Session
1. Read `CODEX.md` and `GEMINI.md`.
2. Read this file.
3. Check `git status`.
4. If working on macOS runtime, restore a usable Python environment first.
5. Install missing dependencies and restore `models/` before judging runtime behavior.
6. Keep risky sync logic behind toggles.
3. Check `git status` and use `agent/transcript-exchange` for the new exchange work.
4. Run `python -m unittest -v test_transcript_exchange.py` before restoring heavy dependencies.
5. If working on macOS runtime, restore a usable Python environment first.
6. Install missing dependencies and restore `models/` before judging runtime behavior.
7. Run `python transcribe_cli.py --help`.
8. Transcribe one short clip to a new JSON path without overwrite.
9. Inspect segment and word timestamps and hand the file to Apple Pro Video MCP.
10. Keep risky sync logic behind toggles.

## Files Worth Inspecting For Future Work
- `main.py`: startup splash and boot flow
- `gui_app.py`: analysis option wiring, state reset, playback loop, UI state
- `engine_core.py`: ASR pipeline and word timestamp generation
- `ui_block_editor.py`: active word highlighting logic
- `video_player.py`: VLC timing behavior and platform embedding
- `transcript_exchange.py`: stable interchange contract
- `transcribe_cli.py`: headless ASR entry point

## Testing Notes
- For startup behavior, verify the splash appears before heavy import work and reaches 100%.
- For new video loading, verify old transcript rows and word blocks are cleared.
- For macOS runtime, verify VLC video renders inside the Tk window and subtitle preview still works.
- For sync work, always test on a real problematic sample, not just synthetic assumptions.
- If sync work is experimental, provide a rollback path and default it to off.
- For transcript exchange, preserve existing JSON by default and compare the exported word times to the in-memory transcript.

## Current Non-Code Diffs
- Temporary preview subtitle files like `temp_preview_A.ass` and `temp_preview_B.ass` may be modified during app usage.
Expand Down
41 changes: 41 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,45 @@ Local video transcription and subtitle editing app built with Tkinter, VLC, Fast
- Playback wrapper: `video_player.py`
- Transcription engine: `engine_core.py`
- Timeline state: `timeline_manager.py`
- Headless transcript export: `transcribe_cli.py`
- Transcript interchange: `transcript_exchange.py`

## Timestamp-Preserving JSON Export

The analyzer already produces `TranscriptSegment(s, e, t, words)` values with word-level `TranscriptWord(word, s, e)` timing. The new isolated exchange path preserves that data in a versioned JSON document without changing the GUI, playback, or word-highlight behavior.

```bash
python transcribe_cli.py "/Users/me/Movies/interview.mov" \
--language ko \
--silero-vad \
--output "/Users/me/Desktop/interview.vibe-transcript.json"
```

Optional precise alignment:

```bash
python transcribe_cli.py input.mov --whisperx-align
```

This requires the existing runtime dependencies and optional WhisperX package. Existing JSON output is preserved unless `--overwrite` is explicit.

The JSON can be handed to Apple Pro Video MCP without losing word timing:

```text
Vibe transcript JSON
→ vibe_transcript_import
→ highlight_rank
→ edit_plan_build
→ FCPXML + SRT
```

See [`docs/TRANSCRIPT_EXCHANGE.md`](docs/TRANSCRIPT_EXCHANGE.md) for the schema, options, and validation boundary.

Lightweight exchange tests do not import VLC, Torch, or Whisper:

```bash
python -m unittest -v test_transcript_exchange.py
```

## Current Platform Status
- Windows packaging flow exists via `build_release.bat` and `VibeAnalyzer.spec`.
Expand All @@ -30,6 +69,7 @@ Verified on 2026-03-20 in this workspace:
4. Restore or download the local speech models into `models/` if local bundled models are expected.
5. Run `python main.py` from the project root.
6. Verify video preview, subtitle preview, analysis start, and export paths.
7. Run `python transcribe_cli.py --help`, then produce one `.vibe-transcript.json` fixture from a short clip.

## Recommended macOS Smoke Test
1. Launch the app from the project root.
Expand All @@ -39,6 +79,7 @@ Verified on 2026-03-20 in this workspace:
5. Start a basic transcription run using CPU mode.
6. Confirm subtitle rows populate and preview subtitles appear in VLC playback.
7. Export `SRT` and confirm the file is created correctly.
8. Run the headless CLI on the same clip and confirm segment/word timestamps exist in the JSON.

## Notes
- `build_release.bat` is Windows-only.
Expand Down
112 changes: 112 additions & 0 deletions docs/TRANSCRIPT_EXCHANGE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# Transcript JSON exchange

`transcript_exchange.py` provides a standard-library-only interchange format that preserves the segment and word timestamps produced by VibeCoding_VideoAnalyzer.

It is intentionally separate from the GUI, playback, and word-highlight paths. Existing application behavior is unchanged.

## Schema

```json
{
"schema": "vibe-video-analyzer/transcript",
"schemaVersion": 1,
"createdAt": "2026-08-16T00:00:00Z",
"media": {
"path": "/Users/me/Movies/interview.mov",
"fileName": "interview.mov",
"durationSeconds": 120.5
},
"engine": {
"application": "VibeCoding_VideoAnalyzer",
"modelId": "large-v3-turbo",
"language": "ko",
"wordTimestamps": true
},
"segments": [
{
"id": "segment-1",
"startSeconds": 1.25,
"endSeconds": 2.5,
"durationSeconds": 1.25,
"text": "안녕하세요",
"words": [
{
"text": "안녕",
"startSeconds": 1.25,
"endSeconds": 1.8
}
]
}
],
"summary": {
"segmentCount": 1,
"wordCount": 1
}
}
```

The exporter accepts the existing `TranscriptSegment(s, e, t, words)` and `TranscriptWord(word, s, e)` data classes as well as their legacy dictionary equivalents.

## Headless transcription

The CLI uses the existing `HyperTranscriptionEngine` but does not launch Tkinter or VLC.

```bash
python transcribe_cli.py "/Users/me/Movies/interview.mov" \
--language ko \
--model large-v3-turbo \
--silero-vad \
--whisperx-align \
--output "/Users/me/Desktop/interview.vibe-transcript.json"
```

`--whisperx-align` requires the optional WhisperX dependency. Without it, Faster-Whisper or the existing MPS backend still emits word timestamps when supported.

Useful options:

```text
--device auto
--beam-size 5
--min-silence-ms 2000
--speech-pad-ms 250
--vad-threshold 0.35
--denoise
--dominant-speaker
--whisper-vad
--silero-vad
--whisperx-align
--remove-punctuation
--overwrite
```

Existing output is preserved unless `--overwrite` is explicit.

## Runtime boundary

The exchange module and its tests do not require media dependencies. Actual transcription still requires the same runtime as the main application, including the configured Whisper model, FFmpeg tooling, NumPy/Torch stack, and optional WhisperX/VAD packages.

The repository currently has no dependency manifest and the macOS runtime remains unverified until the project environment and models are restored.

## Apple Pro Video MCP handoff

The generated JSON is designed to be consumed without losing word timing:

```text
VibeCoding_VideoAnalyzer
→ *.vibe-transcript.json
→ Apple Pro Video MCP vibe_transcript_import
→ highlight_rank
→ edit_plan_build
├─ fcpxml_create_project
└─ subtitle_segment → subtitle_write_srt
```

The analyzer supplies transcription and timestamps. Candidate quality ratings and actual Final Cut Pro compatibility remain separate evidence layers.

## Tests

The lightweight suite verifies normalization, schema round trips, protected writes, ordering validation, and that CLI help works without importing heavy runtime dependencies.

```bash
python -m unittest -v test_transcript_exchange.py
```
Loading
Loading