Meeting_Assistant/openspec/specs/transcription/spec.md

# transcription Specification

## Purpose
TBD - created by archiving change add-meeting-assistant-mvp. Update Purpose after archive.
## Requirements
### Requirement: Edge Speech-to-Text
The Electron client SHALL perform speech-to-text conversion locally using faster-whisper int8 model.

#### Scenario: Successful transcription
- **WHEN** user records audio during a meeting
- **THEN** the audio SHALL be transcribed locally without network dependency

#### Scenario: Transcription on target hardware
- **WHEN** running on i5 processor with 8GB RAM
- **THEN** transcription SHALL complete within acceptable latency for real-time display

### Requirement: Traditional Chinese Output
The transcription engine SHALL output Traditional Chinese (繁體中文) text.

#### Scenario: Simplified to Traditional conversion
- **WHEN** whisper outputs Simplified Chinese characters
- **THEN** OpenCC SHALL convert output to Traditional Chinese

#### Scenario: Native Traditional Chinese
- **WHEN** whisper outputs Traditional Chinese directly
- **THEN** the text SHALL pass through unchanged

### Requirement: Real-time Display
The Electron client SHALL display transcription results in real-time.

#### Scenario: Streaming transcription
- **WHEN** user is recording
- **THEN** transcribed text SHALL appear in the left panel within seconds of speech

### Requirement: Python Sidecar
The transcription engine SHALL be packaged as a Python sidecar using PyInstaller.

#### Scenario: Sidecar startup
- **WHEN** Electron app launches
- **THEN** the Python sidecar containing faster-whisper and OpenCC SHALL be available

#### Scenario: Sidecar communication
- **WHEN** Electron sends audio data to sidecar
- **THEN** transcribed text SHALL be returned via IPC

### Requirement: Streaming Transcription Mode
The sidecar SHALL support a streaming mode where audio chunks are continuously received and transcribed in real-time with VAD-triggered segmentation.

#### Scenario: Start streaming session
- **WHEN** sidecar receives `{"action": "start_stream"}` command
- **THEN** it SHALL initialize audio buffer and VAD processor
- **AND** respond with `{"status": "streaming", "session_id": "<uuid>"}`

#### Scenario: Process audio chunk
- **WHEN** sidecar receives `{"action": "audio_chunk", "data": "<base64_pcm>"}` during active stream
- **THEN** it SHALL append audio to buffer and run VAD detection
- **AND** if speech boundary detected, transcribe accumulated audio
- **AND** emit `{"segment_id": <int>, "text": "<transcription>", "is_final": true}`

#### Scenario: Stop streaming session
- **WHEN** sidecar receives `{"action": "stop_stream"}` command
- **THEN** it SHALL transcribe any remaining buffered audio
- **AND** respond with `{"status": "stream_stopped", "total_segments": <int>}`

### Requirement: VAD-based Speech Segmentation
The sidecar SHALL use Voice Activity Detection to identify natural speech boundaries for segmentation.

#### Scenario: Detect speech end
- **WHEN** VAD detects silence exceeding 500ms after speech
- **THEN** the accumulated speech audio SHALL be sent for transcription
- **AND** a new segment SHALL begin for subsequent speech

#### Scenario: Handle continuous speech
- **WHEN** speech continues for more than 15 seconds without pause
- **THEN** the sidecar SHALL force a segment boundary
- **AND** transcribe the 15-second chunk to prevent excessive latency

### Requirement: Punctuation in Transcription Output
The sidecar SHALL output transcribed text with appropriate Chinese punctuation marks.

#### Scenario: Add sentence-ending punctuation
- **WHEN** transcription completes for a segment
- **THEN** the output SHALL include period (。) at natural sentence boundaries
- **AND** question marks (？) for interrogative sentences
- **AND** commas (，) for clause breaks within sentences

#### Scenario: Detect question patterns
- **WHEN** transcribed text ends with question particles (嗎、呢、什麼、怎麼、為什麼)
- **THEN** the punctuation processor SHALL append question mark (？)

### Requirement: Audio File Upload
The Electron client SHALL allow users to upload pre-recorded audio files for transcription.

#### Scenario: Upload audio file
- **WHEN** user clicks "Upload Audio" button in meeting detail page
- **THEN** file picker SHALL open with filter for supported audio formats (MP3, WAV, M4A, WebM, OGG)

#### Scenario: Show upload progress
- **WHEN** audio file is being uploaded
- **THEN** progress indicator SHALL be displayed showing upload percentage

#### Scenario: Show transcription progress
- **WHEN** audio file is being transcribed in chunks
- **THEN** progress indicator SHALL display "Processing chunk X of Y"

#### Scenario: Replace existing transcript
- **WHEN** user uploads audio file and transcript already has content
- **THEN** confirmation dialog SHALL appear before replacing existing transcript

#### Scenario: File size limit
- **WHEN** user selects audio file larger than 500MB
- **THEN** error message SHALL be displayed indicating file size limit

### Requirement: VAD-Based Audio Segmentation
The sidecar SHALL segment large audio files using Voice Activity Detection before cloud transcription.

#### Scenario: Segment audio command
- **WHEN** sidecar receives `{"action": "segment_audio", "file_path": "...", "max_chunk_seconds": 300}`
- **THEN** it SHALL load audio file and run VAD to detect speech boundaries

#### Scenario: Split at silence boundaries
- **WHEN** VAD detects silence gap >= 500ms within max chunk duration
- **THEN** audio SHALL be split at the silence boundary
- **AND** each chunk exported as WAV file to temp directory

#### Scenario: Force split for continuous speech
- **WHEN** speech continues beyond max_chunk_seconds without silence gap
- **THEN** audio SHALL be force-split at max_chunk_seconds boundary

#### Scenario: Return segment metadata
- **WHEN** segmentation completes
- **THEN** sidecar SHALL return list of segments with file paths and timestamps

### Requirement: Dify Speech-to-Text Integration
The backend SHALL integrate with Dify STT service for audio file transcription.

#### Scenario: Transcribe uploaded audio with chunking
- **WHEN** backend receives POST /api/ai/transcribe-audio with audio file
- **THEN** backend SHALL call sidecar for VAD segmentation
- **AND** send each chunk to Dify STT API sequentially
- **AND** concatenate results into final transcript

#### Scenario: Supported audio formats
- **WHEN** audio file is in MP3, WAV, M4A, WebM, or OGG format
- **THEN** system SHALL accept and process the file

#### Scenario: Unsupported format handling
- **WHEN** audio file format is not supported
- **THEN** backend SHALL return HTTP 400 with error message listing supported formats

#### Scenario: Dify chunk transcription
- **WHEN** backend sends audio chunk to Dify STT API
- **THEN** chunk size SHALL be under 25MB to comply with API limits

#### Scenario: Transcription timeout per chunk
- **WHEN** Dify STT does not respond for a chunk within 2 minutes
- **THEN** backend SHALL retry up to 3 times with exponential backoff

#### Scenario: Dify STT error handling
- **WHEN** Dify STT API returns error after retries
- **THEN** backend SHALL return HTTP 502 with error details

### Requirement: Dual Transcription Mode
The system SHALL support both real-time local transcription and file-based cloud transcription.

#### Scenario: Real-time transcription unchanged
- **WHEN** user records audio in real-time
- **THEN** local sidecar SHALL process audio using faster-whisper (existing behavior)

#### Scenario: File upload uses cloud transcription
- **WHEN** user uploads audio file
- **THEN** Dify cloud service SHALL process audio via chunked upload

#### Scenario: Unified transcript output
- **WHEN** transcription completes from either source
- **THEN** result SHALL be displayed in the same transcript area in meeting detail page

### Requirement: Model Download Progress Display
The sidecar SHALL report Whisper model download progress to enable UI feedback.

#### Scenario: Emit download start
- **WHEN** Whisper model download begins
- **THEN** sidecar SHALL emit JSON to stdout: `{"status": "downloading_model", "model": "<size>", "progress": 0, "total_mb": <size>}`

#### Scenario: Emit download progress
- **WHEN** download progress updates
- **THEN** sidecar SHALL emit JSON: `{"status": "downloading_model", "progress": <percent>, "downloaded_mb": <current>, "total_mb": <total>}`
- **AND** progress updates SHALL occur at least every 5% or every 5 seconds

#### Scenario: Emit download complete
- **WHEN** model download completes
- **THEN** sidecar SHALL emit JSON: `{"status": "model_downloaded", "model": "<size>"}`
- **AND** proceed to model loading

#### Scenario: Skip download for cached model
- **WHEN** model already exists in huggingface cache
- **THEN** sidecar SHALL NOT emit download progress messages
- **AND** proceed directly to loading

### Requirement: Frontend Model Download Progress Display
The Electron frontend SHALL display model download progress to users.

#### Scenario: Show download progress in transcript panel
- **WHEN** sidecar emits download progress
- **THEN** whisper status element SHALL display download percentage and size
- **AND** format: "Downloading: XX% (YYY MB / ZZZ MB)"

#### Scenario: Show download complete
- **WHEN** sidecar emits model_downloaded status
- **THEN** whisper status element SHALL briefly show "Model downloaded"
- **AND** transition to loading state

#### Scenario: Forward progress events via IPC
- **WHEN** main process receives download progress from sidecar
- **THEN** it SHALL forward to renderer via `model-download-progress` IPC channel