LYRICCAPTION
Speech to synced Arabic captions, locally.
Upload a video and paste Arabic lyrics; faster-whisper transcribes with word timestamps, words are fuzzy-matched after Arabic normalisation, and karaoke captions are burned in with FFmpeg + libass — with the RTL word order verified by pixel measurement.
LyricCaption is a FastAPI service with a no-build RTL web app. The pipeline: validate → extract and clean audio → faster-whisper word-level transcription biased by the lyrics → Arabic normalisation (diacritics stripped, أ/إ/آ→ا, ة→ه, ى→ي) → Needleman-Wunsch alignment with rapidfuzz → gap recovery → .ass generation with per-word highlighting → burn-in with embedded OFL fonts. If lyrics are empty the AI writes the captions; if the audio is unclear timing falls back to proportional distribution, so the app always produces a result.
The standout engineering detail is the RTL fix: libass lays out tag-separated runs in logical order, so a highlighted-word line renders reversed; pipeline._visual_order() emits runs back-to-front for RTL lines and a test renders real frames to assert the highlight moves right → left. Six templates, seven video filters and four caption animations re-render in seconds because word timing is cached. Claude adds spell-correction, translation, reel selection and titles — all with local fallbacks.
Overview
Audio waveform → speech recognition → transcript → caption timing → rendered video. Every step degrades gracefully and every render embeds its fonts.
Pipeline
The RTL rule
libass lays out tag-separated runs in logical order, so a line where each word carries a colour override renders reversed. The fix emits word runs back-to-front for RTL lines while Latin runs keep their internal order — and the test proves it on pixels, not by eye.
Output
Honest status
- 01No authentication — anyone on the network can open a project; rated the most serious hole
- 02In-memory job state is lost on restart; no concurrency cap
- 03CPU-only rendering is slow on long or 4K video
- 04Source separation (Demucs) is not yet integrated, so accuracy drops with loud music
