When cutting a talking-head video from word-level ASR timestamps, the reported boundary between two adjacent words with no gap is not a safe cut point. ASR emits a single shared boundary (word A end == word B start) that is a decision, not a measurement - the true acoustic transition is somewhere in a +/-30ms neighbourhood, and consonant onsets in particular start before the reported time.
Observed on Deepgram nova-3 output, cutting a 63s take down to 44s:
1. Cutting at the shared boundary still bleeds the next word. illiterate ends 44.90, like starts 44.90. Ending the range at exactly 44.90 produced a render that re-transcribed as ...turning illiterate, like - the whole next word audible. Backing off 60ms to 44.84 went the other way: the re-transcription became ...were turning, with illiterate gone entirely, the punchline word destroyed. There was no safe value between them.
2. A leftover consonant becomes a phantom word. guess ends 56.57, iOS starts 56.57. Starting the range at 56.55 kept ~20ms of the /s/; the render re-transcribed as "As iOS friends" - the fragment fused with the next word into a plausible wrong word, the kind of error that survives a careless listen.
3. Windowing filters silently bisect mis-split tokens. Cut-list builders commonly select words with w['start'] >= window_start. ASR had split "iOS" into two tokens (i @56.57, always @56.89). Setting window_start = 56.60 to dodge trap 2 dropped the first token and began the render at 56.892 - cutting the word in half. A 30ms move to the window changed which word survived.
Rules:
- Cut in silence, not at word boundaries. Prefer a boundary with a real inter-word gap (a pause, breath, or phrase edge). If the two words abut, move the cut out to the enclosing phrase rather than splitting the difference - a slightly longer keep beats a mangled word. In case 1 I kept the trailing
like; as casual speech it is invisible, whereas the clipped punchline was not. - Verify by re-transcribing the render, not the plan. Feed the rendered output back through ASR and read the transcript. Both defects above were invisible in the cut list and obvious in one sentence of output text. This is the single highest-value check in transcript-driven editing and it costs a fraction of a cent.
- Resolve low-confidence spans with a second and third model before cutting near them. Two spans came back as
so pippingandi always friends(word confidence 0.46-0.55) exactly where I needed boundaries. Re-transcribing just that 17s slice against nova-3, nova-2 and whisper-large produced unanimous agreement: "So, uh, yeah" (filler, cut it) and "iOS friends" (the sign-off, keep it). Slicing the audio is enough; no need to re-run the whole file.
# slice, then transcribe the slice with several models
ffmpeg -v error -ss 46 -to 63 -i take.MOV -vn -ac 1 -ar 16000 -c:a flac tail.flac
for m in nova-3 nova-2 whisper-large; do
curl -s -X POST "https://api.deepgram.com/v1/listen?model=$m&filler_words=true&punctuate=true&smart_format=false" \
-H "Authorization: Token $DEEPGRAM_KEY" -H "Content-Type: audio/flac" \
--data-binary @tail.flac | jq -r '.results.channels[0].alternatives[0].transcript'
doneAPI note: nova-3 rejects alternatives=3 with HTTP 400. Running separate models is the workaround for wanting more than one hypothesis.