Most bad video summaries are transcript problems, not model problems. Clean the transcript first — speaker labels, filler, misheard jargon — then run a prompt that names the exact fields you want out. Keep timestamps in every line so any claim can be checked against the source in seconds.
Getting an AI video transcription and summary out of the same recording is now a solved problem on paper. Paste a link, wait a minute, get text and bullets.
Then you read the bullets. Nobody is named, half the decisions were never decisions, and one number is simply wrong.
That is almost never the summariser’s fault. It is what happens when a model is handed a raw transcript and asked to do two jobs at once.
This guide covers the one-pass workflow that actually works, with six copy-paste prompts along the way: what to fix in the transcript before you summarise, prompt patterns for meetings, tutorials, and interviews, how to chunk a long transcript without losing the thread, and how to audit a summary before you send it to anyone.
TL;DR
- Summarising a raw transcript fails for four specific reasons: no speaker attribution, no structure, filler, and misheard jargon. All four are fixable before the summary step.
- Clean first, summarise second. One cleanup prompt that deletes filler and corrects a named glossary — and paraphrases nothing — removes most of the problem.
- Name the output fields. “Summarise this meeting” gets you prose. “DECISIONS / ACTIONS / OPEN QUESTIONS, each with a speaker and a timestamp” gets you notes.
- Keep timestamps on every line. They are the only thing that makes a summary checkable in seconds instead of minutes.
- Chunk anything past an hour. Extract structured notes per chunk, then merge — do not ask for a summary of a summary.
- Audit before you send. A second pass that marks each line supported, unsupported, or contradicted catches the errors that matter.
Why summarising a raw transcript fails
A raw transcript is not a document. It is a stream of words with no author, no paragraphs, and no idea which of them mattered.
Hand that to a model and ask for a summary, and it does what it can: it finds the most frequently repeated topics and writes fluent sentences about them. Four things go wrong.
No speaker attribution. Without diarization labels, there is no signal telling the model who committed to what. It will still write “Priya agreed to own the migration” because notes look like that — and the attribution is a guess. This is the single most common way a summary becomes actively misleading rather than merely thin.
No structure. Transcripts are chronological, and meetings are not. A decision made at 04:12 gets reopened at 38:50 and reversed at 41:03. A model summarising linearly reports all three as if they were three separate decisions.
Filler. “Um”, “you know”, “I mean”, and half-started sentences are 5 to 10% of natural speech, and they dilute every span of text the model attends to. Worse, self-corrections read as contradictions: “we’ll ship Friday — no, sorry, Monday” becomes two dates in the notes.
Misheard jargon. Automatic speech recognition transcribes what is phonetically likely, not what your company calls things. Your product “Kestrel” becomes “kestral”, “central”, or “the strel”, and the summary confidently discusses whichever one appeared most.
There is a fifth, rarer failure worth knowing about because it is invisible: the transcript can contain sentences nobody said.
Cornell researchers led by Allison Koenecke ran more than 13,000 speech clips through Whisper and found that roughly 1% of transcriptions contained entirely fabricated phrases — text with no counterpart in the audio at all. Their paper, “Careless Whisper: Speech-to-Text Hallucination Harms,” was presented at ACM FAccT in 2024.
Silence and disfluent speech were the main triggers.
One percent sounds small. On a 45-minute meeting transcript, it is a handful of sentences that a summariser will treat as gospel.
The AI video transcription and summary workflow, end to end
Five stages. The middle one is the one everybody skips.
| Stage | What happens | Where it goes wrong |
|---|---|---|
| 1. Transcribe | Audio to timestamped text, with speaker labels | Diarization off by default; jargon misheard |
| 2. Label | Speaker 1 and Speaker 2 become real names | Skipped, so attribution is invented later |
| 3. Clean | Filler removed, glossary corrected, timestamps kept | Skipped, or done with a paraphrasing prompt that destroys the source |
| 4. Summarise | A field-named prompt produces structured output | Asked for “a summary”, gets prose nobody can act on |
| 5. Audit | Every claim checked against the transcript | Skipped entirely |
“One pass” means one sitting, not one prompt. You are running four prompts over one transcript, and the whole loop takes about ten minutes on an hour of footage.
If you want the mechanics of stage 1 — how acoustic models, diarization, and forced alignment produce that timestamped text — that is covered separately in our guide to how AI video transcription works and what it costs. For a tool that does stages 1 and 4 in one product, see the AI video readers and summarizers ranked by summary quality.
This guide is about what you do with the transcript once you have it.
Clean the transcript before you summarise it
Three fixes, in this order. Two of them you do by hand because they take ninety seconds and a model will get them wrong.
Fix the speaker labels yourself. Open the transcript, find the first turn from each speaker, and rename Speaker 1 to the person’s actual name. Do a global find-and-replace. Then spot-check the first turn after each long pause — that is where diarization systems most often reassign labels, because a new voice after silence looks like a new speaker.
Overlapping speech is the hard case. When two people talk at once, diarization systems either drop one or braid them together, and no prompt recovers it.
If a decision was made during a crosstalk moment, listen to that thirty seconds yourself.
Build a glossary before you clean. Skim the transcript for your product names, people’s surnames, and acronyms. Write down every wrong spelling you see next to the right one. This is the input to the next prompt, and it is worth more than any other single thing you do.
Then run one cleanup prompt. The critical constraint is that it must not paraphrase. The moment a cleanup prompt is allowed to rewrite, the transcript stops being evidence and the audit step in stage 5 becomes worthless.
You are cleaning a raw ASR transcript. Do not summarise, paraphrase,
reorder, or improve anything.
Rules:
1. Keep every substantive word. You may delete only filler: um, uh, er,
"you know", "I mean", "sort of", "like" used as filler, and immediate
self-repetitions ("the the", "we we").
2. Keep speaker labels exactly as given. If a label is obviously wrong —
a line that answers its own question, or a speaker switch mid-sentence —
merge it into the neighbouring turn and mark that line [merged].
3. Fix only these terms, wherever they appear:
kestral / central / the strel -> Kestrel
<heard> -> <correct>
<heard> -> <correct>
Change no other wording.
4. Keep every timestamp exactly where it is.
5. Add punctuation and sentence breaks. Do not add any word that carries
meaning.
6. Where the text is incoherent or marked [inaudible], leave it and add
[unclear].
Return the cleaned transcript only. No preamble, no commentary.Run it, then diff the output against the input if your editor can. You are looking for one thing: sentences that changed meaning, of which there should be none.
Keep timestamps alive through summarisation
A summary without timestamps is a claim you have to take on faith. A summary with them is a claim you can check in five seconds.
Models drop timestamps constantly when you ask for them as a general instruction, because “include timestamps” competes with every other instruction in the prompt. They keep them reliably when the timestamp is a named slot in a template.
Compare:
- Weak: “Summarise the decisions and include timestamps where relevant.”
- Strong:
- <decision> — decided by <speaker> [<timestamp>]
The second is a shape to fill in. The first is advice.
Two more rules that hold across every pattern below:
- Feed timestamps in. If your transcript is a wall of prose with no time markers, no prompt can add them. Export SRT or VTT, or a timestamped text format, rather than plain text. The four ways to transcribe a YouTube video guide covers which export formats keep them.
- Round, don’t invent. Ask for the timestamp of the nearest transcript line, not a precise moment. A model asked for precision it does not have will manufacture it.
Prompt pattern 1: meeting notes
The goal is notes someone who missed the meeting can act on. That means four fields — decisions, owners, deadlines, open questions — and a hard line between what was decided and what was merely discussed.
You are turning a cleaned meeting transcript into notes for people who
were not in the room.
Source: the transcript below. Use nothing else. If it is not in the
transcript, it does not go in the notes.
Produce exactly these five sections, in this order.
1. DECISIONS
- <decision> — decided by <speaker> [<timestamp>]
A decision is something a named person committed to. "We should
probably..." and "maybe we could..." are NOT decisions — put those in
OPEN QUESTIONS. If a decision was made and later reversed, report only
the final state and add "(reversed earlier decision at <timestamp>)".
2. ACTIONS
- <owner>: <action> — due <date, or "no date stated"> [<timestamp>]
The owner must be a named person who accepted the action out loud.
Never write "the team", "someone", or "TBD" as an owner. If nobody
accepted it, it is an open question, not an action.
3. OPEN QUESTIONS
- <question> — raised by <speaker> [<timestamp>]
Anything raised and left unresolved, including disagreements that did
not land anywhere.
4. CONTEXT
At most five bullets of background a new reader needs to understand the
decisions above. No timestamps needed.
5. NOT COVERED
Anything referenced as "we'll come back to this" or "next time" that
never got discussed. [<timestamp>]
Rules:
- Every line in sections 1-3 ends with a timestamp from the transcript.
- If a section has no entries, write "None." Do not pad it.
- Do not infer intent. Where a decision is ambiguous, quote the words.
- Do not merge two people's statements into one line.
Transcript:
<paste cleaned transcript>The “None.” instruction matters more than it looks. Without it, a model faced with a meeting that decided nothing will manufacture decisions — the template has a slot, and slots get filled.
The reversal clause is the other quiet win. It is the fix for the chronology problem — a linear reading reports a decision, its reversal, and its replacement as three parallel outcomes.
Prompt pattern 2: tutorials and walkthroughs
Different job. Here you are converting a demonstration into a procedure someone can follow with the video closed.
The failure mode is different too: models smooth tutorials into narrative prose, and they silently add the obvious steps the presenter skipped.
Turn this cleaned tutorial transcript into a written procedure someone can
follow without watching the video.
Output these four sections.
**What you need**
Prerequisites stated or shown before the first step. Bullet list.
**Steps**
A numbered list. Each step is exactly this shape:
N. [<timestamp>] <imperative sentence: the action> — <what the result
should look like>
One action per step. If the speaker does two things in one sentence, split
it into two steps.
**Where people get stuck**
Every point where the speaker says "make sure", "don't forget", "this is
where it usually breaks", "the mistake people make", or similar. Quote the
warning and give the timestamp.
**Not shown**
Anything the speaker skipped, sped through, or did off-camera ("I've
already set this up"). Give the timestamp.
Rules:
- Use the speaker's exact names for buttons, menus, files, and settings.
Do not standardise or correct them.
- If a step's outcome is only visible on screen and never spoken, write
"[visual only — verify]" as the result instead of guessing.
- Do not add steps that are not in the transcript, even obvious ones like
"save the file" or "log in".
- Do not reorder steps into a more logical sequence. Keep the recorded order.
Transcript:
<paste cleaned transcript>[visual only — verify] is the important line. A screen recording carries most of its information in the picture, and the transcript only has the audio — so a model filling those gaps is guessing at UI it has never seen.
Marking the gap is far more useful than a plausible sentence covering it up.
Prompt pattern 3: interviews
Interviews need attribution and verbatim quotes, because the output usually becomes something published. The rule that does the heavy lifting is the one forbidding an unquotable claim.
Extract the claims from this interview transcript.
For each substantive claim the interviewee makes, output this block:
- CLAIM: <one sentence, in your words>
SAID BY: <speaker name>
QUOTE: "<verbatim from the transcript, 30 words maximum>"
TIMESTAMP: <timestamp>
TYPE: fact | opinion | prediction | anecdote
EVIDENCE GIVEN: <what they offered to support it, or "none">
Then two more sections.
**Contradictions**
Any place the same speaker says two things that cannot both be true.
Quote both, with timestamps.
**Unanswered**
Questions the interviewer asked that were not answered, or were answered
about something else. Quote the question, with its timestamp.
Rules:
- Never attribute the interviewer's framing to the interviewee. If the
interviewer states a claim and the interviewee only says "right" or
"mm-hm", the claim belongs to the interviewer — label it as such.
- Every QUOTE must appear word for word in the transcript. If you cannot
quote it verbatim, drop the claim entirely.
- TYPE "fact" is only for checkable statements about the past or present.
Anything about the future is a prediction, however confidently said.
- Skip pleasantries, scheduling, and small talk.
Transcript:
<paste cleaned transcript>“If you cannot quote it verbatim, drop the claim” is a cheap and surprisingly effective hallucination brake. It forces every output line to have a physical anchor in the input.
The interviewer-framing rule catches the most common misquote in published interview writeups, where a leading question gets reported as the subject’s own position.
Chunking a long transcript without losing the thread
Context windows now hold multi-hour transcripts. Attention across them is not uniform.
Liu et al., in “Lost in the Middle: How Language Models Use Long Contexts” (Transactions of the Association for Computational Linguistics, 2024), found that performance is highest when the relevant information sits near the beginning or end of the input and degrades significantly when the model has to reach into the middle — including in models built for long contexts.
For a transcript, “the middle” is the second half of the meeting. Which is usually where the decisions are.
So past about an hour of speech, chunk it. But do not summarise each chunk and then summarise the summaries — that is multi-document summarisation, and it is measurably worse.
Belem et al. found in “From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization” that up to 75% of content in the generated summaries was hallucinated. Asked to summarise sources about topics that did not exist, GPT-4o still produced a summary 44% of the time.
Extract structured notes per chunk instead, then merge them. The extract keeps timestamps and speakers; a summary of a summary keeps neither.
How to split. Cut at topic boundaries, not at fixed word counts. Look for the phrases people say when they change subject — “okay, moving on”, “next up”, “let’s talk about” — and cut there. Overlap each chunk by about two minutes so nothing gets orphaned mid-sentence.
The map prompt:
This is chunk <n> of <total> from a transcript. It covers <start> to <end>.
Do NOT summarise. Produce a structured extract:
- TOPICS: topics discussed, in order, each with the timestamp it starts.
- ITEMS: every decision, action, claim, or number. One bullet each, with
speaker and timestamp.
- OPEN THREADS: anything started in this chunk and not finished by the end
of it.
- CARRY-IN: anything referenced here that must have been introduced before
this chunk started (pronouns without antecedents, "as we said earlier").
Keep every proper noun, number, and date exactly as spoken. Do not
normalise, convert, or round anything.Then the reduce step: paste all the chunk extracts together and run whichever of the three patterns above fits your use case. The extracts are already structured and timestamped, so the final prompt is doing organisation rather than compression — a much easier job, on much less text.
CARRY-IN and OPEN THREADS are what stitch the chunks back together. They tell the reduce step which items in chunk four are the resolution of something raised in chunk two.
Audit the summary before you send it
Summarisation is the task where hallucination was first studied, and it is still not solved. Vectara’s hallucination leaderboard, which by its February 2026 update covered more than 80 models across 7,700+ documents, found the best-performing major model still hallucinated on 3.3% of constrained summarisation tasks — where the model is given a source and told to stick to it.
3.3% on a page of meeting notes is roughly one wrong line. You do not get to know which one without checking.
Run this as a second pass, in a fresh context:
Here is a summary and the transcript it was generated from.
For every factual line in the summary, output:
- The line, verbatim.
- VERDICT: SUPPORTED | UNSUPPORTED | CONTRADICTED
- SOURCE: the verbatim transcript sentence that supports it, with its
timestamp. If there is none, write "no source found".
Do not correct or rewrite the summary. Only audit it.
Check these with particular care and flag any mismatch:
- numbers and quantities
- dates and deadlines
- names, and who said what
- negations (did someone say they WOULD or WOULDN'T)
Summary:
<paste summary>
Transcript:
<paste cleaned transcript>Then check four things yourself, because these are where an automated audit is least reliable and the damage is worst:
- Every number. Budgets, dates, headcounts, percentages. Open the transcript and read them.
- Every negation. “We won’t ship Friday” and “we will ship Friday” are one token apart and opposite in meaning.
- Every owner. Confirm the named person actually accepted the action out loud, rather than being volunteered by someone else.
- Three random timestamps. Jump to them in the video. If all three land in the right place, the rest of them probably do too. If one is off by minutes, your timestamps are decorative and you should not ship the summary with them.
That is maybe two minutes of work, and it is the difference between notes people trust and notes people quietly re-verify every time.
Common failures and what fixes them
| Symptom | Actual cause | Fix |
|---|---|---|
| Nobody is named in the notes | Diarization off, or labels never renamed | Enable diarization; find-and-replace real names before summarising |
| Decisions that were never decided | No definition of “decision” in the prompt | Add the commitment test and an OPEN QUESTIONS section to catch the rest |
| A product name spelled three ways | ASR guessing at jargon | Build a glossary, correct it in the cleanup pass |
| Contradictory dates in the notes | Self-corrections read as two statements | Cleanup prompt removes immediate self-repetitions; add the reversal clause |
| Timestamps missing or wrong | Asked as advice, not as a template slot | Make the timestamp a required field in the output shape |
| Second half of a long meeting under-covered | Middle-of-context attention drop | Chunk at topic boundaries, extract, then merge |
| Confident sentence nobody said | ASR or summariser hallucination | Run the audit pass; verify numbers, negations, and owners by hand |
Where this ends and re-voicing begins
Everything above turns one recording into text you can act on. It does not change the video.
If the next step is putting that footage in front of a different audience rather than a different reader, the transcript you just cleaned is also the input a dubbing pipeline needs. Vidwip’s AI Video Dubbing transcribes inside that workflow, then translates and re-voices the result into up to 94 languages, keeping each speaker’s own voice.
It is not a standalone transcription or summarisation tool, and it will not write your meeting notes. Video dubbing runs 240 credits per minute of source media per target language on v1, which is $2.40, and it is free to start — no card, and no watermark on any tier.
For summarising, use a general model with the prompts above, or one of the free AI summary generators if you want a one-click path. For getting the transcript itself, the free AI video transcription tools comparison names the real ceilings on each free tier.
The short version
The quality of an AI video transcription and summary is set before the summariser ever runs.
Label the speakers. Correct the jargon. Strip the filler without letting anything paraphrase. Keep the timestamps.
Then stop asking for “a summary” and start naming the fields you want out — decisions with owners, steps with timestamps, claims with quotes. Audit what comes back, check the numbers and the negations by hand, and ship it.
Ten minutes of workflow on an hour of footage. It is the cheapest quality upgrade available in this stack.