How YouTube captions actually work
Where timed text lives, what a cue really contains, how the official Captions API differs from the public player track, and why cleanup is the whole job.
- 01A caption is not part of the picture. It is a separate list of cues, and a cue is a start time, an end time and a line of text.
- 02YouTube gets that list one of two ways: speech recognition generates it, or the creator uploads a file. The two behave very differently.
- 03The official YouTube Captions API is an owner tool. Downloading or changing a track through it needs OAuth as the channel that owns the video.
A caption on YouTube is not burned into the picture. It is a separate file of timed text that the player draws over the video at playback time. You can switch it off, restyle it, swap it for another language, or read it with something that is not a video player at all. Once that lands, most of the confusing behaviour around YouTube captions stops being confusing.
This post explains how YouTube captions work at the level an engineer needs. What a caption track is. What a cue is. Where the two kinds of track come from. What the documented YouTube Captions API offers to the person who owns the video. And what a text export has to do to raw timed text before a human can comfortably read it. It is a conceptual post on purpose. There are no request recipes here and there will not be any.
The short version, if the short version is all you want: a caption track is a list of cues, a cue is a start time plus an end time plus a text payload, YouTube either generates that list with automatic speech recognition or accepts one the creator uploaded, and every tool that reads captions is reading a list that already exists rather than listening to the audio.
Disclosure
YouTubeScribe is our product. The parts of this post about cleanup, export behaviour and failure handling describe what our own engine does, so read those as a maintainer's notes and not as a neutral survey. The parts about YouTube itself come from Google's own documentation and the W3C timed text specs, all listed at the end.
What a caption track actually is
Timed text is the general name for the category. It covers subtitles, captions, karaoke lines, chapter labels and anything else that has to appear on screen at a specific moment and then go away. YouTube's captions are timed text, and so are the subtitle files you download from a broadcast pipeline, and so is the track element in an HTML5 video page. The vocabulary is shared because the problem is shared: you have a spoken line, and you have a window of time during which it should be visible.
A caption track is a container for a list of cues, plus a small amount of metadata about the track itself. The metadata is what the language menu in the player is built from: a language code, a human readable name, whether the track came from automatic speech recognition or from a person, and, in the API's view of the world, whether the track is still a draft.
What is a caption cue
A cue is the atomic unit. It has a start time, an end time (or a duration, which is the same thing expressed differently), and a text payload. That is the whole idea. Everything else that a format adds is optional decoration around those three things.
The W3C WebVTT spec is the clearest place to see the shape. A WebVTT cue may carry an optional identifier on its own line, then a timing line with the start and end timestamps, then optional cue settings that tell the player where on screen to draw it, then the payload text. The payload can contain simple markup and voice spans, which are the mechanism for marking which speaker is talking. TTML, the other W3C timed text standard, models the same data with XML and a much larger styling and layout vocabulary.
Two consequences fall out of this model immediately, and they explain a lot of reader confusion.
- A cue boundary is not a sentence boundary. A cue ends when the player should stop showing that text, which is a display decision, not a grammar decision. One sentence routinely spans three cues, and one cue routinely holds the tail of one sentence and the head of the next.
- Cues can overlap. Two cues whose time ranges intersect are legal and common, because a player often wants the previous line to linger while the next one arrives. Read the file as a document rather than as a timeline and it looks like duplicated text.
- The payload is text, not audio. There is no signal in a caption file about volume, tone, or who is louder. If a speaker label is not written into the payload, no amount of processing will recover it.
- Timestamps are relative to the start of the video, not to wall clock time. This is why cues survive re-encoding and why a track from 2013 still lines up with the same video today.
There is one more distinction worth keeping straight, because the two words get used interchangeably and they do not mean the same thing. Subtitles assume you can hear the audio and just need the words, often in another language. Captions assume you cannot hear the audio and therefore include sound events, speaker changes and other non-speech information. YouTube's interface mostly says captions and mostly delivers subtitles, because an automatic track has no reliable way to describe a door slam.
Naming
In the YouTube Data API the object is called a caption resource and the thing it points at is called a caption track. In the WebVTT spec the unit is called a cue. In everyday use people say subtitle file, caption file and transcript for all three. This post uses track for the list, cue for the unit, and transcript for the flattened human readable output, because those three never collide.
Where the two kinds of track come from
Every caption track on YouTube arrives one of two ways. Either a speech recognition system listened to the audio and produced a track, or a human uploaded a file. The difference between those two origins accounts for almost every quality difference you will ever notice in a downloaded transcript.
Automatic speech recognition captions
When a video is uploaded, YouTube may run automatic speech recognition over the audio and attach the result as an automatic caption track. Coverage is roughly 70 languages, which is a lot but is nowhere near every language on the platform. Do not believe the figure you sometimes see quoted at 125 or more: that number is about caption and subtitle language support in general, including translation and creator uploads, and not about the automatic recogniser.
Automatic captions are also not guaranteed even inside a supported language. Google publishes six specific reasons a video may not have them:
- The automatic captions are still processing. Complex audio takes longer, and very long or unusual videos can sit in the queue for a while.
- The language of the video is not one the automatic captioning system supports.
- The video is too long.
- The sound quality is poor, or the speech in it is not recognised by the system.
- There is a long stretch of silence at the beginning of the video.
- Multiple people are speaking over each other, or multiple languages are spoken at once.
Read that list again if you maintain a caption tool, because it is the honest answer to most support tickets. A video with no captions is usually not a bug in your code. It is a video that hit one of those six conditions, and no request you can construct will change that. The track was never made.
Creator uploaded tracks
The other path is a person writing or commissioning a caption file and uploading it. YouTube accepts a wide range of formats here, wider than most people expect, because the platform has to absorb output from broadcast pipelines, subtitling houses and free desktop tools alike.
| Format | Extension | Positioning | Styling | Typical origin |
|---|---|---|---|---|
| SubRip | .srt | No | None. Plain UTF-8 text. | The default hand-off format almost everywhere |
| WebVTT | .vtt | Yes | Limited to bold, italic and underline | Web players and modern tooling |
| SAMI | .smi, .sami | No | Simple markup | Older Windows media workflows |
| RealText | .rt | No | Simple markup | Legacy archives |
| TTML and DFXP | .ttml, .dfxp, .xml | Yes | Yes, full styling | Standards driven and broadcast pipelines |
| Scenarist Closed Caption | .scc | Yes, under CEA-608 rules | Yes, under CEA-608 rules | Broadcast. This is YouTube's preferred broadcast format. |
| SBV, MPsub, LRC, Videotron Lambda, EBU-STL | various | Varies | Varies | The long tail YouTube still accepts |
The practical point of that table is not that you should care about RealText in 2026. It is that a creator uploaded track can carry positioning and styling information that an automatic track never has, and that the format the creator chose sets a ceiling on what survives. An .srt upload cannot carry italics because SubRip has nowhere to put them. A .scc upload carries broadcast styling that most web players will quietly flatten.
Here is how the two origins compare on the things that actually affect a transcript you are going to read.
| Property | Automatic track | Creator uploaded track |
|---|---|---|
| Speaker labels | None. Two people in conversation produce one unbroken stream of words. | Whatever the creator typed. Often names, dashes or voice spans. |
| Punctuation | Added by the model. Thin, inconsistent, and sentence ends are frequently missing. | The author's own. Sometimes better punctuated than the audio deserves. |
| Cue length | Short. Often one to three seconds, with boundaries set by the recogniser rather than by grammar. | Usually one or two lines held for a readable duration. |
| Typical errors | Homophones, proper nouns, product names, technical jargon, and rolling repeated lines. | Typos, drift from the spoken line, and the occasional missing section near the end. |
| Sound events | Occasional music or applause markers, applied inconsistently. | Often complete, if the creator captioned for accessibility rather than for reach. |
| Availability | Roughly 70 languages, and only when the audio and the video clear the six conditions above. | Any language the creator chose, in any accepted format. |
One nuance that trips people up: a video can have both. A creator can upload an English track while the automatic English track also exists, and the language menu will show one entry that reads like a normal language name. Which one you get depends on which the player selects, and a tool that reads the same track the player would read inherits that choice. This is the most common reason two people download captions for the same video and get visibly different text.
What the official Captions API gives an owner
The YouTube Data API has a documented Captions resource. If you own the channel, this is the supported way to work with caption tracks programmatically, and it is the only caption interface Google actually promises to keep working. It supports five operations.
| Operation | What it does | What it needs |
|---|---|---|
| list | Returns the caption tracks that exist for a video, along with metadata such as the language, the track name, the track kind and whether the track is a draft. | OAuth authorisation. The useful case is a channel listing the tracks on its own videos. |
| insert | Uploads a new caption track and attaches it to a video. | OAuth as the owning channel, the video identifier, and a caption file in one of the accepted formats. |
| update | Replaces the file or the metadata on a caption track that already exists. | OAuth as the owning channel and the caption track identifier. |
| download | Returns the content of a caption track, optionally converted into another caption format. | OAuth as the owning channel and the caption track identifier. |
| delete | Removes a caption track from the video. | OAuth as the owning channel and the caption track identifier. |
Notice what is missing from that table. There is no public read. Downloading or modifying a caption track through this API requires OAuth authorisation as the channel that owns the video. It is an owner tool. A great many blog posts describe the Captions resource as though it were a general purpose way to fetch anyone's captions, and it is not, and building on that misreading will waste a weekend.
What that is good for
If you run a channel, the Captions API is genuinely useful and underused. A publishing pipeline can insert a proofread track the moment a video goes live, update it when someone reports an error, and delete stale drafts on a schedule. A localisation workflow can push one track per language without anyone opening the web interface. An archive job can download every track a channel owns and store it next to the video metadata, using the format conversion on download to normalise everything into one shape.
It also has costs, in the literal sense. Data API calls draw against a quota, and caption uploads are not the cheapest calls in the catalogue. Plan a bulk backfill against your quota before you plan it against your calendar.
How tools read a public track
This is the question everyone actually arrives with, so here is the honest answer at the only level of detail that is appropriate to publish. The player is served timed text alongside the video. A tool that reads captions for a public or unlisted video reads the track the player would read, then formats it. That is the whole description.
We do not publish request shapes, parameter names or anything else that would function as a recipe, and we would rather other people did not either. Interfaces that are not documented are not promised. They change without notice, they break at the worst possible moment, and treating them as a public contract is how tools end up broken for a month while their maintainer works out what moved. If you need a caption interface with a stability guarantee, the owner API above is the one that has one.
A caption track is data the video already carries. If it is not there, no clever request will conjure it, and anything claiming otherwise is running speech recognition and calling it extraction.
YouTubeScribe. Engine notes, 2021 to 2026.
What automatic captions get wrong, and why
If you have only ever seen captions in the player, they look fine. The player shows you one or two lines at a time, replaces them quickly, and your eye never has time to audit them. Flatten the same track into a document and the problems become loud.
The rolling repeat
This is the big one, and it is the single reason raw automatic captions read so badly on the page. An automatic recogniser does not decide on a line and then move on. It emits a partial line, then re-emits the same line with one more word appended, then again with another word, so the player can grow the sentence in place as the speaker talks. On screen that looks like natural typing. In a file it looks like this: the same phrase four times in a row, each copy one word longer than the last.
A three minute video can produce hundreds of these. Nothing is wrong with the data. It is doing exactly what the player needs. It is simply not what a reader needs, and any tool that hands you a raw automatic track without collapsing this pattern has skipped the only step that mattered.
Missing structure
Automatic tracks carry no speaker information. An interview between two people becomes one continuous paragraph in which the questions and the answers are indistinguishable except by content. There is no marker to recover, because the recogniser never wrote one. Any tool that claims to add speaker labels to an automatic YouTube track is running its own diarisation over something, which is a different product with different failure modes and different costs.
Punctuation is thin. Modern recognisers insert commas and full stops, and they are much better at it than they were five years ago, but sentence ends are still frequently missing and question marks are rarer than questions. Capitalisation follows the punctuation, so a missing full stop usually drags a missing capital along behind it.
Errors that look like confidence
Speech recognition does not hedge. It returns a word, not a probability you can see, so a wrong word looks exactly as certain as a right one. The predictable failure categories are worth knowing because they tell you where to check:
- Proper nouns. Personal names, company names and place names are the worst performing category by a wide margin, and they are usually the words a reader most wants correct.
- Domain jargon. A technical term the model has not seen often becomes the nearest common word, which is frequently a real word and therefore invisible to a spellcheck.
- Homophones. There, their and they're. To, too and two. Context usually saves these and sometimes does not.
- Numbers and units. Digits, spelled numbers and units drift between forms within the same video.
- Crosstalk. Two people speaking at once produces one interleaved stream that reads as nonsense, which is also one of the six official reasons a track may not be generated at all.
The cue length trap
Automatic cues are short, often one to three seconds. That makes for smooth playback and terrible reading. If you are building anything on top of a caption track, do not assume one cue equals one sentence, and do not assume a cue boundary is a safe place to split. Merge on sentence structure, not on timing, whenever the output is meant for a human eye.
What a clean export has to do with all this
Everything above is background for one practical question: what should a tool do between reading a caption track and handing you a file? Here is our answer, stated plainly enough that you can hold us to it.
YouTubeScribe reads the caption track a public or unlisted video already has. It never runs speech recognition on audio and it never invents speech. If the track does not exist, we say so. That constraint is the entire design, and it is why the work is fast and why it is honest.
What we clean
- Rolling repeats. Cues that repeat the previous line while appending one new word are collapsed to the final complete line. Automatic speech recognition does this constantly, and it is the highest value fix in the whole pipeline.
- Empty cues. Cues that contain nothing, or only a newline, or only a music note with no other text, are dropped.
- Markup that is not part of the spoken line. Formatting and positioning instructions belong to the player. They do not belong in a document you are going to read or paste.
- Split sentences. Overlapping cues that are one sentence broken up for the player are merged back together when the export is plain text.
What we leave alone
This list matters more than the previous one, because a cleanup step that gets ambitious stops being cleanup and starts being editing.
- Names are not corrected. If the recogniser heard the wrong name, the wrong name is what you get, because guessing at the right one is how a transcript quietly becomes fiction.
- Nothing is translated unless you asked the translator for it. A track in Portuguese comes back in Portuguese.
- Profanity is not removed and not masked. It was said, so it is in the transcript.
- A creator's own punctuation is not restyled. If they typed it, it stays. Their capitalisation, their dashes, their line breaks in an uploaded track are theirs.
Timing survives where timing matters
The cleanup is not uniform across formats, because the formats are for different readers. On SRT and VTT exports the cue boundaries stay close to what YouTube sent, so a player still hits the same second when you load the file next to the video. Aggressive merging there would produce a file that reads beautifully and syncs badly, which is the wrong trade for a subtitle file.
Plain text is the opposite case. Nobody is syncing a TXT export to anything, so it can merge more freely and produce paragraphs a person can actually read. Two exports of the same video, two different sets of choices, both defensible.
The Transcript Generator at /tools/transcript-generator is the tool for the reading case, and it is the one most people on this page want. The Subtitle Downloader is the tool for the timing case, when you need a file that lines up with the video rather than a document.
How it fails, stated honestly
A tool is defined more by its failures than by its successes, so here are ours. Four things go wrong, and each of them is reported as a clear miss or as a language you can pick again, never as a half finished file dressed up as a complete one.
- There is no track at all. This is the most common case by a distance, and it is usually one of the six official reasons above. There is nothing to extract, so nothing is returned.
- A track is listed and then fails to fetch. The metadata says it exists, the content does not arrive. We report the miss rather than returning an empty document.
- A language code that does not match the menu. Old and hand-edited tracks sometimes carry a code that disagrees with what the player displays. You get the language list back so you can choose again.
- Very old uploads with odd data in a caption slot. Videos from the earliest years of the platform occasionally carry something in a caption slot that is not usable timed text. That is a miss, not a mangled transcript.
What is stored
On the free tier nothing is stored. You run a video, you get text, and there is no record of it afterwards. On Pro, history stores the cleaned text you actually saw, and you can delete any of it. The raw upstream response is not offered as a public export, and that is a deliberate choice rather than an oversight: what we can stand behind is the cleaned output, so that is what we keep and what we hand back.
The one thing to take away
Reading captions and generating captions are different problems with different economics. Reading is fast, cheap and limited by whether the track exists. Generating is slow, costly and limited by audio quality. Any tool that is instant and free is doing the first one. If a tool is instant, free, and returns text for a video with no caption track, something in that sentence is not true.
Questions people ask
What is a caption cue?
A cue is the smallest unit of a caption track: a start time, an end time, and a line of text. Everything else a caption format offers, such as identifiers, positioning settings and simple markup, is optional decoration around those three fields. A track is just an ordered list of cues. Because a cue boundary is a display decision rather than a grammar decision, one sentence often spans several cues, and one cue often holds the end of one sentence and the start of the next.
What is the difference between subtitles and captions?
Subtitles assume you can hear the audio and only need the words, often in another language. Captions assume you cannot hear the audio, so they also describe sound events, speaker changes and other non-speech information. YouTube's interface mostly says captions but usually delivers something closer to subtitles, because an automatic speech recognition track has no reliable way to describe a door closing or to tell two speakers apart.
Are YouTube captions part of the video file?
No. Captions are separate timed text served alongside the video and drawn over the picture by the player at playback time. That is why you can switch them off, restyle them, pick a different language, or read them without watching anything. Burned in captions, sometimes called open captions, are a different thing entirely: those are pixels in the picture and cannot be turned off or read as text by any tool.
How many languages have automatic captions?
Automatic captions cover roughly 70 languages. That is smaller than the total number of languages in which captions and subtitles can exist on YouTube, because creators can upload a track in any language they want and translation covers more ground still. If you see a figure like 125 quoted for automatic captions, it is measuring something broader than the speech recogniser.
How does YouTube decide which caption track to show?
The player picks a track based on the video's available tracks, your language settings and your caption preferences. When a video has both a creator uploaded track and an automatic track in the same language, the menu can show one entry that reads like an ordinary language name, and which underlying track you get is the player's choice. This is the usual explanation when two people download captions for the same video and get noticeably different text.
What does the YouTube Captions API let me do?
The YouTube Data API's Captions resource supports five operations: list, insert, update, download and delete. List returns the tracks on a video with their metadata. Insert attaches a new caption file. Update replaces a track's file or metadata. Download returns a track's content, optionally converted to another format. Delete removes a track. Access to download or modify a track requires OAuth authorisation as the channel that owns the video.
Can I use the Captions API to download anyone's captions?
No. It is an owner tool, not a public read interface. Downloading or modifying a caption track through the Captions resource requires OAuth authorisation as the channel that owns the video, so it works for your own uploads and not for someone else's. A lot of tutorials describe it as a general purpose caption fetcher, which is a misreading that will cost you a weekend if you build on it.
Which caption file formats does YouTube accept on upload?
A wide range. SubRip .srt is plain UTF-8 with no styling. WebVTT .vtt supports positioning, with styling limited to bold, italic and underline. SAMI and RealText allow simple markup but no positioning. TTML and DFXP support both styling and positioning. Scenarist .scc is YouTube's preferred broadcast format. SBV, MPsub, LRC, Videotron Lambda, EBU-STL and other CEA-608 broadcast formats are also accepted.
Should I export SRT, VTT or plain text?
Pick by what will read the file. Choose SRT or VTT when a player or an editor needs to sync the text to the video, because those exports keep cue boundaries close to what YouTube sent so playback still hits the same second. Choose TXT when a person is going to read it, because plain text can merge split cues into real paragraphs without worrying about timing. Same source track, two different sets of trade-offs.
How do caption tools read a public video's track?
The player is served timed text alongside the video, and a tool reads the track the player would read, then formats it. That is as specific as this post gets on purpose. Undocumented interfaces are not promised to anyone, they change without notice, and publishing recipes for them is a bad idea for both the publisher and the reader. If you need a caption interface with a stability guarantee, use the owner API.
Does YouTubeScribe transcribe audio?
No. YouTubeScribe reads the caption track a public or unlisted video already has, cleans it, and formats it. It never runs speech recognition on audio and it never invents speech. That single constraint is why the work takes seconds and why a video with no caption track returns a clear miss rather than a plausible looking document. The Transcript Generator at /tools/transcript-generator is the reading focused tool, and the Subtitle Downloader covers the timing focused case.
Why does my raw caption file repeat the same line over and over?
That is the rolling repeat pattern, and it is normal. An automatic recogniser emits a partial line, then re-emits it with one more word appended, then again, so the player can grow the sentence in place. On screen it looks like natural typing. In a file it looks like the same phrase four times, each copy one word longer. Cleanup collapses each run to the final complete line, which is the highest value fix in the whole pipeline.
Why does a video have no captions at all?
Google lists six official reasons automatic captions may be unavailable: the captions are still processing because the audio is complex; the language is not supported by automatic captioning; the video is too long; the sound quality is poor or the speech is not recognised; there is a long silence at the start of the video; or several people are speaking over each other or several languages are spoken at once. In all six cases no track was created, so there is nothing for any tool to read.
A language shows in the menu but the download fails. What happened?
Two different things produce that. Either the track is listed in the metadata but its content does not arrive when requested, or the track carries a language code that disagrees with what the player displays, which happens with old and hand-edited uploads. We report the first as a miss rather than returning an empty file, and for the second we hand back the language list so you can pick again. Neither should ever produce a half finished document.
Will the tool fix wrong names or bad punctuation in the transcript?
No, and that is deliberate. Names are not corrected, because guessing at the right spelling is how a transcript quietly turns into fiction. Profanity is not removed. Nothing is translated unless you asked the translator. A creator's own punctuation in an uploaded track is left exactly as they typed it. Cleanup removes artefacts that belong to the player, such as rolling repeats, empty cues and markup. It does not rewrite what was said.
Sources
- YouTube Help. Use automatic captioning. support.google.com· Checked 17 August 2026.
- YouTube Help. Supported subtitle and caption files. support.google.com· Checked 17 August 2026.
- YouTube Help. Add subtitles and captions. support.google.com· Checked 17 August 2026.
- W3C. WebVTT: The Web Video Text Tracks Format. w3.org· Checked 17 August 2026.
- Google for Developers. YouTube Data API: Captions. developers.google.com· Checked 17 August 2026.
- Google for Developers. YouTube Data API: Captions: list. developers.google.com· Checked 17 August 2026.
- Google for Developers. YouTube Data API: Captions: insert. developers.google.com· Checked 17 August 2026.
- Google for Developers. YouTube Data API: Captions: update. developers.google.com· Checked 17 August 2026.
- Google for Developers. YouTube Data API: Captions: download. developers.google.com· Checked 17 August 2026.
- Google for Developers. YouTube Data API: Captions: delete. developers.google.com· Checked 17 August 2026.
- Google for Developers. YouTube Data API Overview. developers.google.com· Checked 17 August 2026.
- Google for Developers. Implementing OAuth 2.0 Authorization. developers.google.com· Checked 17 August 2026.
- Google for Developers. YouTube Data API Quota Calculator. developers.google.com· Checked 17 August 2026.
- Google for Developers. YouTube API Services Terms of Service. developers.google.com· Checked 17 August 2026.
- W3C. Timed Text Markup Language 2 (TTML2). w3.org· Checked 17 August 2026.
- W3C. Timed Text Markup Language 1 (TTML1). w3.org· Checked 17 August 2026.
- W3C. Media Accessibility User Requirements. w3.org· Checked 17 August 2026.
- MDN Web Docs. WebVTT API. developer.mozilla.org· Checked 17 August 2026.
- MDN Web Docs. The track element. developer.mozilla.org· Checked 17 August 2026.
- YouTube Official Blog. Caption my YouTube videos. blog.youtube· Checked 17 August 2026.
- YouTubeScribe. Engine notes, 2021 to 2026.
Written by Faisal Ashfaq. Faisal Ashfaq maintains the YouTubeScribe caption extractor at Deeporax AI LTD in Oldham. He writes the under-the-hood notes on timed text.
Reviewed on 17 August 2026. If a line is wrong, email support@youtubescribe.com.