PDF vs Markdown for AI: Token, Speed, and Storage Benchmarks
A file-level benchmark comparing PDF and Markdown on storage, prep speed, and token estimates, including an image-bearing Grimm fairy tales PDF.
If you are building your own personal context layer, the working question is simple: should text-heavy source material stay as PDF, or should you convert it to Markdown and store it that way?
This experiment tests the practical claim.
- Markdown should be better for repeated AI use when the document is mostly text.
- The gains should show up in storage, prep speed, and sometimes token usage.
- The original PDF should still be kept as source when layout or images matter.
I compared two books in both formats.
- Mark Twain, *The £1,000,000 Bank-Note*: 19-page text PDF, no embedded images
- Jacob and Wilhelm Grimm, *Grimm's Fairy Tales: Complete and Illustrated*: 768-page PDF with 254 embedded images
For each pair, I measured on-disk size, local prep time, and local text-only token counts. If you want the conversion step itself, use the PDF to Markdown converter.
What We See
- In both examples, Markdown is smaller and faster to prepare.
- The difference is visible but modest for the Twain text PDF.
- The difference becomes very large for Grimm because the PDF carries many embedded images.
- The chart is enough to support the storage and speed claim on its own: Markdown is the better working format for text-first reuse.
Results
| Document | Storage result | Prep result | Text token result | What it means |
|---|---|---|---|---|
| Mark Twain, *The £1,000,000 Bank-Note* | 65,800 bytes as PDF vs 44,882 bytes as Markdown | 17.54 ms to extract PDF text vs 0.025 ms to read Markdown | 11,132 extracted-PDF tokens vs 10,617 Markdown tokens; normalized counts were effectively identical | Markdown saves space and time; token savings are small once the text is cleaned |
| Jacob and Wilhelm Grimm, *Grimm's Fairy Tales: Complete and Illustrated* | 6,151,274 bytes as PDF vs 1,496,080 bytes as Markdown | 679.57 ms to extract PDF text vs 0.263 ms to read Markdown | 367,858 extracted-PDF tokens vs 367,286 Markdown tokens; normalized counts stayed very close | Markdown saves much more space and time; embedded images inflate the PDF far more than they change extracted text counts |
The key pattern is that token savings are not the whole story. In both files, the big win is operational: smaller files, faster loading, easier search, easier chunking, easier reuse.
What This Proves
If the document is mostly text, converting it to Markdown is usually in your interest.
- you save storage
- you reduce repeated PDF extraction overhead
- you make the file easier for an agent to search, diff, and rewrite
- you keep a cleaner long-term working artifact inside your context layer
The Grimm case is the important one. Its text token count stays close to Markdown after extraction, but the PDF is still much worse as a working file because it is larger, slower, and image-heavy.
Important Caveat
These token counts are local text-only estimates. They compare extracted PDF text against Markdown text.
Direct PDF uploads to multimodal APIs can cost more than this benchmark suggests. OpenAI's PDF guide says PDF inputs are processed from extracted text plus an image of each page. Anthropic documents PDF support in the same general direction. That means a PDF can be more expensive in live API usage even when local extracted-text token counts look similar.
How AI Token Usage Works
For practical budgeting, there are four buckets to keep in mind.
- input tokens: everything the model reads, including instructions, prompts, history, retrieved context, tools, and file content
- output tokens: the generated answer
- cached input tokens: reused prompt prefixes that some providers bill differently
- thinking or reasoning tokens: internal reasoning work on models that expose that category
The implication is straightforward: attachment cost is only one part of the total. Long history, retrieved context, cached prefixes, and reasoning can all matter as well.
Practical Rule
For a context layer, the default workflow should be:
- Keep the original PDF.
- Convert text-first documents with the PDF to Markdown converter.
- Store the Markdown file as the searchable, reusable working copy.
- Return to the PDF only when page layout, figures, citations, or archival fidelity matter.
Use Markdown when the document is mostly text and you expect to search it, chunk it, rewrite it, or hand it back to agents repeatedly.
Keep the PDF as the source artifact when images, diagrams, page references, or the original layout are part of the task.
Sources
Keep Up
Get the next guide by email
Practical notes, frameworks, and new tools from applied AI work. Occasional only.
Related entries
A Practical Guide to Entity Resolution
A general guide to matching messy records into governed entities, with blocking, scoring, selective review, clustering, survivorship, and evidence-first reference tables.
A Context Layer for Teams
Most teams already have the raw material for a context layer in Drive, Slack, and scattered transcripts; the hard part is turning that sprawl into a shared record that people and agents can use without rebuilding the history each time.
What TurboQuant Is and Why It Matters
Google’s new TurboQuant method compresses key-value cache and search vectors more aggressively by cutting quantization overhead, which could make long-context inference and retrieval cheaper and faster.