The conversion benchmark

The full benchmark is incomplete for this release.

Completed lenses are shown below with their own identities. Older lenses remain reference values and are not presented as measurements of the current release.

Recorded benchmark lenses

Each lens retains its own version and revision. Values from different releases are shown for context but are not combined into one result.

Native CLI benchmark

Earlier reference native lens: zenfmt 0.3.3 at 45d9001. These values describe one modest reference machine.

Native CLI

Geometric means over the 14 files converted by zenfmt and AnyDoc.

Speed ratio6.9×AnyDoc wall time / zenfmt

CPU ratio7.9×AnyDoc CPU time / zenfmt

Peak memory ratio10.1×AnyDoc peak RSS / zenfmt

Docling uses parser-only backends. OCR, VLM, ASR, layout models, table models, enrichment, and accelerators are disabled. Unsupported files remain visible rather than switching to an AI pipeline.

Tool	Converted	Corpus
zenfmt	16	16
Docling parser only	5	16
AnyDoc	14	16
Pandoc	6	16

Each ratio divides the comparison tool by zenfmt over shared successful files. Speed is elapsed wall time, CPU is user plus system processor time, and peak memory is resident set size. Above 1.0 means the comparison used more of that measure in this run. These ratios are context, not quality scores.

Comparison tool / zenfmt	Shared files	Speed	CPU use	Peak memory
Docling parser only / zenfmt	5	190.4×	205.5×	47.5×
AnyDoc / zenfmt	14	6.9×	7.9×	10.1×
Pandoc / zenfmt	6	18.2×	16.5×	16.6×

Long-running server benchmark

Earlier reference server lens: zenfmt 0.3.3 at 45d9001. These values describe one modest reference machine.

Long-running server, measured separately

Warm HTTP conversion against Apache Tika Server on the same host and corpus. These values are not mixed into the native CLI ratios.

Warm latency ratio31.1×Tika / zenfmt over 16 shared files

Peak memory ratio34.2×Tika / zenfmt sampled RSS

Throughput at 1 client552.7 / 4.4zenfmt / Tika documents per second

Startup0.11 / 6.97 szenfmt / Tika Server

Peak RSS53.0 / 1814.9 MiBparent and direct parser children

Corpus16 filesthe same verified manifest

File	zenfmt median ms	Tika median ms	zenfmt p95 ms	Tika p95 ms
article.pdf	0.76	20.54	0.85	120.96
book.epub	38.04	861.21	38.52	902.74
data.csv	50.73	118.06	51.07	493.77
deck.ppt	5.57	1744.86	5.93	1968.61
grid.xlsb	0.48	12.63	0.52	25.24
letter.odt	0.43	6.91	0.44	9.14
memo.doc	0.38	106.08	0.43	153.63
notes.rtf	0.62	8.14	0.65	16.95
page.html	33.18	90.29	33.54	163.32
report.docx	1.79	231.26	1.84	233.44
sheet.ods	0.74	12.39	0.75	13.86
sheet.xlsx	1.29	13.47	1.30	15.58
slides.odp	13.64	2919.60	13.83	2927.00
slides.pptx	16.52	3045.48	16.90	3186.91
spec.pdf	0.33	9.42	0.34	31.63
table.xls	0.48	7.58	0.50	10.59

Server throughput

Concurrent clients	zenfmt documents/s	Tika documents/s
1	552.7	4.4
2	1074.5	8.3
4	2015.6	14.3
8	2654.7	14.5

What is measured, and separately

Three questions are kept apart because they have different answers. Coverage is which files each tool claims and actually converts. Performance is cold startup and warm conversion latency, CPU time, and peak memory. Output preservation is checked per format against a tool-neutral fixture oracle, and is never collapsed into a single quality score.

Correctness before timing

The native and browser lenses admit timing only after their output checks pass. Their head-to-head comparisons use only shared successful files. The server lens has a smaller gate: a successful response with a nonempty body. It does not yet prove semantic quality or direct-to-server byte parity, so its timings should be read with that limitation.

Three lenses, never blended

Native process startup, warm library calls, WebAssembly download and compile, warm WebAssembly execution, and a long-running HTTP service answer different questions. The native, browser, and server lenses are therefore reported separately, with cold and warm measurements clearly labelled. A file a tool does not support is shown as unsupported, never as infinitely slow.

Reproducing it

The corpus is not redistributed here: it is assembled from third-party documents whose licences differ, and a conversion of one is a derivative work. What is published is each file's identity, format, size, SHA-256, and source, which is what a reader needs to fetch the same bytes. benchmarks/fetch_corpus.sh verifies every digest and refuses to proceed on a mismatch.