Lyonite Tested · a measurement series
What PDF tools actually do with your file
Online file tools ask you to take a lot on trust. That your document is deleted after an hour. That it never left your device. That the metadata is gone. Almost none of it is checkable from the outside, and almost nobody checks.
So we check. Merging, splitting, compressing, converting, cleaning — each report takes one claim, puts every tool through the same file, and publishes that file so you can get your own number. Our tools are in the tables too. The first report is the one that caught a bug in ours.
Reports
Each one states when it was measured, because these are live services that change.
Which PDF metadata tools keep current fields, saved versions and raw evidence in context?
Ten workflows compared, eight completed results, and a public fixture that exposes the difference between current metadata and old strings.
Read the report →Do PDF metadata removers actually remove what identifies you?
Six of seven handed the file back with the GPS coordinates still in it. None removed the embedded script.
Read the report →Does cleaning a PDF remove the versions you already saved over?
On our five-state fixture, exiftool -all= added a sixth state and retained all four checked historical strings.
Read the report →Which compressor gets a PDF smallest, and what does it cost you?
iLovePDF, Smallpdf and Ghostscript each deleted every accessibility tag in every tagged file — 124,009 of them. PDF24 and we did not. Ghostscript also returned a blank page carrying its own figure caption and made three of five files larger.
Read the report →How these are done
Four rules, and why each one is there.
Pick a claim with an answer
Not “is this any good”. Did the file leave your device? Did the text survive? How much bigger did it get, and how long did it take? Every report starts from a question a stopwatch, a network panel or a byte count can settle.
Give every tool the same file
One input, built for the job and published with the report. Comparing tools on files they each chose is how you get a table where everybody wins.
Read what the tool claims first
A tool that says it clears the author field and clears the author field is working, however narrow that is. Judging it against a standard it never set makes a report easy to dismiss, and rightly so.
Include ourselves, and date the losses
We build some of the tools we test. Ours go in the same table as everyone else’s, and where we lost, the report says so with a date.
Why checking your work does not help
ExifTool on the PDF reads the document’s own metadata. It does not descend into embedded image streams, so a photo full of GPS tags produces no output at all. The tool is not wrong — it is answering a narrower question than the one you meant to ask.
qpdf --qdf expands the file so you can read its structure, and it is the natural second thing to reach for. But qpdf garbage-collects unreferenced objects while it rewrites, so an object that was unlinked but never deleted disappears from the expanded output while remaining in the real file. The check removes the evidence it was supposed to show you.
Compressed object streams close the last gap. A surviving script or reviewer name packed into one is invisible to a plain text search of the bytes, so the third obvious approach — grep the file — also comes back empty.
Between the three, a leak can be invisible to every check short of writing a parser and walking the object graph yourself. That is the actual reason these tools ship broken for years: nobody who checks casually ever sees it.
What this does not tell you
This is a test of metadata removal, not of anonymity. A PDF can identify its author through things no metadata tool touches: the writing itself, a signature image, a letterhead, the specific subset of fonts a particular installation embeds, or text sitting under a black rectangle that was drawn rather than applied as a redaction.
If your safety depends on a document not being traced to you, cleaning its metadata is one step in a longer process, and the last step should not be an automated one.
The corpus lives in its own repository.
What we measure
Four questions, asked of every tool in the category — merging and compressing as much as cleaning. Each report takes one and answers it for one kind of tool.
Did the document leave the device at all?
Watch the network while the tool runs. A tool that says “processed in your browser” either makes no request carrying the file, or it does. There is no middle answer.
Did it do the specific thing it says it does?
Build an input where the answer is checkable — planted values for a cleaner, a known outline and form fields for a merger, extractable text for a compressor — then read the output back and count what survived.
What did you quietly lose?
Compare in to out. Bookmarks, internal links, form fields, selectable text, accessibility tagging. The failure that matters here is the silent one: the file opens fine and something inside it is gone.
Did it take longer or come back bigger than it needed to?
Time the same file through each tool, upload and queue included, and measure the output. A “compressor” that returns a larger file, or a merger that doubles the page count in bytes, is a real and common result.
In progress
What is being measured next. If you want one of these sooner, or have a claim worth checking, say so.
These sites say your upload is deleted after an hour. Is it?
Which tools that claim local processing actually do it?
Do bookmarks, links, form fields and tagging survive being merged?
Who publishes this
The questions you should be asking of any comparison, answered about this one.
- Who runs it
- One person, who also builds the Lyonite tools being measured. That is the conflict of interest, so: they go in the same tables as everyone else’s, and where they lost, the report says so with a date.
- What it runs on
- Files built so the answer is checkable, and a script that fails loudly when it is not. Public, one dependency, no account. Clone it and run it.
- How often
- Every published report is re-run and re-dated. A tool that fixes what we found gets a correction with its own date — that update is the story we most want to publish.
- What we get out of it
- You reading this. We sell nothing, there is no account, and every tool is free with no daily quota. The tables are also how we find our own bugs, which is how the first one started.
Why we run these
Lyonite exists because of the gap these reports keep finding: the thorough answer to "do this to my document properly" has been a paid desktop subscription, and the free alternatives tend to do the visible half and describe it as the whole thing.
We have not tested the paid tools, so we will not tell you how we compare to them — on this series' own rules an untested claim is worth nothing. What we can tell you is what we measured: on the seven things the first report checks, we now remove all of them, in a browser tab, for free, without your file leaving the device. And you should not have to take our word for it — the code that proves nothing is uploaded is in the repository, and the tests we grade ourselves against are public and include the runs we failed.
If you check and we come up short, we would rather hear it. Write to hello@lyonite.com and you will get an answer within 48 hours. Findings get added, dated and credited.
Check this yourself. If I got it wrong, tell me.
Every number here came from files you can download and a checker you can run, so you do not have to take my word for any of it — clone the corpus and get your own result.
If it disagrees with mine, or if you build one of the tools named here and I measured it unfairly, out of date, or with a setting you would not have used, send it to hello@lyonite.com.
I reply within 48 hours. If you are right, the page is corrected with the date on it and your correction credited, and the old number stays visible so the change is legible. If a tool has since been fixed, that is the update I most want to publish. Nothing here is worth defending past the point it stops being true.