Split PDF parsing into cheap and heavy passes

PDF parsing pipelines usually skip the cheapest check there is. Run this against each page before you spend anything on OCR or embeddings:

raw = page.get_text("rawdict")
native_chars = 0
ocr_chars = 0
for block in raw["blocks"]:
    if block["type"] != 0:
        continue
    for line in block["lines"]:
        for span in line["spans"]:
            if span.get("render_mode", 0) == 3:
                ocr_chars += len(span["text"])
            else:
                native_chars += len(span["text"])

Render mode 3 means invisible text, the layer an OCR tool hides under a scanned image so the file becomes searchable. Plenty of pipelines treat a present text layer as a reason to skip OCR, but OCR text can be garbage. The example that sticks comes from Kezhan Shi’s piece on Towards Data Science: “Renewal fee EUR 250” arriving as “Renewa1 fee EUR 25O”. Retrieval never matches, so the model picks a different number and the wrong answer ships. Counting render mode 3 separately costs milliseconds and tells you which pages need re-OCR.

The piece splits parsing into two layers: declared signals (metadata, the native table of contents, the producing software) and page-level content, meaning what each page holds. The signals route the file and the page walk checks whether to believe them. I’ve started applying the same split to store document ingestion.

PDF parsing in two passes

The alternative most projects pick is one engine for everything. Docling, IBM Research’s MIT-licensed toolkit, runs layout and table structure models and returns a structured document. For messy files it’s the right tool. But you pay the layout-model price on the Word export of a spec sheet that would have extracted cleanly with direct text extraction. The cheap pass reads doc.metadata (Creator and Producer), doc.get_toc(), the render mode counts above, and per-page image coverage. The article reports roughly nine of ten PDFs on a mixed corpus land in the right routing bucket from metadata alone. That’s good enough to route on, because the page walk catches the rest.

Two details worth keeping. Metadata lies sometimes: Ghostscript and qpdf overwrite the Producer field when they recompress, so a Word PDF re-distilled twice claims to be Ghostscript, which is why you store the raw strings next to your inferred label. And page.get_images() returns intrinsic pixel dimensions rather than the area drawn on the page, so full-page-image detection (the article uses 95 percent coverage as the scan test) needs page.get_image_info(), or it fires on the wrong pages.

Where this fits in a WordPress upload flow

On a store this belongs at ingest rather than query time. Hook the upload, run the cheap pass, store the flags as post meta (has_native_text, has_full_page_image, n_vector_tables), and queue the expensive work only for pages the flags mark. Action Scheduler already ships the batch loop for that. Parsing decisions become data you can inspect later, which is what you want when a client asks why one supplier’s PDF answers questions and another’s doesn’t.

Column layout is the failure I’d watch for on store documents. Two-column spec sheets interleave lines when extracted naively, and the symptom looks like the model can’t read when the lines were never in the right order. The article annotates each line as single, left, right or multi instead of pretending to fix reading order, and escalates multi pages. That’s the cheaper and more honest default.

Which pass for which store

If uploads are mostly Office exports and browser prints, which is most client corpora I see, run the cheap pass plus direct extraction and keep the heavy engine for flagged pages. If the corpus is mostly scans and design-tool exports, invert it: Docling or your OCR engine becomes the default and the cheap pass works as triage. Either way, record the signals once, because every downstream stage in your RAG pipeline reads them.

If you have a store collecting supplier PDFs and want ingestion that routes files instead of guessing, this is the kind of build I take on. Start from a sample of the real documents.

What I still haven’t settled is where the line sits between an OCR layer good enough to index and one worth re-running. The honest quality signals, replacement characters and dictionary hit rate, are corpus-dependent, and on a store with thousands of supplier documents the wrong call gets paid for twice.

author avatar
Ahmad Wael
I'm a WordPress and WooCommerce developer with 15+ years of experience building custom e-commerce solutions and plugins. I specialize in PHP development, following WordPress coding standards to deliver clean, maintainable code. Currently, I'm exploring AI and e-commerce by building multi-agent systems and SaaS products that integrate technologies like Google Gemini API with WordPress platforms, approaching every project with a commitment to performance, security, and exceptional user experience.

Leave a Comment