Skip to content

03_usage.md

Latest commit

 

History

History
64 lines (48 loc) · 2.02 KB

03_usage.md

File metadata and controls

64 lines (48 loc) · 2.02 KB
Raw

Usage — decision tree, batch, OCR, post-processing

All commands use the installed pdf2md. Run pdf2md doctor first if unsure.

Decision tree

Got a PDF
  │
  ├─ Academic paper (formulas / tables / multi-column)?
  │     ├─ no  → pdf2md file x.pdf -o out/ --pipeline doc      (PyMuPDF4LLM, fast)
  │     └─ yes → pdf2md file x.pdf -o out/ --pipeline paper    (MinerU, GPU)
  │                └─ MinerU formulas unsatisfying (English only)?
  │                     → pdf2md file x.pdf -o out/ -t nougat  (fallback)
  │
  ├─ Scanned / image-only?  → --pipeline paper --lang ch|en    (MinerU auto-OCR)
  │
  └─ Batch?
        ├─ flat directory  → pdf2md dir  papers/ -o out/ --pipeline paper
        └─ nested tree     → pdf2md tree papers/ -o out/ --pipeline doc

Is a PDF scanned (no text layer)?

pdftotext input.pdf - | head     # empty output => image-only, needs OCR
pdfimages -list input.pdf | head # shows embedded image streams

For image-only PDFs, always use --pipeline paper (MinerU) with a --lang hint. doc/docling will emit empty or garbled text.

Batch

# General documents, flat directory (fast)
pdf2md dir reports/ -o out/ --pipeline doc

# Papers, recursive, mirror the input tree
pdf2md tree library/ -o out/ --pipeline paper -l en

Language hints

Common --lang values: ch, en, ja, ko. Full list in MinerU docs.

Post-processing

Runs automatically. To inspect / re-run manually:

python -m pdf2md.postprocess out/paper.md --detect-only   # report PUA chars
python -m pdf2md.postprocess out/paper.md -i              # fix in place
python -m pdf2md.postprocess out/*.md -i                  # batch

Known limits

Tool Limit
PyMuPDF4LLM no OCR, no formula recognition
Docling first run downloads ~1 GB of models
MinerU needs GPU; transformers pinned 4.49.0; Chinese decimal points need post-processing
Nougat English papers only, may hallucinate, unmaintained