markitdown-ocr: placeholder prefix collision drops OCR blocks in DOCX with 11 or more images
markitdown-ocr 0.1.0, packages/markitdown-ocr/src/markitdown_ocr/_docx_converter_with_ocr.py, checked against main @ b752951.
The internal placeholder is MARKITDOWNOCRBLOCK{} formatted with the plain index, and the substitution walks the indices in ascending order with str.replace. MARKITDOWNOCRBLOCK1 is a prefix of MARKITDOWNOCRBLOCK10 through MARKITDOWNOCRBLOCK19, so once a document has eleven images or more, replacing index 1 corrupts every two-digit placeholder. Some OCR blocks get duplicated in the wrong place, others disappear. Nothing raises.
The code
Line 30:
_PLACEHOLDER = "MARKITDOWNOCRBLOCK{}"convert, line 139:
for i, raw_text in enumerate(ocr_texts):
placeholder = _PLACEHOLDER.format(i)
ocr_block = f"*[Image OCR]\n{raw_text}\n[End OCR]*"
md = md.replace(placeholder, ocr_block)At i == 1 the search string is MARKITDOWNOCRBLOCK1, which also matches the first 20 characters of MARKITDOWNOCRBLOCK10, ...11 and so on. Those become <block for index 1>0, <block for index 1>1, and when the loop later reaches indices 10 and 11 there is nothing left to replace.
No document needed to see it:
from markitdown_ocr._docx_converter_with_ocr import _PLACEHOLDER
md = " ".join(_PLACEHOLDER.format(i) for i in range(13))
md = md.replace(_PLACEHOLDER.format(0), "<text-0>")
md = md.replace(_PLACEHOLDER.format(1), "<text-1>")
print(md)
# <text-0> <text-1> MARKITDOWNOCRBLOCK2 ... <text-1>0 <text-1>1 <text-1>2
# placeholders 10, 11, 12 are gone
Reproduction
repro_markitdown_ocr.py builds its own documents and calls no external service. The OCR service is a stub returning a marker derived from the image bytes.
pip install markitdown markitdown-ocr python-docx pillow
python repro_markitdown_ocr.py
Case A is a document with 12 distinct images in natural order, so relationship order equals body order and the matching logic is correct by construction. Only this bug can show:
=== A. 12 images, body order == relationship order ===
images in the document body : 12
OCR blocks produced : 12
blocks in the right place : 10/12
Looking at which markers came out:
expected : 1a14 7b8c 868d b349 c9bf b88e 118b 7c27 39b6 b7fa ffb5 4f5c
produced : 1a14 7b8c 868d b349 c9bf b88e 118b 7c27 39b6 b7fa 7b8c 7b8c
repeated : {'7b8c': 3} the text for index 1, emitted three times
lost : ['ffb5', '4f5c'] the texts for indices 10 and 11
Case C in the same script is a three-image control that passes.
Why the current tests don't catch it
Every fixture in packages/markitdown-ocr/tests/ocr_test_data/ has at most three images, and the threshold here is exactly eleven. Ten images or fewer are unaffected.
Possible fix
Give the token a terminator so no index token is a prefix of another:
_PLACEHOLDER = "MARKITDOWNOCRBLOCK{}END"MARKITDOWNOCRBLOCK1END is not a prefix of MARKITDOWNOCRBLOCK11END, and the token stays alphanumeric so it still survives the HTML to markdown step unescaped. Both call sites already read the constant, so that's the whole change.
If changing the token isn't wanted, iterating the replacement in descending index order works too, or re.sub with a word boundary.
I can send a PR.
Found while looking into a different bug in the same converter, where OCR text is matched to images by position. Filed separately since the cause is unrelated, though a document with eleven or more images usually hits both.
markitdown-ocr: placeholder prefix collision drops OCR blocks in DOCX with 11 or more images
markitdown-ocr 0.1.0,
packages/markitdown-ocr/src/markitdown_ocr/_docx_converter_with_ocr.py, checked againstmain@b752951.The internal placeholder is
MARKITDOWNOCRBLOCK{}formatted with the plain index, and the substitution walks the indices in ascending order withstr.replace.MARKITDOWNOCRBLOCK1is a prefix ofMARKITDOWNOCRBLOCK10throughMARKITDOWNOCRBLOCK19, so once a document has eleven images or more, replacing index 1 corrupts every two-digit placeholder. Some OCR blocks get duplicated in the wrong place, others disappear. Nothing raises.The code
Line 30:
convert, line 139:At
i == 1the search string isMARKITDOWNOCRBLOCK1, which also matches the first 20 characters ofMARKITDOWNOCRBLOCK10,...11and so on. Those become<block for index 1>0,<block for index 1>1, and when the loop later reaches indices 10 and 11 there is nothing left to replace.No document needed to see it:
Reproduction
repro_markitdown_ocr.py builds its own documents and calls no external service. The OCR service is a stub returning a marker derived from the image bytes.
Case A is a document with 12 distinct images in natural order, so relationship order equals body order and the matching logic is correct by construction. Only this bug can show:
Looking at which markers came out:
Case C in the same script is a three-image control that passes.
Why the current tests don't catch it
Every fixture in
packages/markitdown-ocr/tests/ocr_test_data/has at most three images, and the threshold here is exactly eleven. Ten images or fewer are unaffected.Possible fix
Give the token a terminator so no index token is a prefix of another:
MARKITDOWNOCRBLOCK1ENDis not a prefix ofMARKITDOWNOCRBLOCK11END, and the token stays alphanumeric so it still survives the HTML to markdown step unescaped. Both call sites already read the constant, so that's the whole change.If changing the token isn't wanted, iterating the replacement in descending index order works too, or
re.subwith a word boundary.I can send a PR.
Found while looking into a different bug in the same converter, where OCR text is matched to images by position. Filed separately since the cause is unrelated, though a document with eleven or more images usually hits both.