fix(pdf): merge paragraph breaks spuriously inserted mid-sentence

[Pujitha Paladugu (pujitha24)]

Motivation:
PDF text extraction occasionally inserts a spurious blank-line paragraph
break in the middle of a sentence. A genuine paragraph, heading, or list
item essentially never starts with a lowercase letter, so this can be
detected and corrected.

Approach:
Add _merge_wrapped_paragraph_breaks, a text-level post-processing pass
(alongside the existing _merge_partial_numbering_lines in the same
file) applied to the final markdown output. It splits text on blank
lines and rejoins a block with the previous one (using a single space)
when the previous block doesn't end in sentence-terminal punctuation
(.!?:;"'"') and the next block starts with a lowercase letter
(optionally after a single opening parenthesis, to catch wrapped
parentheticals). Short lettered/numbered list markers ("a)", "(i)",
"1.", with or without a following space) are excluded via
_LIST_MARKER_PATTERN so list items are not glued together.

Validation:
Added packages/markitdown/tests/test_pdf_wrapped_paragraph_breaks.py
with 9 unit tests covering: merging a lowercase continuation, merging a
lowercase parenthetical continuation, leaving a real paragraph break
alone, leaving an uppercase-starting next block alone, a no-op single
paragraph, not merging lettered list items ("a)"/"b)"), not merging
parenthetical list items ("(i)"/"(ii)"), not merging a standalone
numeric paragraph into a following paragraph, and not merging list
markers that have no space after the marker (e.g. "a)text").

Report: #2370
Signed-off-by: Pujitha Paladugu 10557236+pujitha24@users.noreply.github.com
Assisted-by: claude-sonnet-5 (via Claude Code)

[Pujitha Paladugu (pujitha24)]

@microsoft-github-policy-service agree