fix(rss): read an element's whole text, not only its first node

[kevin (kevin9327)]

What this changes

An RSS item body written as a pretty-printed CDATA section is dropped. The item keeps its
heading; the content underneath it disappears.

<item>
  <title>Example item</title>
  <description>
    <![CDATA[<p>The <strong>body</strong> of the item.</p>]]>
  </description>
</item>

# Example feed
Example feed description

## Example item

That layout is ordinary in hand-written and CMS-generated feeds, and it is what WordPress
emits for content:encoded.

Why

_get_data_by_tag_name reads one node:

fc = nodes[0].firstChild
if fc:
    if hasattr(fc, "data"):
        return fc.data
return None

An element's text is not necessarily one node. The example above reaches the parser as three
children -- a text node holding "\n ", the CDATA section, then another text node -- so
firstChild is the indentation and the value is never read. The same happens whenever a
CDATA section sits next to text:

<title>Quarterly <![CDATA[R&D]]> report</title>   ->  ## Quarterly

_get_data_by_tag_name backs title, description, pubDate and content:encoded on the
RSS side and title, updated, summary and content on the Atom side, so every one of
them truncates at the first CDATA boundary.

Fix

Join the element's own text and CDATA children instead of reading only the first one. This is
what _collect_node_text in _epub_converter already does for the same minidom shape.

The join is deliberately not recursive: only the element's direct text and CDATA children are
read, which is exactly the set firstChild was sampling from. Element children keep behaving
as before -- <summary type="xhtml"> is still handled by _get_atom_content before this
function is reached, and an element whose first child is markup still returns None.

Tests

Added to packages/markitdown/tests/test_rss_converter.py:

• an RSS <description> whose CDATA sits on its own line
• an RSS <title> with a CDATA section in the middle of the text
• an Atom <summary type="html"> whose CDATA sits on its own line
• pinned: a <description> that is nothing but a CDATA section, which already worked
• pinned: an item with no <description> at all, which must stay absent rather than become
  an empty string

Against unmodified main:

E  AssertionError: assert 'The **body** of the item.' in '# Example feed\nExample feed description\n\n## Example item\n'
E  AssertionError: assert '## Quarterly R&D report' in '# Example feed\nExample feed description\n\n## Quarterly \nBody.'
E  AssertionError: assert 'A *structured* summary.' in '# Example feed\n\n## Example entry\n'
3 failed, 14 passed

With the fix:

17 passed

How I tested

Windows 11, Python 3.12, editable install of packages/markitdown[all].

pytest tests/test_rss_converter.py   ->  3 failed, 14 passed  (before)
pytest tests/test_rss_converter.py   ->  17 passed            (after)
pytest tests/                        ->  18 failed, 431 passed, 4 skipped

Those 18 are unchanged by this PR: main gives 18 failed, 426 passed, 4 skipped on this
machine before any edit, and the 5 added tests account for the difference. They are the CLI
stdout-encoding, Windows file-URI and speech-transcription tests that need a UTF-8 console,
a case-sensitive path and network/ffmpeg.

black --check clean on both files.

[Copilot]

🟡 Changes recommended

Elements beginning with markup no longer preserve the stated previous behavior.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Fixes RSS/Atom extraction so adjacent text and CDATA nodes are preserved.

Changes:

• Joins direct text and CDATA children.
• Adds RSS and Atom regression tests.

File summaries

File	Description
_rss_converter.py	Updates element text extraction.
test_rss_converter.py	Adds CDATA regression coverage.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[afourney]

Addressed alternatively in #2432