gh-149489: Fix ElementTree serialization to HTML by serhiy-storchaka · Pull Request #149490 · python/cpython

serhiy-storchaka · 2026-05-07T11:21:00Z

The content of comments, processing instructions and elements "xmp", "iframe", "noembed", "noframes", and "plaintext" is no longer escaped.
The "plaintext" element no longer have the closing tag.
Add support of empty attributes (with value None).

Issue: Fix ElementTree serialization to HTML #149489

* The content of comments, processing instructions and elements "xmp", "iframe", "noembed", "noframes", and "plaintext" is no longer escaped. * The "plaintext" element no longer have the closing tag. * Add support of empty attributes (with value None).

StanFromIreland · 2026-05-07T21:12:50Z

Updating to fix some errors we introduced on the main branch.

scoder · 2026-05-08T07:12:30Z

I'm not done with the review yet, but I find it risky to silently change output in a point release like 3.1[34].x. If we change this, it's probably still fine for 3.15, but I'd rather see the maintenance releases excluded.

serhiy-storchaka · 2026-05-08T08:23:18Z

Well, the fix for XML was applied to 2.7 and 3.2 and was not backported to 2.6 and 3.1. It introduces a risk of XML/HTML injection if the comment content was not previously sanitized. See #149468.

scoder · 2026-05-08T08:51:59Z

The content of comments, processing instructions and elements "xmp", "iframe", "noembed", "noframes", and "plaintext" is no longer escaped.
[...] It introduces a risk of XML/HTML injection if the comment content was not previously sanitized.

Well, yes. If we remove the current escaping, then we leave user code unprotected. Definitely not something that users should expect from a point release.

serhiy-storchaka · 2026-05-10T09:13:43Z

Test failures are related to #149571.

ezio-melotti · 2026-05-28T21:33:07Z

It would be good to add some comments to clearly state the intent of the tests (e.g. comments shouldn't be excaped, comments are omitted in text serialization, empty attrs only work in html, etc.).

Is there any reason why the there are 3 similar but different test strings ('<spam> & ham', 'ham & eggs < spam', '<spam>&ham')?

Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>

miss-islington-app · 2026-05-29T21:04:55Z

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

bedevere-app · 2026-05-29T21:05:18Z

GH-150592 is a backport of this pull request to the 3.15 branch.

bedevere-app · 2026-05-29T21:05:21Z

GH-150593 is a backport of this pull request to the 3.14 branch.

bedevere-app · 2026-05-29T21:05:26Z

GH-150594 is a backport of this pull request to the 3.13 branch.

miss-islington-app · 2026-05-29T21:07:11Z

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

* The content of comments, processing instructions and elements "xmp", "iframe", "noembed", "noframes", and "plaintext" is no longer escaped. * The "plaintext" element no longer have the closing tag. * Add support of empty attributes (with value None). (cherry picked from commit bcd29e4) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

miss-islington-app · 2026-05-29T21:08:13Z

Sorry @serhiy-storchaka, I had trouble completing the backport.
Please retry by removing and re-adding the "needs backport to 3.15" label.
Please backport backport using cherry_picker on the command line.

cherry_picker bcd29e466f55d8b4e3849ed6ada8ce86a46f5072 3.15

miss-islington-app · 2026-05-29T21:18:42Z

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

* The content of comments, processing instructions and elements "xmp", "iframe", "noembed", "noframes", and "plaintext" is no longer escaped. * The "plaintext" element no longer have the closing tag. * Add support of empty attributes (with value None). (cherry picked from commit bcd29e4) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

miss-islington-app · 2026-05-29T21:19:15Z

Sorry @serhiy-storchaka, I had trouble completing the backport.
Please retry by removing and re-adding the "needs backport to 3.15" label.
Please backport backport using cherry_picker on the command line.

cherry_picker bcd29e466f55d8b4e3849ed6ada8ce86a46f5072 3.15

bedevere-app · 2026-05-29T21:20:27Z

GH-150595 is a backport of this pull request to the 3.15 branch.

…H-150595) * The content of comments, processing instructions and elements "xmp", "iframe", "noembed", "noframes", and "plaintext" is no longer escaped. * The "plaintext" element no longer have the closing tag. * Add support of empty attributes (with value None). (cherry picked from commit bcd29e4)

…GH-149490) (pythonGH-150595) * The content of elements "xmp", "iframe", "noembed", "noframes", and "plaintext" is no longer escaped. * The "plaintext" element no longer have the closing tag. (cherry picked from commit bcd29e4) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…GH-149490) * The content of elements "xmp", "iframe", "noembed", "noframes", and "plaintext" is no longer escaped. * The "plaintext" element no longer have the closing tag. (cherry picked from commit bcd29e4) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

bedevere-app · 2026-05-29T22:14:07Z

GH-150596 is a backport of this pull request to the 3.14 branch.

…H-150596) * The content of elements "xmp", "iframe", "noembed", "noframes", and "plaintext" is no longer escaped. * The "plaintext" element no longer have the closing tag. (cherry picked from commit bcd29e4)

…H-150596) (GH-150609) * The content of elements "xmp", "iframe", "noembed", "noframes", and "plaintext" is no longer escaped. * The "plaintext" element no longer have the closing tag. (cherry picked from commit c42e6d3) (cherry picked from commit bcd29e4) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

KowalskiThomas · 2026-06-01T13:34:59Z

    if tag is Comment:
-        write("<!--%s-->" % _escape_cdata(text))
+        write("<!--%s-->" % text)


I'm a bit late to the party, but what happens if text contains -->? Shouldn't this be escaped (if there's a way to escape it, I don't think so 🤔) or probably instead be rejected altogether?

KowalskiThomas · 2026-06-01T13:35:12Z

+        write("<!--%s-->" % text)
    elif tag is ProcessingInstruction:
-        write("<?%s?>" % _escape_cdata(text))
+        write("<?%s?>" % text)


I guess same question here -- what if text contains ?>?

scoder · 2026-06-01T15:20:45Z

what happens if text contains --> ?

Yes, that was my concern in #149490 (comment) as well. Note that this part of the changes was not backported and will only appear in 3.15+.

The HTML5 spec is explicit about what is not allowed in comments:
https://html.spec.whatwg.org/#comments

It's not clear about what is allowed in processing instructions, but that probably just means that ?> should be rejected and nothing else. PIs are rather relaxed by design.

I think it's worth rejecting non well-formed text. lxml does it in the API when creating a Comment / ProcessingInstruction element, but for xml.etree, it might be more reasonable to reject it in the serialiser.

bedevere-app Bot mentioned this pull request May 7, 2026

Fix ElementTree serialization to HTML #149489

Open

bedevere-app Bot added the awaiting core review label May 7, 2026

serhiy-storchaka requested review from ezio-melotti and scoder May 7, 2026 11:21

serhiy-storchaka added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels May 7, 2026

serhiy-storchaka force-pushed the xml-etree-serialize-html branch from 9f169a8 to e41d61c Compare May 7, 2026 17:22

serhiy-storchaka mentioned this pull request May 7, 2026

gh-149468: Add option to validate ElementTree during serialization #149469

Open

Merge branch 'main' into xml-etree-serialize-html

a9795d1

StanFromIreland added the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label May 7, 2026

serhiy-storchaka added 2 commits May 28, 2026 18:09

Merge branch 'main' into xml-etree-serialize-html

0f35342

Update tests after fixing itertext().

e111ef9

ezio-melotti reviewed May 28, 2026

View reviewed changes

serhiy-storchaka and others added 2 commits May 29, 2026 09:14

Apply suggestion from @ezio-melotti

752f4e7

Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>

Add comments to tests.

57b5739

ezio-melotti approved these changes May 29, 2026

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting core review labels May 29, 2026

serhiy-storchaka merged commit bcd29e4 into python:main May 29, 2026
59 checks passed

bedevere-app Bot removed the awaiting merge label May 29, 2026

serhiy-storchaka deleted the xml-etree-serialize-html branch May 29, 2026 21:04

serhiy-storchaka removed needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels May 29, 2026

serhiy-storchaka removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label May 29, 2026

serhiy-storchaka added the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label May 29, 2026

miss-islington-app Bot assigned serhiy-storchaka May 29, 2026

serhiy-storchaka added needs backport to 3.15 pre-release feature fixes, bugs and security fixes and removed needs backport to 3.15 pre-release feature fixes, bugs and security fixes labels May 29, 2026

bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label May 29, 2026

KowalskiThomas reviewed Jun 1, 2026

View reviewed changes

Uh oh!

Conversation

serhiy-storchaka commented May 7, 2026 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

StanFromIreland commented May 7, 2026

Uh oh!

scoder commented May 8, 2026

Uh oh!

serhiy-storchaka commented May 8, 2026

Uh oh!

scoder commented May 8, 2026

Uh oh!

serhiy-storchaka commented May 10, 2026

Uh oh!

ezio-melotti May 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

miss-islington-app Bot commented May 29, 2026

Uh oh!

bedevere-app Bot commented May 29, 2026

Uh oh!

bedevere-app Bot commented May 29, 2026

Uh oh!

bedevere-app Bot commented May 29, 2026

Uh oh!

miss-islington-app Bot commented May 29, 2026

Uh oh!

miss-islington-app Bot commented May 29, 2026

Uh oh!

miss-islington-app Bot commented May 29, 2026

Uh oh!

miss-islington-app Bot commented May 29, 2026

Uh oh!

bedevere-app Bot commented May 29, 2026

Uh oh!

bedevere-app Bot commented May 29, 2026

Uh oh!

KowalskiThomas Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

KowalskiThomas Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

scoder commented Jun 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

serhiy-storchaka commented May 7, 2026 •

edited by bedevere-app Bot

Loading