Skip to content

junit-xml output attribute can be too big to handle with -vv #12223

Open
@Redoubts

Description

@Redoubts

Suppose you have a test that may fail with a very large diff; say:

def test_xyz() -> None:
    assert ["x"] * 10_000 == ["x\n"] * 10_000

With normal verbosity (0), the junit-xml output would look like this:

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
 <testsuite errors="0" failures="1" hostname="..." name="pytest" skipped="0" tests="1" time="1.172" timestamp="...">
  <testcase classname="test_file" file="test_file.py" line="0" name="test_xyz" time="0.767">
   <failure message="AssertionError: assert ['x', 'x', 'x...'x', 'x', ...] == ['x\n', 'x\n'...', 'x\n', ...]
  At index 0 diff: 'x' != 'x\n'
  Use -v to get more diff">
    def test_xyz() -&gt; None:
&gt;       assert ["x"] * 10_000 == ["x\n"] * 10_000
E       AssertionError: assert ['x', 'x', 'x...'x', 'x', ...] == ['x\n', 'x\n'...', 'x\n', ...]
E         At index 0 diff: 'x' != 'x\n'
E         Use -v to get more diff

test_file.py:2: AssertionError
   </failure>
  </testcase>
 </testsuite>
</testsuites>

noting the <failure message='...'> tag's message and body both have a Use -v to get more diff.

If you do supply a -vv, then both the body of the <failure> tag, AND the content of the message attribute can be very large. The latter can be problematic for parsers, and projects like BeautifulSoup and lxml will fail to parse this correctly (truncating or crashing, respectively).

Maybe this is a limitation of the junit schema, but is there anything acceptable that can be done to reduce only the content of the message attribute? I think it's more expected and "fine" for the body of a tag to be really long, but an attribute less so.

applies to pytest 8.1.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    plugin: junitxmlrelated to the junitxml builtin plugintype: performanceperformance or memory problem/improvement

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions