Skip to content

[Bug] - Failure report needs to tell us which file #64

@matthew-carroll

Description

@matthew-carroll

Here's a failure I just had in a project:

00:10 +7 -3: /golden_tester/follow_the_leader/test_goldens/preferred_position_aligner_test.dart: Follower > preferred position aligner > not enough space on either side                               
Golden scene failed (✅ 2/4, ❌ 2/4):
✅ Left (Constrained)
❌ Top (Constrained) (240px, 1.07%)
❌ Right (Constrained) (220px, 0.98%)
✅ Bottom (Constrained)

The output tells me the test file that failed, but it doesn't tell me the golden file that failed, or where I can find the failure scene. Both of those pieces of information should be available in failure output.

Each failing scene should also mention the scene description for further clarity.

Also, the final output from the test run shows the usual Dart/Flutter output:

00:10 +7 -4: Some tests failed.   

We may not be able to change that, but we should at least add a final line of our own. The number of tests doesn't correspond to the number of passing/failing goldens, so its easy to get the wrong idea. We should have a line that's something like:

Golden Tests: 34 Passed, 12 Failed

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions