[Backend Tester] Report delegation statistics #12846
Conversation
| in the test report. | ||
| """ | ||
| if counter is not None: | ||
| return str(dict(sorted(counter.items()))) |
There was a problem hiding this comment.
we can make this pretty but OK for now
| elapsed = time.perf_counter() - lower_start_time | ||
| extra_stats["lower_time"] = timedelta(seconds=elapsed) | ||
| except Exception as e: | ||
| elapsed = time.perf_counter() - lower_start_time | ||
| extra_stats["lower_time"] = timedelta(seconds=elapsed) | ||
| return build_result(TestResult.LOWER_FAIL, e) | ||
|
|
||
| # Compute delegation statistics. Use the ETRecord to access the edge dialect graph between |
There was a problem hiding this comment.
Curious why do we need to go through ETRecord?
There was a problem hiding this comment.
This is because we need the graph in between to_edge and to_backend when using to_edge_transform_and_lower in order to see the edge-dialect graph before partitioning. In theory, we have the info with the exported program, but it's aten dialect and pre-decompositions, so it's hard to reason about what's delegated.
With to_edge_transform_and_lower, we don't otherwise have a way to inspect the graph between the edge and lowering steps. ETRecord has effectively the same problem (they need the pre-delegation edge graph), so I'm using it here.
GregoryComer
added
the
release notes: none
|
As a note: This might have broken the Arm backend testing, we have a fix to be merged soon if testing goes well. But seem like a nice thing, would be fun to play with this! |
Report total number of delegated and undelegated nodes and breakdown by operator count.
Example from CoreML add: