Add configurable event tracing names for method execution

[cptspacemanspiff]

Summary

This is kind of a quality of life thing when working with exported modules with multiple methods.

The Issue:

By default, all methods of a module use the same profiling event and event block. This is fine, when one only has a single method exported, however the profiling data becomes less meaningful if there are multiple methods.

This is related to #8030 where I am exporting multiple methods on a single module and having them all share internal state.

The change:

In Method::execute, allow the caller to optionally pass in const char* block name and profiling event name.

Error execute(
      char const* block_name = "Method::execute",
      const char* profile_event_name = "Method::execute");

By default these are the same as what was hardcoded internally.

Additionally change the behavior of Module::execute to pass in the name of the current method.

  auto block_name = std::string("Execute::" + method_name);
  auto profile_event_name = std::string("Method::execute::" + method_name);

This does change the current behavior of Module, as anyone using that api will no longer get the previous hardcoded "Method::execute" names.

cc @JacobSzwejbka @dbort

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8033

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 605d0f1 with merge base 3eea1f1 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[JacobSzwejbka]

added @shoumikhin and @tarun292

[dbort]

The core runtime API already lets users pass a different EventTracer instance to each Method via Program::load_method(). I'd rather find some way to have the tracer instance record that a given trace is for a specific method, instead of modifying core runtime APIs.

The higher-level Module interface could construct a tracer for each method, and could pass the method name to each tracer instance.

[shoumikhin]

@dbort is there any chance to log the method name for the event tracer in Method::execute(), e.g. for the event_tracer_entry?

[dbort]

is there any chance to log the method name for the event tracer in Method::execute(), e.g. for the event_tracer_entry?

Sure we could do something like that. The question is: what's the best way to log it? @tarun292 ? Would it make sense to update the EventTracer API to have a hook like begin_method(const char* name)? Or just stick it in a profiling event string?

The main pain is that if we want to construct a string with the method, like "Method(NAME)::execute" then we'd need to do so in a stack buffer like

char buf[32];
snprintf(buf, sizeof(buf), "Method(%s)::execute", method_name);

which would take some extra time, so we'd want to be sure to avoid it when no event tracer is installed, and possibly compile it out if profiling is disabled.

[cptspacemanspiff]

So, looking back at this, I do not have a full understanding of the Event Tracer design / thought process.

For the use case of Executorch programs with multiple graphs/ methods is the thought process that one would generate an ET_Dump for each method independently? IE method_1.etdump method_2.etdump?

Or would this all get collapsed into a single etdump, with internal separation based on the block_name/ profile_name / (is there some other separation possibility?)

[dbort]

s the thought process that one would generate an ET_Dump for each method independently? IE method_1.etdump method_2.etdump?

I believe so; in fact I think it's typical to generate a separate dump for each execution of a given method. @tarun292 or @Gasoonjia would know better

[Gasoonjia]

one would generate an ET_Dump for each method independently? IE method_1.etdump method_2.etdump

Correct. Each ETDump file corresponds to a method's inference; every time a method is executed, all debug/runtime related data will be stored in a single ETDump.

[cptspacemanspiff]

If my understanding is correct, the best way to handle this is to create and pass an independent event tracer instance for each method that gets called.

The way my current code works is that I end up adding an additional abstraction layer and wrapping the module class. This wrapping class calls load_method on all methods.

I guess additionally during this stage, my wrapper should (optionally) create and pass event tracers for each method, storing them for later write-out. (I am currently doing this for the root event tracer only)

This seems easy and if this is the recommended method, I can close out this as nothing in executorch needs to change. The only real downside is that I end up with multiple-files per program.

Side-note:

I have made some changes on the python side to etrecord generation/inspector, which are similar to this issue, but again the question is what is the recommended approach?

#8336