Skip to content

Add timestamps for pte generation in CI #10761

@lucylq

Description

@lucylq
Contributor

🚀 The feature, motivation and pitch

Regression tracking for aot export times: add some timestamps to indicate how long the llama export took:

$PYTHON_EXECUTABLE -m examples.models.llama.export_llama ${EXPORT_ARGS}

And fail if the export time is over some threshold. For now, we can set the threshold to be the time it takes to export the longest-running configuration.

The breakdown can look something like this:

  1. Add timestamps for the export command in CI.
  2. Observe the time it takes for each llama test in CI runs. Expect different export commands to show different export times.
  3. Add a fail, if the export time exceeds the largest time observed in (2), with some buffer; I guess we expect some variability with each run.
  4. Optional add-on: make (3) more fine-grained, and targeted per-command.

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

Activity

self-assigned this
on May 7, 2025
lucylq

lucylq commented on May 7, 2025

@lucylq
ContributorAuthor
added
module: ciIssues related to continuous integration
triagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
on May 9, 2025
removed their assignment
on May 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issueGood for newcomersmodule: ciIssues related to continuous integrationtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

    Type

    No type

    Projects

    Status

    To triage

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @Jack-Khuu@lucylq

        Issue actions

          Add timestamps for pte generation in CI · Issue #10761 · pytorch/executorch