-
Notifications
You must be signed in to change notification settings - Fork 636
Description
🐛 Describe the bug
Over the weekend I start observing timeout when exporting models.
Affected models:
- mv3
- mv2
Affected configs: - xnnpack_q8
- coreml_fp16
Affected platforms:
- MocOS only.
Expected:
The export jobs used to finish within 30mins
Actual:
Some jobs start hitting the timeout threshold 60min since 1/25
1/26
- mv3, xnnpack_q8: 60min: https://github.com/pytorch/executorch/actions/runs/12980356644/job/36197276541
- mv3, coreml_fp16: 60min: https://github.com/pytorch/executorch/actions/runs/12980356644/job/36197276201
1/25
- mv3 xnnpack_q8: 60min: https://github.com/pytorch/executorch/actions/runs/12970194466/job/36175156930
- mv2 xnnpack_q8: 60min: https://github.com/pytorch/executorch/actions/runs/12970194466/job/36175156520
View historical runs of (mv3, xnnpack_q8) on macOS: https://hud.pytorch.org/hud/pytorch/executorch/main/1?per_page=50&name_filter=apple-perf%20%2F%20export-models%20(mv3%2C%20xnnpack&mergeLF=true
View historical runs of (mv3, coreml_fp16) on macOS: https://hud.pytorch.org/hud/pytorch/executorch/main/1?per_page=50&name_filter=apple-perf%20%2F%20export-models%20(mv3%2C%20coreml&mergeLF=true
Use this query to view all details from the scheduled runs: https://github.com/pytorch/executorch/actions/workflows/apple-perf.yml?query=event%3Aschedule
Versions
trunk
Activity
guangy10 commentedon Jan 27, 2025
There isn't much data about how long each step takes. To understand where the slowness comes from, we probably can start adding timer to each step.