Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
100 commits
Select commit Hold shift + click to select a range
727782e
Add iOSInnerLoopParser, wire Startup.cs, fix Reporter.cs
davidnguyen-tech Apr 3, 2026
ee23cf0
Add shared Python infrastructure for iOS inner loop
davidnguyen-tech Apr 3, 2026
664f061
Add mauiiosinnerloop scenario directory (pre/test/post.py)
davidnguyen-tech Apr 3, 2026
e20e8a4
Add setup_helix.py for iOS Helix bootstrap
davidnguyen-tech Apr 3, 2026
cb6315e
Add maui_scenarios_ios_innerloop.proj Helix workitem definition
davidnguyen-tech Apr 3, 2026
7c9e0c6
Wire iOS inner loop into CI pipeline YAML and routing
davidnguyen-tech Apr 3, 2026
0d86772
Add physical device support for iOS inner loop
davidnguyen-tech Apr 3, 2026
edab33f
Fix review findings: devicectl commands, UDID regex, edit paths, Xcod…
davidnguyen-tech Apr 3, 2026
9942170
Quality fixes: tempfile.mkstemp, devicectl fallback, report guard
davidnguyen-tech Apr 3, 2026
d21e9b7
[TEMP] Disable all pipeline jobs except iOS Inner Loop
davidnguyen-tech Apr 3, 2026
25a2846
Fix iOS Inner Loop build output capture and logging
davidnguyen-tech Apr 3, 2026
53e647e
Capture and print dotnet build output for iOS Inner Loop
davidnguyen-tech Apr 3, 2026
bb1e6d2
Increase iOS Inner Loop Helix timeout to 2.5 hours
davidnguyen-tech Apr 3, 2026
565f8eb
Bypass Xcode version validation for iOS Inner Loop
davidnguyen-tech Apr 3, 2026
60d8954
Fix iOS simulator RID for Intel x64 Helix machines
davidnguyen-tech Apr 3, 2026
259c10e
Fix FileExistsError when copying traces on incremental iterations
davidnguyen-tech Apr 3, 2026
f62020a
Remove verbose build output from iOS Inner Loop
davidnguyen-tech Apr 3, 2026
62aecd2
Fix ILLink error on Helix machines with older Xcode
davidnguyen-tech Apr 3, 2026
68d6b4c
Re-enable all pipeline jobs after iOS inner loop CI validation
davidnguyen-tech Apr 3, 2026
ba4e0d1
Add install timing, CoreCLR config, and device support for iOS inner …
davidnguyen-tech Apr 3, 2026
60f1106
Add fallback for workload install version skew in pre.py
davidnguyen-tech Apr 3, 2026
995fe94
Add workload install fallback to setup_helix.py
davidnguyen-tech Apr 3, 2026
ae4c34c
Exclude simulator work item from device jobs
davidnguyen-tech Apr 3, 2026
3fbe535
Clean up iOS inner loop: unify ioshelper API, simplify runner and pos…
davidnguyen-tech Apr 4, 2026
92a538a
Fix device job failures: iOSRid propagation and setup_helix exit hand…
davidnguyen-tech Apr 4, 2026
5c092e4
Pass iOSRid as explicit MSBuild property to fix device job dispatch
davidnguyen-tech Apr 4, 2026
edc2ccb
Rename simulator job identifiers for clarity
davidnguyen-tech Apr 4, 2026
d8b4ff1
Switch ioshelper.py from simctl/devicectl to mlaunch for install/launch
davidnguyen-tech Apr 4, 2026
c398150
Use --installsim for granular simulator install timing
davidnguyen-tech Apr 4, 2026
8e89a1f
Re-select Xcode to match iOS SDK's required version after workload in…
davidnguyen-tech Apr 5, 2026
5d80e1f
Resolve booted simulator UDID for mlaunch and remove ValidateXcodeVer…
davidnguyen-tech Apr 5, 2026
6756d03
Run --launchsim as background process to prevent timeout
davidnguyen-tech Apr 5, 2026
803ec21
Add post-build code signing for physical device deployment
davidnguyen-tech Apr 6, 2026
2bcbb64
Make Xcode handling diagnostic-only on Helix — match Device Startup s…
davidnguyen-tech Apr 7, 2026
c2a65e3
Add --skip-manifest-update to workload install retry on Helix
davidnguyen-tech Apr 7, 2026
4253fcd
Resolve sign tool path for Helix device deployment
davidnguyen-tech Apr 7, 2026
bf97725
Remove /usr/local/bin from PATH - shadows venv python
davidnguyen-tech Apr 7, 2026
259573f
Restore active Xcode selection in setup_helix.py — fixes Xcode versio…
davidnguyen-tech Apr 7, 2026
6d7dfc8
Fix sign tool resolution — fall back to login shell execution
davidnguyen-tech Apr 7, 2026
e74cb12
Fix Xcode version parsing — regex greedily captured trailing dot
davidnguyen-tech Apr 7, 2026
15ed711
Use in-box manifest for workload install — avoid Xcode version skew
davidnguyen-tech Apr 7, 2026
189b555
Use in-box manifest on build agent — stop shipping incompatible SDK p…
davidnguyen-tech Apr 8, 2026
482d22e
Revert to --from-rollback-file and make Xcode diagnostic-only
davidnguyen-tech Apr 8, 2026
c8ca867
Add timeout to workload install to prevent NuGet download hangs
davidnguyen-tech Apr 9, 2026
2a6cdf7
Temporarily pin iOS workload to 26.2 for Xcode compatibility
davidnguyen-tech Apr 9, 2026
2f8fd09
Add active Xcode selection and version validation to setup_helix.py
davidnguyen-tech Apr 9, 2026
c29f631
fixup! Add active Xcode selection and version validation to setup_hel…
davidnguyen-tech Apr 9, 2026
9f4ddab
Fix Xcode selection: match workload-required version, not highest
davidnguyen-tech Apr 9, 2026
b859032
fixup! Fix Xcode selection: match workload-required version, not high…
davidnguyen-tech Apr 9, 2026
2c1d0e7
Remove temporary workload pin — restore dynamic NuGet feed resolution
davidnguyen-tech Apr 13, 2026
ece266a
Remove silent workload fallback — fail loud if nightly packs unavailable
davidnguyen-tech Apr 13, 2026
c380cbb
Add manifest patching fallback for missing net10.0 packs
davidnguyen-tech Apr 13, 2026
c64ec2f
Fix review findings: packs type handling, timeouts, diagnostics
davidnguyen-tech Apr 13, 2026
ca4f030
Document Helix manifest-patching dependency in setup_helix.py
davidnguyen-tech Apr 13, 2026
4148296
Fix iOS inner loop local device support
davidnguyen-tech Apr 13, 2026
7ca397b
Improve code quality: deduplicate workload install, fix correctness b…
davidnguyen-tech Apr 13, 2026
28f94ac
Handle missing sign tool gracefully for local device deployment
davidnguyen-tech Apr 13, 2026
afe6351
Add run-local.sh for local iOS inner loop measurements
davidnguyen-tech Apr 14, 2026
625c595
Add SDK-aware Xcode selection for local measurements
davidnguyen-tech Apr 14, 2026
5882cfa
Preserve binlogs in results directory
davidnguyen-tech Apr 14, 2026
b3f8ed6
Copy versions.json to results directory
davidnguyen-tech Apr 14, 2026
e2657fb
Uninstall stale app from physical device before first deploy
davidnguyen-tech Apr 14, 2026
c3d1ab6
Clean stale app/traces dirs before setup to fix XAML duplicate errors
davidnguyen-tech Apr 14, 2026
a0d7d63
Validate cold startup measurement before including in results
davidnguyen-tech Apr 14, 2026
321dcb9
Track ILLink, CompileNativeCode, R2R targets in iOS parser
davidnguyen-tech Apr 14, 2026
bb15078
Fix correctness bugs found during code review
davidnguyen-tech Apr 14, 2026
78cb6b6
Measure .app bundle size in iOS inner loop results
davidnguyen-tech Apr 14, 2026
e2f7b89
Fix watchdog event ordering and find_app_bundle simulator/device pref…
davidnguyen-tech Apr 14, 2026
5844d6a
Add restore timeout to prevent hung NuGet restores from consuming Hel…
davidnguyen-tech Apr 14, 2026
dbaf86e
Fix channel derivation in run-local.sh
davidnguyen-tech Apr 15, 2026
0477e20
Update reference test data with app-size counter
davidnguyen-tech Apr 15, 2026
0c5fb2c
Use exact SDK version for bootstrap instead of channel resolution
davidnguyen-tech Apr 15, 2026
db65738
Revert bootstrap to use -channel main for nightly SDK
davidnguyen-tech Apr 15, 2026
0dcac50
Merge remote-tracking branch 'origin/main' into nguyendav/maui-ios-in…
davidnguyen-tech Apr 15, 2026
2c46f0d
Remove 'daily' quality from channels without daily builds
davidnguyen-tech Apr 15, 2026
e44815c
Update measurement results with .NET 11 preview SDK
davidnguyen-tech Apr 15, 2026
8ed94a9
Revert "Update measurement results with .NET 11 preview SDK"
davidnguyen-tech Apr 15, 2026
62e5e53
Revert "Remove 'daily' quality from channels without daily builds"
davidnguyen-tech Apr 15, 2026
b7d0005
Fix workload install feed mismatch and JSON parsing
davidnguyen-tech Apr 15, 2026
22d2235
Try Mac.iPhone.13.Perf queue for iOS inner loop
davidnguyen-tech Apr 21, 2026
47358a9
Use sudo for logarchive cleanup in physical-device watchdog measurement
davidnguyen-tech Apr 28, 2026
428290a
Create per-workitem simulator under user namespace to fix CoreSimulat…
davidnguyen-tech May 5, 2026
405b3a9
Take ownership of CoreSimulator folders before booting simulator
davidnguyen-tech May 5, 2026
5615edf
[iOS Inner Loop] Cross-arch RID + mlaunch for device install/launch
davidnguyen-tech May 5, 2026
3fb438a
[iOS Inner Loop] Reliable simulator launch detection + device signing…
davidnguyen-tech May 5, 2026
cc62ba9
iOS inner loop: simctl launch + sentinel skip for unsigned device
davidnguyen-tech May 5, 2026
7b2522c
iOS inner loop: stabilization check uses host ps, not simctl spawn ps
davidnguyen-tech May 5, 2026
8805533
iOS inner loop: upload is best-effort, do not fail work item on infra…
davidnguyen-tech May 5, 2026
ede4cac
iOS inner loop: revert upload best-effort, document mlaunch/xcrun split
davidnguyen-tech May 6, 2026
83e4a4b
Remove local measurement results from PR; gitignore results/
davidnguyen-tech May 6, 2026
960266a
iOS inner loop: prune Apple read-only mounts when chowning CoreSimulator
davidnguyen-tech May 6, 2026
2208af7
Revert unrelated init.sh PYTHONPATH change leaked into prior commit
davidnguyen-tech May 6, 2026
1e6ff5a
iOS inner loop: fail loudly when device signing infra is missing
davidnguyen-tech May 6, 2026
d05d763
iOS inner loop: sweep leaked PerfTest-iPhone-* simulators before booting
davidnguyen-tech May 6, 2026
c97143d
iOS inner loop: device job hard-fails on missing physical device — no…
davidnguyen-tech May 6, 2026
6b2a901
iOS inner loop: detect physical device BEFORE signing artifacts on de…
davidnguyen-tech May 6, 2026
c877a5f
Address review feedback: align with Android inner-loop PR
kotlarmilos May 20, 2026
9542806
Address review feedback: msbuild arg splitting, source restore, log s…
kotlarmilos May 20, 2026
56dd9f6
iOS inner loop: fix End-to-End typo, drop unused intermediate_binlogs
kotlarmilos May 20, 2026
11f5aae
Revert iPhone13 swap; iPhone17 matches main
kotlarmilos May 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 56 additions & 54 deletions eng/performance/maui_scenarios_ios.proj
Original file line number Diff line number Diff line change
Expand Up @@ -2,45 +2,50 @@
<Import Project="Scenarios.Common.props" />

<PropertyGroup>
<IncludeXHarnessCli>true</IncludeXHarnessCli>
<_IsInnerLoop Condition="'$(RunKind)' == 'maui_scenarios_ios_innerloop'">true</_IsInnerLoop>
<IncludeXHarnessCli Condition="'$(_IsInnerLoop)' != 'true'">true</IncludeXHarnessCli>
</PropertyGroup>

<PropertyGroup>
<PropertyGroup Condition="'$(_IsInnerLoop)' != 'true'">
<AfterPreparePayloadWorkItemCommand>$(Python) post.py</AfterPreparePayloadWorkItemCommand>
<PreparePayloadOutDirectoryName>scenarios_out</PreparePayloadOutDirectoryName>
<PreparePayloadWorkItemBaseDirectory Condition="'$(TargetsWindows)' == 'true'">$(CorrelationPayloadDirectory)$(PreparePayloadOutDirectoryName)\</PreparePayloadWorkItemBaseDirectory>
<PreparePayloadWorkItemBaseDirectory Condition="'$(TargetsWindows)' != 'true'">$(CorrelationPayloadDirectory)$(PreparePayloadOutDirectoryName)/</PreparePayloadWorkItemBaseDirectory>

<_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'mono'">/p:UseMonoRuntime=true</_MSBuildArgs>
<_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr'">/p:UseMonoRuntime=false</_MSBuildArgs>

<!-- Mono FullAOT and CoreCLR R2R are considered as default configurations, so no need for explicit properties -->

<!-- We have to set the following two variables until https://github.com/dotnet/runtime/pull/124604 flows to us -->
<_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr'">$(_MSBuildArgs);/p:PublishReadyToRunStripDebugInfo=false;/p:PublishReadyToRunStripInliningInfo=false</_MSBuildArgs>

<!-- CoreCLR Interpreter -->
<_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr' and '$(CodegenType)' == 'Interpreter'">$(_MSBuildArgs);/p:PublishReadyToRun=false</_MSBuildArgs>
<!-- CoreCLR NativeAOT -->
<_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr' and '$(CodegenType)' == 'NativeAOT'">$(_MSBuildArgs);/p:PublishAot=true;/p:PublishAotUsingRuntimePack=true</_MSBuildArgs>
</PropertyGroup>

<PropertyGroup Condition="'$(_IsInnerLoop)' == 'true'">
<_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'mono'">/p:UseMonoRuntime=true</_MSBuildArgs>
<_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr'">/p:UseMonoRuntime=false</_MSBuildArgs>
<iOSRid Condition="'$(iOSRid)' == ''">iossimulator-x64</iOSRid>
<_MSBuildArgs>$(_MSBuildArgs) /p:RuntimeIdentifier=$(iOSRid)</_MSBuildArgs>
<_MSBuildArgs Condition="'$(iOSRid)' == 'ios-arm64'">$(_MSBuildArgs) /p:EnableCodeSigning=false</_MSBuildArgs>
<_MSBuildArgs>$(_MSBuildArgs) /p:MtouchLink=None</_MSBuildArgs>
<RunConfigsString>$(RuntimeFlavor)_Default</RunConfigsString>
<InnerLoopIterations Condition="'$(InnerLoopIterations)' == ''">3</InnerLoopIterations>
<_MacEnvVars>export DOTNET_ROOT=$HELIX_CORRELATION_PAYLOAD/dotnet;export DOTNET_CLI_TELEMETRY_OPTOUT=1;export DOTNET_MULTILEVEL_LOOKUP=0;export NUGET_PACKAGES=$HELIX_WORKITEM_ROOT/.packages;export PATH=$HELIX_CORRELATION_PAYLOAD/dotnet:$PATH</_MacEnvVars>
</PropertyGroup>

<Target Name="RemoveDotnetFromCorrelationStaging" BeforeTargets="BeforeTest">
<Message Text="Removing Dotnet Packs from Correlation Staging" Importance="high" />
<RemoveDir Directories="$(CorrelationPayloadDirectory)dotnet/packs" />
<!-- Remove scenario build intermediates to reduce payload size (published output is in scenarios_out/) -->
<RemoveDir Directories="@(MAUIiOSScenario -> '%(PayloadDirectory)/app/obj')" />
<RemoveDir Directories="@(MAUIiOSScenario -> '%(PayloadDirectory)/app/bin')" />
<RemoveDir Condition="'$(_IsInnerLoop)' != 'true'" Directories="@(MAUIiOSScenario -> '%(PayloadDirectory)/app/obj')" />
<RemoveDir Condition="'$(_IsInnerLoop)' != 'true'" Directories="@(MAUIiOSScenario -> '%(PayloadDirectory)/app/bin')" />
</Target>

<ItemDefinitionGroup>
<HelixWorkItem>
<Timeout>00:30</Timeout>
<Timeout Condition="'$(_IsInnerLoop)' != 'true'">00:30</Timeout>
<Timeout Condition="'$(_IsInnerLoop)' == 'true'">02:30</Timeout>
</HelixWorkItem>
</ItemDefinitionGroup>

<ItemGroup>
<ItemGroup Condition="'$(_IsInnerLoop)' != 'true'">
<MAUIiOSScenario Include=".NET iOS Default Template">
<ScenarioDirectoryName>netios</ScenarioDirectoryName>
<PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
Expand All @@ -67,15 +72,15 @@
</MAUIiOSScenario>
</ItemGroup>

<ItemGroup>
<ItemGroup Condition="'$(_IsInnerLoop)' != 'true'">
<PreparePayloadWorkItem Include="@(MAUIiOSScenario)">
<!-- Select the highest installed versioned Xcode bundle and avoid runner-image symlink aliases as they do not work with iOS SDK -->
<Command>XCODE_PATH=`find /Applications -maxdepth 1 -type d -name 'Xcode_*.app' | sort -t_ -k2 -V | tail -1` &amp;&amp; echo &quot;Selected Xcode: $XCODE_PATH&quot; &amp;&amp; sudo xcode-select -s &quot;$XCODE_PATH&quot; &amp;&amp; $(Python) pre.py publish -f $(PERFLAB_Framework)-ios --self-contained -c $(BuildConfig) -r ios-arm64 --msbuild=&quot;$(_MSBuildArgs)&quot; --binlog $(PreparePayloadWorkItemBaseDirectory)%(PreparePayloadWorkItem.ScenarioDirectoryName)/%(PreparePayloadWorkItem.ScenarioDirectoryName).binlog -o $(PreparePayloadWorkItemBaseDirectory)%(PreparePayloadWorkItem.ScenarioDirectoryName) &amp;&amp; rm -rf app/obj app/bin &amp;&amp; cd ../ &amp;&amp; zip -r %(PreparePayloadWorkItem.ScenarioDirectoryName).zip %(PreparePayloadWorkItem.ScenarioDirectoryName)</Command>
<WorkingDirectory>%(PreparePayloadWorkItem.PayloadDirectory)</WorkingDirectory>
</PreparePayloadWorkItem>
</ItemGroup>

<ItemGroup>
<ItemGroup Condition="'$(_IsInnerLoop)' != 'true'">
<HelixWorkItem Include="@(MAUIiOSScenario -> 'SOD - %(Identity) IPA Size')">
<PreCommands>cp -r $HELIX_CORRELATION_PAYLOAD/$(PreparePayloadOutDirectoryName)/%(HelixWorkItem.ScenarioDirectoryName) $HELIX_WORKITEM_ROOT/pub</PreCommands>
<Command>$(Python) test.py sod --scenario-name &quot;%(Identity)&quot; $(ScenarioArgs)</Command>
Expand Down Expand Up @@ -124,7 +129,7 @@
<![CDATA[
# PreCommands
export XHARNESSPATH=$XHARNESS_CLI_PATH

cp -v $HELIX_CORRELATION_PAYLOAD/$(PreparePayloadOutDirectoryName)/mauiios/MauiiOSDefault.ipa $HELIX_WORKITEM_ROOT/mauiios/MauiiOSDefault.zip
mkdir $HELIX_WORKITEM_ROOT/mauiios/pub
cp -v $HELIX_CORRELATION_PAYLOAD/$(PreparePayloadOutDirectoryName)/mauiios/versions.json $HELIX_WORKITEM_ROOT/mauiios/pub/versions.json
Expand Down Expand Up @@ -152,7 +157,7 @@
<![CDATA[
# PreCommands
export XHARNESSPATH=$XHARNESS_CLI_PATH

cp -v $HELIX_CORRELATION_PAYLOAD/$(PreparePayloadOutDirectoryName)/mauisamplecontentios/MauiSampleContentiOSDefault.ipa $HELIX_WORKITEM_ROOT/mauisamplecontentios/MauiSampleContentiOSDefault.zip
mkdir $HELIX_WORKITEM_ROOT/mauisamplecontentios/pub
cp -v $HELIX_CORRELATION_PAYLOAD/$(PreparePayloadOutDirectoryName)/mauisamplecontentios/versions.json $HELIX_WORKITEM_ROOT/mauisamplecontentios/pub/versions.json
Expand All @@ -172,44 +177,41 @@
]]>
</CustomCommands>
</XHarnessAppBundleToTest>
<!-- Now also disabled for normal mono runs. (Reenable for mono and native aot once fixed). Tracking issue: https://github.com/dotnet/performance/issues/3148 -->
<!-- <XHarnessAppBundleToTest Condition="'$(RuntimeFlavor)' == 'mono'" Include="Device Startup - iOS Maui Blazor Default Template">
<AppBundlePath>$(ScenariosDir)mauiblazorios.zip</AppBundlePath>
<WorkItemTimeout>00:15:00</WorkItemTimeout>
<TestTarget>ios-device</TestTarget>
<CustomCommands>
<![CDATA[
# PreCommands
export XHARNESSPATH=$XHARNESS_CLI_PATH

cp -v $HELIX_CORRELATION_PAYLOAD/$(PreparePayloadOutDirectoryName)/mauiblazorios/MauiBlazoriOSDefault.ipa $HELIX_WORKITEM_ROOT/mauiblazorios/MauiBlazoriOSDefault.zip
mkdir $HELIX_WORKITEM_ROOT/mauiblazorios/pub
cp -v $HELIX_CORRELATION_PAYLOAD/$(PreparePayloadOutDirectoryName)/mauiblazorios/versions.json $HELIX_WORKITEM_ROOT/mauiblazorios/pub/versions.json
unzip -d $HELIX_WORKITEM_ROOT/mauiblazorios $HELIX_WORKITEM_ROOT/mauiblazorios/MauiBlazoriOSDefault.zip
mv $HELIX_WORKITEM_ROOT/mauiblazorios/Payload/MauiBlazoriOSDefault.app $HELIX_WORKITEM_ROOT/mauiblazorios/MauiBlazoriOSDefault.app
cp -f embedded.mobileprovision $HELIX_WORKITEM_ROOT/mauiblazorios/MauiBlazoriOSDefault.app
cd $HELIX_WORKITEM_ROOT/mauiblazorios
sign MauiBlazoriOSDefault.app

# Testing commands
$(Python) test.py devicestartup -1-device-type ios -1-package-path MauiBlazoriOSDefault.app -1-package-name net.dot.mauiblazortesting -1-scenario-name "%(Identity)" -1-use-fully-drawn-time -1-fully-drawn-magic-string __MAUI_Blazor_WebView_OnAfterRender__ -1-startup-iterations 5 $(ScenarioArgs)
((result=$?))

# Post commands
$(Python) post.py
exit $result
]]>
</CustomCommands>
</XHarnessAppBundleToTest> -->
</ItemGroup>

<ItemGroup Condition="'$(_IsInnerLoop)' == 'true'">
<MAUIiOSInnerLoopScenario Include="MAUI iOS Inner Loop">
<ScenarioDirectoryName>mauiiosinnerloop</ScenarioDirectoryName>
<PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
</MAUIiOSInnerLoopScenario>
</ItemGroup>

<ItemGroup Condition="'$(_IsInnerLoop)' == 'true'">
<PreparePayloadWorkItem Include="@(MAUIiOSInnerLoopScenario)">
<Command>XCODE_PATH=`find /Applications -maxdepth 1 -type d -name 'Xcode_*.app' | sort -t_ -k2 -V | tail -1` &amp;&amp; echo &quot;Selected Xcode: $XCODE_PATH&quot; &amp;&amp; sudo xcode-select -s &quot;$XCODE_PATH&quot; &amp;&amp; $(Python) pre.py default -f $(PERFLAB_Framework)-ios</Command>
<WorkingDirectory>%(PreparePayloadWorkItem.PayloadDirectory)</WorkingDirectory>
</PreparePayloadWorkItem>
</ItemGroup>

<ItemGroup Condition="'$(_IsInnerLoop)' == 'true' and '$(iOSRid)' != 'ios-arm64'">
<HelixWorkItem Include="@(MAUIiOSInnerLoopScenario -> 'Inner Loop Simulator - %(Identity)')">
<PreCommands>$(_MacEnvVars);export IOS_RID=$(iOSRid);$(Python) setup_helix.py $(PERFLAB_Framework)-ios &quot;$(_MSBuildArgs)&quot; || exit $?</PreCommands>
<Command>$(Python) test.py iosinnerloop --csproj-path app/MauiiOSInnerLoop.csproj --edit-src &quot;src/MainPage.xaml.cs;src/MainPage.xaml&quot; --edit-dest &quot;app/Pages/MainPage.xaml.cs;app/Pages/MainPage.xaml&quot; --bundle-id com.companyname.mauiiosinnerloop -f $(PERFLAB_Framework)-ios -c Debug --msbuild-args &quot;$(_MSBuildArgs)&quot; --device-type simulator --inner-loop-iterations $(InnerLoopIterations) --scenario-name &quot;%(Identity)&quot; $(ScenarioArgs)</Command>
<PostCommands>export IOS_RID=$(iOSRid);$(Python) post.py</PostCommands>
<DownloadFilesFromResults>output.log</DownloadFilesFromResults>
</HelixWorkItem>
</ItemGroup>

<ItemGroup Condition="'$(_IsInnerLoop)' == 'true' and '$(iOSRid)' == 'ios-arm64'">
<HelixWorkItem Include="@(MAUIiOSInnerLoopScenario -> 'Inner Loop Device - %(Identity)')">
<PreCommands>$(_MacEnvVars);export IOS_RID=$(iOSRid);$(Python) setup_helix.py $(PERFLAB_Framework)-ios &quot;$(_MSBuildArgs)&quot; || exit $?</PreCommands>
<Command>$(Python) test.py iosinnerloop --csproj-path app/MauiiOSInnerLoop.csproj --edit-src &quot;src/MainPage.xaml.cs;src/MainPage.xaml&quot; --edit-dest &quot;app/Pages/MainPage.xaml.cs;app/Pages/MainPage.xaml&quot; --bundle-id com.companyname.mauiiosinnerloop -f $(PERFLAB_Framework)-ios -c Debug --msbuild-args &quot;$(_MSBuildArgs)&quot; --device-type device --inner-loop-iterations $(InnerLoopIterations) --scenario-name &quot;%(Identity)&quot; $(ScenarioArgs)</Command>
<PostCommands>export IOS_RID=$(iOSRid);$(Python) post.py</PostCommands>
<DownloadFilesFromResults>output.log</DownloadFilesFromResults>
</HelixWorkItem>
</ItemGroup>

<!--
This target is to work around the XHarness command that depend on scripts in ORIGPYPATH
being run before we get to run our normal Post commands. AddXHarnessCLI is the XHarness
Target so we just make sure we add this after that.
-->
<Target Name="ResetPYTHONPATHBeforeXHarnessCommand" AfterTargets="AddXHarnessCLI">
<Target Name="ResetPYTHONPATHBeforeXHarnessCommand" AfterTargets="AddXHarnessCLI" Condition="'$(_IsInnerLoop)' != 'true'">
<PropertyGroup>
<HelixPostCommands>export PYTHONPATH=$ORIGPYPATH;$(HelixPostCommands)</HelixPostCommands>
</PropertyGroup>
Expand Down
80 changes: 80 additions & 0 deletions eng/pipelines/sdk-perf-jobs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -565,6 +565,86 @@ jobs:
${{ each parameter in parameters.jobParameters }}:
${{ parameter.key }}: ${{ parameter.value }}

# Maui iOS Inner Loop (Mono) - Debug - Simulator
- template: /eng/pipelines/templates/build-machine-matrix.yml
parameters:
jobTemplate: /eng/pipelines/templates/run-scenarios-job.yml
buildMachines:
- osx-x64-ios-arm64
isPublic: false
jobParameters:
runKind: maui_scenarios_ios_innerloop
projectFileName: maui_scenarios_ios.proj
channels:
- main
runtimeFlavor: mono
codeGenType: Default
buildConfig: Debug
additionalJobIdentifier: Mono_InnerLoop_Simulator
${{ each parameter in parameters.jobParameters }}:
${{ parameter.key }}: ${{ parameter.value }}

# Maui iOS Inner Loop (CoreCLR) - Debug - Simulator
- template: /eng/pipelines/templates/build-machine-matrix.yml
parameters:
jobTemplate: /eng/pipelines/templates/run-scenarios-job.yml
buildMachines:
- osx-x64-ios-arm64
isPublic: false
jobParameters:
runKind: maui_scenarios_ios_innerloop
projectFileName: maui_scenarios_ios.proj
channels:
- main
runtimeFlavor: coreclr
codeGenType: Default
buildConfig: Debug
additionalJobIdentifier: CoreCLR_InnerLoop_Simulator
${{ each parameter in parameters.jobParameters }}:
${{ parameter.key }}: ${{ parameter.value }}

# Maui iOS Inner Loop (Mono) - Debug - Device
- template: /eng/pipelines/templates/build-machine-matrix.yml
parameters:
jobTemplate: /eng/pipelines/templates/run-scenarios-job.yml
buildMachines:
- osx-x64-ios-arm64
isPublic: false
jobParameters:
runKind: maui_scenarios_ios_innerloop
projectFileName: maui_scenarios_ios.proj
channels:
- main
runtimeFlavor: mono
codeGenType: Default
buildConfig: Debug
additionalJobIdentifier: Mono_InnerLoop_Device
runEnvVars:
- iOSRid=ios-arm64
${{ each parameter in parameters.jobParameters }}:
${{ parameter.key }}: ${{ parameter.value }}

# Maui iOS Inner Loop (CoreCLR) - Debug - Device
- template: /eng/pipelines/templates/build-machine-matrix.yml
parameters:
jobTemplate: /eng/pipelines/templates/run-scenarios-job.yml
buildMachines:
- osx-x64-ios-arm64
isPublic: false
jobParameters:
runKind: maui_scenarios_ios_innerloop
projectFileName: maui_scenarios_ios.proj
channels:
- main
runtimeFlavor: coreclr
codeGenType: Default
buildConfig: Debug
additionalJobIdentifier: CoreCLR_InnerLoop_Device
runEnvVars:
- iOSRid=ios-arm64
${{ each parameter in parameters.jobParameters }}:
${{ parameter.key }}: ${{ parameter.value }}

# Maui scenario benchmarks
- ${{ if false }}:
- template: /eng/pipelines/templates/build-machine-matrix.yml
Expand Down
2 changes: 1 addition & 1 deletion eng/pipelines/templates/run-performance-job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ jobs:
- '--is-scenario'
- ${{ if ne(length(parameters.runEnvVars), 0) }}:
- "--run-env-vars ${{ join(' ', parameters.runEnvVars)}}"
- ${{ if and(in(parameters.runKind, 'maui_scenarios_ios', 'maui_scenarios_android'), ne(parameters.runtimeFlavor, '')) }}:
- ${{ if and(in(parameters.runKind, 'maui_scenarios_ios', 'maui_scenarios_android', 'maui_scenarios_ios_innerloop'), ne(parameters.runtimeFlavor, '')) }}:
- '--runtime-flavor ${{ parameters.runtimeFlavor }}'
- ${{ if ne(parameters.osVersion, '') }}:
- '--os-version ${{ parameters.osVersion }}'
Expand Down
27 changes: 26 additions & 1 deletion scripts/run_performance_job.py
Original file line number Diff line number Diff line change
Expand Up @@ -590,6 +590,15 @@ def get_run_configurations(
if build_config is not None and build_config != DEFAULT_BUILD_CONFIG:
configurations["BuildConfig"] = build_config

# .NET MAUI iOS inner loop (build+deploy) scenarios
if run_kind == "maui_scenarios_ios_innerloop":
if not runtime_flavor in ("mono", "coreclr"):
raise Exception("Runtime flavor must be specified for maui_scenarios_ios_innerloop")
configurations["CodegenType"] = str(codegen_type)
configurations["RuntimeType"] = str(runtime_flavor)
if build_config is not None and build_config != DEFAULT_BUILD_CONFIG:
configurations["BuildConfig"] = build_config

return configurations

def get_work_item_command(os_group: str, target_csproj: str, architecture: str, perf_lab_framework: str, internal: bool, wasm: bool, bdn_artifacts_dir: str, wasm_coreclr: bool = False, only_sanity_check: bool = False):
Expand Down Expand Up @@ -1164,6 +1173,12 @@ def publish_dotnet_app_to_payload(payload_dir_name: str, csproj_path: str, self_
os.environ["CodegenType"] = args.codegen_type or ''
os.environ["BuildConfig"] = args.build_config or DEFAULT_BUILD_CONFIG

# Propagate run_env_vars to os.environ so they reach MSBuild
# evaluation as properties (e.g., iOSRid for device builds).
if args.run_env_vars:
for key, value in args.run_env_vars.items():
os.environ[key] = value

# TODO: See if these commands are needed for linux as they were being called before but were failing.
if args.os_group == "windows" or args.os_group == "osx":
break_system_packages = ["--break-system-packages"] if args.os_group == "osx" else []
Expand Down Expand Up @@ -1193,7 +1208,7 @@ def publish_dotnet_app_to_payload(payload_dir_name: str, csproj_path: str, self_
verbose=True).run()

# Search for additional binlogs generated by the maui scenarios prepare payload work items to copy to the artifacts log dir
if args.run_kind in ["maui_scenarios_android", "maui_scenarios_ios"]:
if args.run_kind in ["maui_scenarios_android", "maui_scenarios_ios", "maui_scenarios_ios_innerloop"]:
for binlog_path in glob(os.path.join(payload_dir, "scenarios_out", "**", "*.binlog"), recursive=True):
shutil.copy(binlog_path, ci_artifacts_log_dir)

Expand Down Expand Up @@ -1324,10 +1339,20 @@ def get_work_item_command_for_artifact_dir(artifact_dir: str):
only_sanity_check=args.only_sanity_check,
ios_strip_symbols=args.ios_strip_symbols,
ios_llvm_build=args.ios_llvm_build,
ios_rid=args.run_env_vars.get("iOSRid"),
fail_on_test_failure=fail_on_test_failure,
scenario_arguments=scenario_arguments or None)

if args.send_to_helix:
# Re-apply run_env_vars so they reach SendToHelix MSBuild evaluation.
# The env was snapshot/restored earlier (environ_copy), and while
# os.environ.update() preserves new keys, some shell wrappers may not
# inherit them reliably. Explicitly re-setting ensures properties like
# iOSRid (used in .proj ItemGroup conditions) are available.
if args.run_env_vars:
for key, value in args.run_env_vars.items():
os.environ[key] = value

perf_send_to_helix(perf_send_to_helix_args)

results_glob = os.path.join(helix_results_destination_dir, '**', '*perf-lab-report.json')
Expand Down
Loading
Loading