[ARM64] Performance regression: PerfLabTests.CastingPerf2.CastingPerf.IntObj

[adamsitnik]

After running benchmarks for 3.1 vs 5.0 using "Ubuntu arm64 Qualcomm Machines" owned by the JIT Team, I've found out that PerfLabTests.CastingPerf2.CastingPerf.IntObj has regressed x2.

It looks like these are ARM64 specific regressions, I was not able to reproduce it for ARM (the 32-bit variant).

Repro

git clone https://github.com/dotnet/performance.git
py ./performance/scripts/benchmarks_ci.py -f netcoreapp3.1 netcoreapp5.0 --architecture arm64 --filter 'PerfLabTests.CastingPerf2.CastingPerf.IntObj'

BenchmarkDotNet=v0.12.1.1405-nightly, OS=ubuntu 16.04
Unknown processor
  [Host]     : .NET Core 3.1.8 (CoreCLR 4.700.20.41105, CoreFX 4.700.20.41903), Arm64 RyuJIT
  Job-PVNQZA : .NET Core 3.1.8 (CoreCLR 4.700.20.41105, CoreFX 4.700.20.41903), Arm64 RyuJIT
  Job-PXIHWO : .NET Core 5.0.0 (CoreCLR 5.0.20.41714, CoreFX 5.0.20.41714), Arm64 RyuJIT

Method	Toolchain	Mean	Ratio
IntObj	netcoreapp3.1	466.4 us	1.00
IntObj	netcoreapp5.0	1,001.1 us	2.15

cc @kunalspathak

category:cq
theme:needs-triage
skill-level:expert
cost:large

[JulieLeeMSFT]

@CarolEidt please help look into this.

CC @dotnet/jit-contrib

[CarolEidt]

@adamsitnik - could you clarify how to run these on one of the Ubuntu arm64 systems? The command you give above appears to be using windows pathnames, and if I try this:

python scripts/benchmarks_ci.py -f netcoreapp3.1 netcoreapp5.0 --architecture arm64 --filter 'PerfLabTests.CastingPerf2.CastingPerf.IntObj'

I get a syntax error on line 39

[AndyAyersMS]

Assuming you have the right builds of 3.1 and 5.0 installed and dotnet is on your path, and you have cloned the perf repo (to say ~/repos/performance, then

cd ~/repos/performance/src/benchmarks/micro
dotnet run -c Release -f net5.0 -- -f netcoreapp3.1 netcoreapp5.0 --architecture arm64 --filter 'PerfLabTests.CastingPerf2.CastingPerf.IntObj'

should work. Add -d to get the BDN produced disassembly.

[CarolEidt]

Thanks @AndyAyersMS !

[AndyAyersMS]

Also this may be touching on the relatively new cast caching, so cc @VSadov.

[VSadov]

If this is a simple unbox, it should be jit-inlined.
Even if not inlined it should not hit cast cache, since unbox requires exact type match*, but it could be calling a managed helper now and there is a small penalty due to tiering/R2R indirection. It is not a lot, but may be noticeable on super fast casts.

I will take a look.

*enums match with underlying types as well and thus have some special handling, but that still does not need to use cache.

[adamsitnik]

could you clarify how to run these on one of the Ubuntu arm64 systems? The command you give above appears to be using windows pathname

please excuse me for that, I must have copy-pasted it from previous Windows specifc issue. I've fixed the description

I get a syntax error on line 39

If you append 3 to python it should use python 3.x and work:

python3 scripts/benchmarks_ci.py -f netcoreapp3.1 netcoreapp5.0 --architecture arm64 --filter 'PerfLabTests.CastingPerf2.CastingPerf.IntObj'      

Add -d to get the BDN produced disassembly.

Unfortunately, the BDN disassembler does not support ARM. Some time ago we have switched to use Iced library and it does not support ARM yet (https://github.com/0xd4d/iced/issues/79, https://github.com/0xd4d/iced/issues/80)

[CarolEidt]

Looking at the code generation, the only difference is that in 5.0 several address constants have been CSE'd. The loop goes from 108 bytes down to 64 bytes, but in both cases is small enough that alignment could make a big difference. Unless there's objection I would suggest we close this, and reference #8108 once again. (Perhaps we should add support for a specified loop alignment, even if only for use in benchmarking to validate or invalidate hypotheses such as this.)

BEFORE:

G_M590_IG03:        ; offs=000028H, size=003CH, gcrefRegs=0000 {}, byrefRegs=0000 {}, byref, isz

IN0008: 000028      movz    x1, #0xd1ffab1e
IN0009: 00002C      movk    x1, #0xd1ffab1e LSL #16
IN000a: 000030      movk    x1, #0xd1ffab1e LSL #32
IN000b: 000034      ldr     x20, [x1]
IN000c: 000038      ldr     x1, [x20]
IN000d: 00003C      movz    x0, #0xd1ffab1e
IN000e: 000040      movk    x0, #0xd1ffab1e LSL #16
IN000f: 000044      movk    x0, #0xd1ffab1e LSL #32
IN0010: 000048      cmp     x1, x0
IN0011: 00004C      beq     G_M590_IG04
IN0012: 000050      mov     x1, x20
IN0013: 000054      movz    x0, #0xd1ffab1e
IN0014: 000058      movk    x0, #0xd1ffab1e LSL #16
IN0015: 00005C      movk    x0, #0xd1ffab1e LSL #32
IN0016: 000060      bl      CORINFO_HELP_UNBOX

G_M590_IG04:        ; offs=000064H, size=0030H, gcrefRegs=100000 {x20}, byrefRegs=0000 {}, byref, isz

IN0017: 000064      ldr     w0, [x20,#8]
IN0018: 000068      movz    x1, #0xd1ffab1e
IN0019: 00006C      movk    x1, #0xd1ffab1e LSL #16
IN001a: 000070      movk    x1, #0xd1ffab1e LSL #32
IN001b: 000074      str     w0, [x1]
IN001c: 000078      add     w19, w19, #1
IN001d: 00007C      movz    x0, #0xd1ffab1e
IN001e: 000080      movk    x0, #0xd1ffab1e LSL #16
IN001f: 000084      movk    x0, #0xd1ffab1e LSL #32
IN0020: 000088      ldr     w0, [x0]
IN0021: 00008C      cmp     w19, w0
IN0022: 000090      blt     G_M590_IG03

AFTER:

G_M50398_IG03:        ; offs=000038H, size=001CH, bbWeight=4    PerfScore 36.00, gcrefRegs=0000 {}, byrefRegs=0000 {}, byref, isz

IN000b: 000038      movz    x1, #0xd1ffab1e
IN000c: 00003C      movk    x1, #0xd1ffab1e LSL #16
IN000d: 000040      movk    x1, #0xd1ffab1e LSL #32
IN000e: 000044      ldr     x22, [x1]
IN000f: 000048      ldr     x1, [x22]
IN0010: 00004C      cmp     x1, x21
IN0011: 000050      beq     G_M50398_IG05

G_M50398_IG04:        ; offs=000054H, size=000CH, bbWeight=1    PerfScore 2.00, gcrefRegs=400000 {x22}, byrefRegs=0000 {}, byref

IN0012: 000054      mov     x1, x22
IN0013: 000058      mov     x0, x21
IN0014: 00005C      bl      CORINFO_HELP_UNBOX

G_M50398_IG05:        ; offs=000060H, size=0018H, bbWeight=4    PerfScore 36.00, gcrefRegs=400000 {x22}, byrefRegs=0000 {}, byref, isz

IN0015: 000060      ldr     w0, [x22,#8]
IN0016: 000064      str     w0, [x20,#8]
IN0017: 000068      add     w19, w19, #1
IN0018: 00006C      ldr     w0, [x20]
IN0019: 000070      cmp     w19, w0
IN001a: 000074      blt     G_M50398_IG03