Make char::is_ascii_whitespace branchless on 32 and 64-bit targets

[lnicola]

No description provided.

[rust-highfive]

r? @cramertj

(rust_highfive has picked a reviewer for you, use r? to override)

[lnicola]

is_ascii_whitespace/std time:   [267.44 ns 267.82 ns 268.32 ns]                                    
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) high mild
  6 (6.00%) high severe
is_ascii_whitespace/pr  time:   [196.86 ns 197.11 ns 197.37 ns]                                   
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) low severe
  2 (2.00%) low mild
  3 (3.00%) high mild
  5 (5.00%) high severe

Benchmark code:

use criterion::{black_box, criterion_group, criterion_main, Criterion};

fn bench_is_ascii_whitespace(c: &mut Criterion) {
    let mut group = c.benchmark_group("is_ascii_whitespace");
    group.bench_function("std", |b| {
        b.iter(|| {
            let mut n = 0;
            for i in 0..128u8 {
                if is_whitespace::std_is_ascii_whitespace(black_box(&(i as char))) {
                    n += 1;
                }
            }
            black_box(n);
        })
    });
    group.bench_function("pr", |b| {
        b.iter(|| {
            let mut n = 0;
            for i in 0..128u8 {
                if is_whitespace::pr_is_ascii_whitespace(black_box(&(i as char))) {
                    n += 1;
                }
            }
            black_box(n);
        })
    });
}

criterion_group!(benches, bench_is_ascii_whitespace);
criterion_main!(benches);

Generated code: https://rust.godbolt.org/z/Ws69P3

[lnicola]

Lies, damned lies and statistics

This might not be a real improvement. Here I ran one version first, and then the other (red is std):

[llogiq]

If you look at the generated assembly of the match-based version, you'll find that LLVM already does the u32 as bit set trick, and generates leaner code.

[lnicola]

I haven't looked at the machine code, but the new version is one instruction shorter and (maybe more importantly) branchless.

[llogiq]

Being branchless will not be very beneficial here, because a) In normal text, there isn't that much whitespace (or even other chars < 33), and so the branch will often be taken resulting in only a cmp, jg and xor being run and b) branch prediction has become very good on most architectures (although it'll be interesting to see the result on ARM, which was historically worse than Intel or AMD; unfortunately cargo asm doesn't work on my phone; will look into --emit asm and report back later). Also even if there's 1 more instruction, with the branchy version not all instructions are actually used, so that shouldn't make a difference.

[bors]

☔ The latest upstream changes (presumably #77630) made this pull request unmergeable. Please resolve the merge conflicts.

Note that reviewers usually do not review pull requests until merge conflicts are resolved! Once you resolve the conflicts, you should change the labels applied by bors to indicate that your PR is ready for review. Post this as a comment to change the labels:

@rustbot modify labels: +S-waiting-on-review -S-waiting-on-author

[lnicola]

I was cleaning up some old git branches and I noticed that this PR was partially vindicated by the LLVM 13, 15 and 19 upgrades, which output branchless code:

is_ascii_whitespace_1_55:
        mov     ecx, dword ptr [rdi]
        add     ecx, -9
        cmp     ecx, 23
        ja      .LBB0_2
        mov     eax, 8388635
        shr     eax, cl
        and     al, 1
        ret
.LBB0_2:
        xor     eax, eax
        ret

is_ascii_whitespace_1_56:
        mov     eax, dword ptr [rdi]
        add     eax, -9
        cmp     eax, 24
        setb    cl
        mov     edx, 8388635
        bt      edx, eax
        setb    al
        and     al, cl
        ret

is_ascii_whitespace_1_65:
        mov     eax, dword ptr [rdi]
        cmp     eax, 33
        setb    cl
        movabs  rdx, 4294981120
        bt      rdx, rax
        setb    al
        and     al, cl
        ret

is_ascii_whitespace_1_82:
        mov     ecx, dword ptr [rdi]
        cmp     ecx, 33
        setb    dl
        movabs  rax, 4294981120
        shr     rax, cl
        and     al, dl
        ret

is_ascii_whitespace_pr_1_55:
        mov     eax, dword ptr [rdi]
        add     eax, -1
        cmp     eax, 32
        setb    cl
        mov     edx, -2147476736
        bt      edx, eax
        setb    al
        and     al, cl
        ret

is_ascii_whitespace_pr_1_82:
        mov     ecx, dword ptr [rdi]
        dec     ecx
        cmp     ecx, 32
        setb    dl
        mov     eax, 1
        shl     eax, cl
        test    eax, -2147476736
        setne   al
        and     al, dl
        ret