Critical issues before stabilization

[calebzulawski]

I wanted to put together a more technical list of issues to be solved before we can stabilize (see rust-lang/rust#86656).

There are many more important issues like performance on some targets, or important missing functions, but IMO those don't prevent stabilization. In this list I've trimmed down the issues to major API-related issues that can't be changed, or are difficult to change, after stabilization.

Issues to be solved

• Restricting the number of lanes. In my opinion, the LaneCount<N>: SupportedLaneCount bound makes the API exceptionally cumbersome. I've also had some trouble writing generics around it, particularly when making functions that change the number of lanes (the trait solver sometimes simply fails). Adding the bound often feels like boilerplate, and I found myself looking for ways to launder the bound, like adding unnecessary const parameters. Making this a post-monomorphization error (I found Design meeting: Non-local errors (aka post-monomorphization errors) lang-team#195 helpful) might be the way to go, or perhaps there's a way to make a const fn panic. Cons: the trait bound is very explicit and hiding the error states could possibly do more harm than good when identifying the source of a build failure.
• Mask element type. I'm not confident that the mask type for Simd<f32, N> should be Simd<i32, N> (see Change Mask element to match Simd element #322 for discussion). I think it would be much more straightforward if the types simply matched. Cons: this would require an extra cast when using i32 masks for f32 vectors or similar, and makes implementing From<Mask> for Mask impossible (since pointer masks must be generic Mask<*const T, N>, maybe all pointers could use the same mask element?).
• Swizzle functions. They're difficult to use and not a very good API, but Rust doesn't currently allow for anything much better. We could hold off stabilizing arbitrary swizzles, but that's a big limitation.

Non-issues, but things that should be done

• Are we happy with the traits the way they are? e.g. SimdPartialEq, SimdInt.
• The API should be partitioned into multiple unstable features that can be stabilized independently.

Updates (after filing this issue)

Lane count

• I tried condensing LaneCount an SimdElement into a single bound Simd<T, N>: Supported. This doesn't work well for a variety of reasons. One example: scatter/gather use Simd<T, N>, Simd<*const T, N>, and Simd<usize, N>. Each of these would need their own bound, rather than using one LaneCount bound since they all share N.
• I recommend either keeping the bounds as they are now (LaneCount and SimdElement) or turning LaneCount into a non-local error.

Swizzles

I tried the following swizzle code. It requires the incomplete adt_const_params feature. Even with this feature enabled, it's impossible to implement functions like reverse, because const generics can't access the generic const parameters.

    pub fn swizzle<const M: usize, const INDEX: &'static [usize]>(self) -> Simd<T, M>
    where
        LaneCount<M>: SupportedLaneCount,
    {
        // SAFETY: `self` is a vector and the swizzle index is a const array of u32
        unsafe {
            intrinsics::simd_shuffle(
                self,
                self,
                const {
                    assert!(M == INDEX.len(), "`M` must equal the length of `INDEX`");
                    let mut r = [0; M];
                    let mut i = 0;
                    while i < M {
                        assert!(
                            INDEX[i] < N,
                            "indices must be less than the input vector length"
                        );
                        r[i] = INDEX[i] as u32;
                        i += 1;
                    }
                    r
                },
            )
        }
    }

[BurntSushi]

w00t! Very excited to see this. :D :D :D

Are we happy with the traits the way they are? e.g. SimdPartialEq, SimdInt.

Is there high level docs anywhere describing how the API is laid out? Like, a guided tour of everything? I realize that's a tall order to ask for before the API is set in stone because it takes work to stabilize. But I'm asking because as of right now, I find the API to be very overwhelming. There are a lot of traits and upon seeing them, I immediately wonder how they all fit together. And in particular, why they exist. That is, if the API were less trait heavy, what would we lose?

Looking at the names of the traits, I can make some guesses as to what some of them are for. And it looks plausible that they could all mostly be explained (at a high level) without too much exposition.

I have opinions on the names of the traits too, but I think that fun can be saved for later. :P

I find the number of concrete types to be large but not overwhelming. At a quick glance, I can immediately understand what they mean and what connects them. While I do have some light SIMD background, I'm confident that the large number of concrete types could be explained by a ~paragraph in the crate docs.

The API should be partitioned into multiple unstable features that can be stabilized independently.

Do you have a 50,000-foot view of what the partitioning might look like? I don't mean to put you on the spot and ask for a detailed proposal, but just a general feeling of what you think might make sense.

Swizzle functions. They're difficult to use and not a very good API, but Rust doesn't currently allow for anything much better. We could hold off stabilizing arbitrary swizzles, but that's a big limitation.

I agree that missing swizzles is a rather large limitation. For me personally, unless there is a language feature that you're pretty confident is right at the horizon, or if there's a small language feature that you can get the lang team on board with quickly, I would go to war with the army you have when everything else is ready to stabilize. But this is just a general sense of things personally, and I'm sure there is a deeper analysis that could be done here. e.g., "If Rust had language feature Foo, then the swizzle API could be written like this which would make things simpler and/or unlock new use cases."

[programmerjake]

e.g., "If Rust had language feature Foo, then the swizzle API could be written like this which would make things simpler and/or unlock new use cases."

that would be: if LLVM had dynamic swizzles (using the shuffle LLVM IR instruction), then we could just have the swizzle pattern be a function argument and rely on inlining and const-propagation, like we do with atomic orderings (technically we could do that to some extent in rustc, but it really should be in LLVM).

[calebzulawski]

Is there high level docs anywhere describing how the API is laid out?

Nope, that's definitely something we should have. Personally, I think we could use more submodules to make the division a little clearer. Each of these could be documented independently, with a bit of guidance in the main std::simd module:

• Simd, Mask, and the various aliases in std::simd
• num containing SimdInt, SimdFloat, etc
• ptr containing SimdConstPtr and SimdMutPtr
• cmp containing SimdPartialEq etc
• swizzle containing Swizzle, Which, etc

That is, if the API were less trait heavy, what would we lose?

The original implementation had no traits. This results in two related issues. To implement recip, you need impl Simd<f32, N> and impl Simd<f64, N> and end up with two identical (but separate in docs) recip functions. For integers this is even more verbose and distracting. The second issue is that you now have overlapping functions, e.g. recip (for floats), count_ones (for ints) and wrapping_add (for pointers) all implemented on the Simd type. With traits, we were able to put generic vector functions on Simd and leave anything element-type-specific to a trait.

The API should be partitioned into multiple unstable features that can be stabilized independently.

Do you have a 50,000-foot view of what the partitioning might look like?

Offhand, I would say something like:

• portable_simd_types: just Simd, Mask, and a handful of generic functions (these types implement Add, Sub, etc)
• portable_simd_swizzle if we want to split off swizzling for later
• portable_simd_scatter_gather: scatter/gather is another fairly complicated API we might want to hold back
• portable_simd_traits: for all of the traits above, once we're comfortable with how they're divided up etc

I agree that missing swizzles is a rather large limitation. For me personally, unless there is a language feature that you're pretty confident is right at the horizon, or if there's a small language feature that you can get the lang team on board with quickly, I would go to war with the army you have when everything else is ready to stabilize.

One option might be to implement a slightly less powerful set of swizzle2<I0, I1>, swizzle4<I0, I1, I2, I3>, etc functions that are not quite as powerful as we'd like (impossible to make generic over N) and wait on stabilizing the completely generic interface for when we can do something like fn swizzle<const N: usize, const I: [usize; N]>() or perhaps something even better.

[RazrFalcon]

I hope it will be released soon. I've tried using it in one of my crates (linebender/tiny-skia#88) and it mostly works. There are some glitches which I haven't debugged yet, but I don't think they are critical.
Performance is all over the place on AArch64 (haven't tested x86 yet). Some stuff become faster, some slower. Maybe a porting bug, maybe imperfect codegen. But I fine with small performance regressions as long as I can ditch manual, unsafe SIMD code.

I personally do no use swizzle, therefore those functions are not important to me.

[thomcc]

That is, if the API were less trait heavy, what would we lose?

Realistically we'd probably gain things. As it is, there are a number of APIs that have been hard to add due to the need to make them fit into the trait-heavy design.

That said, my stance here (that we should not have gone with the traits as it's lead to a cumbersome and inflexible design) is not a popular one, so YMMV.

[calebzulawski]

As far as restricting the number of lanes goes, I tried combining the bound into something like:

pub struct Simd<T, const N: usize>(...) where Self: Supported;

In theory, this seemed like a good compromise between the current verbose bounds and using a non-local error. In practice, it wasn't really possible to implement this. It turns out we use the separation between the element bound (SimdElement) and the lane count bound (SupportedLaneCount) in a fair number of places. For example, functions that use multiple vector types, such as scatter/gather, require the bound for each type, which actually ends up more verbose. (Currently, they all share a lane count bound). Even worse, functions that only use another vector type internally (like Simd<usize, ...> in functions that take pointer vectors) need the extra bounds and leak implementation details.

After this experiment, I think the best option is to add two new compiler attributes like #[portable_simd_max_elements(N)] and #[portable_simd_require_power_of_2_elements], which error when unsupported lane counts are used.

[programmerjake]

could we just use:

impl<T: ..., const N: usize> Simd<T, N> {
    const VALID: () = {
        let valid = N > 0 && N < 64;
        if !cfg!(all_lane_counts_iirc) {
            valid &= N.is_power_of_two();
        }
        assert!(valid, "invalid lane count");
    };
    pub fn each_method(self, ...) -> ... {
        Self::VALID; // use so it triggers a post-mono error
        ...
    }
}

actually, no, we can't, since just copying a Simd type requires a supported lane count.

[calebzulawski]

Also, it would be easy to accidentally forget

[scottmcm]

since just copying a Simd type requires a supported lane count.

Any chance we'd be willing to say that we're ok allowing the type to exist for non-PoT lane counts, but then just not offering anything other than Copy and the array conversions with them? Then it's fine that we don't have a good way to do + or shuffle for Simd<f32, 21>, but we don't force it to be bound everywhere just to exist...

(But I guess that exposes layout questions that we don't want to decide just yet, even if we left them explicitly unspecified.)

it would be easy to accidentally forget

We could maybe have a -Z flag to ICE on non-power-of-two vectors in the codegen, or something, to help catch it.

[programmerjake]

since just copying a Simd type requires a supported lane count.

Any chance we'd be willing to say that we're ok allowing the type to exist for non-PoT lane counts, but then just not offering anything other than Copy and the array conversions with them?

we already have a feature flag to enable non-pot lane counts, the main issue imo is that someone could try to use Simd<T, 1234567> which should error for being too big. or Simd<T, 0> which is a pain therefore not allowed (though I'm ok with adding it for the consistency benefits).

[calebzulawski]

Also, removing the bounds from Simd but not impl Simd doesn't help us much, I think. In my experience, the bounds are no problem when writing code with "concrete" SIMD types, but painful when trying to do anything generic. If functions still require them, those generics will still be difficult.

we already have a feature flag to enable non-pot lane counts, the main issue imo is that someone could try to use Simd<T, 1234567> which should error for being too big. or Simd<T, 0> which is a pain therefore not allowed (though I'm ok with adding it for the consistency benefits).

There is precedent with [T; HUGE] erroring, so I think we may be able to do the same if we can remove the non-PoT bound as well.

[programmerjake]

we already have a feature flag to enable non-pot lane counts, the main issue imo is that someone could try to use Simd<T, 1234567> which should error for being too big. or Simd<T, 0> which is a pain therefore not allowed (though I'm ok with adding it for the consistency benefits).

the other issue I forgot is that imo non-pot lane counts have the wrong layout and that should change: #319

[scottmcm]

the main issue imo is that someone could try to use Simd<T, 1234567> which should error for being too big

This one I feel strongly isn't a problem, since [T; isize::MAX as usize] has exactly the same issue, and I don't think that being a non-local error has really ever been a problem. (I elaborate more on that train of thought in rust-lang/rust#104087 (comment) )

So long as using Simd like this gives the same kind of error as we get for arrays in https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=a8752e42c5601f574d948ff3677a7e0e, that seems entirely acceptable to me. (Saying that with my lang hat on, but not speaking for the team, since I don't think we've discussed Simd in particular.)

or Simd<T, 0> which is a pain therefore not allowed

I would love to have even a hacky solution to this. as_chunks is blocked on exactly the same thing (rust-lang/rust#74985).

lcnr had some thoughts on doing something like this in https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/How.20essential.20is.20the.20compile-time.20check.20for.20empty.20arrays.3F/near/248387965. Maybe someone could pick that up to deal with the zero problem for a while? If it was easier, Simd<T, 0> feels like another of those places where we might do "the type is well-formed but doesn't implement most of the traits".

but painful when trying to do anything generic

I suppose that's another possibility here -- explicitly decide to have v1 of stabilizing this not allow writing generic code over them, but still let people use specific versions when implementing stuff.

That seems like it'd still be pretty useful, since using f32x4 in the implementations of stuff is still much easier even if I can't be generic over stuff yet.

[programmerjake]

Also, removing the bounds from Simd but not impl Simd doesn't help us much, I think.

My proposal of using Simd::CHECK would only have type bounds, all lane count bounds would be post-mono errors only.