Skip to content

Add Runtime dispatch based on custom CPU capabilities function #607

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 11 commits into
base: main
Choose a base branch
from
Draft

Add Runtime dispatch based on custom CPU capabilities function #607

wants to merge 11 commits into from

Conversation

@willieyz
Copy link
Contributor

@willieyz willieyz commented Nov 4, 2025 •
edited
Loading

This PR is still working in progress:

  • Ported:

    • Runtime dispatch mechanism for AArch64, x86 native, and FIPS202 APIs.
    • Configuration for static ON/OFF, AVX2 and AA64PFR1_EL1, as well as the corresponding config-variations tests in CI.

  • Not yet ported:

    • ec2_functests are not included in CI yet, since this section does not exist in mldsa-native. We have already opened Add ec2_functests in ci.yml #608 and will port it in a separate PR before finalizing this one.
      • PR 608 merged, enable in this PR.

  • There is currently a constant-time leak issue due to the runtime dispatch change in mld_poly_chknorm:

    mldsa-native/mldsa/src/poly.c

    Lines 589 to 604 in 05ad5ed

    MLD_INTERNAL_API
    uint32_t mld_poly_chknorm(const mld_poly *a, int32_t B)
    {
    unsigned int i;
    uint32_t t = 0;
    #if defined(MLD_USE_NATIVE_POLY_CHKNORM)
    int ret;
    /* TODO: proof */
    mld_assert_bound(a->coeffs, MLDSA_N, -REDUCE32_RANGE_MAX, REDUCE32_RANGE_MAX);
    ret = mld_poly_chknorm_native(a->coeffs, B);
    if (ret != MLD_NATIVE_FUNC_FALLBACK)
    {
    return ret == 0 ? 0 : 0xFFFFFFFF;
    }
    #endif /* MLD_USE_NATIVE_POLY_CHKNORM */
    mld_assert_bound(a->coeffs, MLDSA_N, -REDUCE32_RANGE_MAX, REDUCE32_RANGE_MAX);

    This has been discussed with @mkannwischer, and asking for help to add the fix.

@willieyz willieyz changed the title Runtime dispatch Add Runtime dispatch based on custom CPU capabilities function Nov 4, 2025
@willieyz willieyz force-pushed the runtime-dispatch branch 5 times, most recently from 6332a01 to b0ac532 Compare November 4, 2025 12:24
This was referenced Nov 4, 2025
Merged
Closed
@willieyz willieyz force-pushed the runtime-dispatch branch 22 times, most recently from 74996f5 to 3e7a817 Compare November 6, 2025 11:47
willieyz added a commit that referenced this pull request Nov 10, 2025
@willieyz
Add ec2_functests for further platforms test
e1848ec 
- This commit adds EC2 functional tests for mldsa-native,
  similar to what we have in mlkem-native.

- Unlike other functional tests that run on GitHub runners, the EC2
  tests provide a bit more flexibility and control over the actual CPU
  architecture being tested.

- During this work, we found that the CI did not raise an error when
  specifying a config_variations value that does not exist.
  For example, the previous commit passed several non-existent config
  headers from PR #607, which has not been merged yet. The CI silently
  accepted these missing configurations without any error.

- Since the configuration header from PR #607 is not yet merged,
  we temporarily comment out the config_variations parameter.
  Once PR #607 is ported, we will re-enable it.

- We also plan to add test ID validation,
  similar to mlkem-native PR #1291, to prevent false-positive test
  results caused by invalid configuration entries.

Signed-off-by: willieyz <[email protected]>
willieyz added a commit that referenced this pull request Nov 11, 2025
@willieyz
Add ec2_functests for further platforms test
debe391 
- This commit adds EC2 functional tests for mldsa-native,
  similar to what we have in mlkem-native.

- Unlike other functional tests that run on GitHub runners, the EC2
  tests provide a bit more flexibility and control over the actual CPU
  architecture being tested.

- During this work, we found that the CI did not raise an error when
  specifying a config_variations value that does not exist.
  For example, In this commit, We passed several non-existent config
  headers from PR #607, which has not been merged yet. The CI silently
  accepted these missing configurations without any error.

- Since the configuration header from PR #607 is not yet merged,
  we temporarily comment out the config_variations parameter.
  Once PR #607 is ported, we will re-enable it.

- We also plan to add test ID validation,
  similar to mlkem-native PR #1291, to prevent false-positive test
  results caused by invalid configuration entries.

Signed-off-by: willieyz <[email protected]>
willieyz added a commit that referenced this pull request Nov 11, 2025
@willieyz
Add ec2_functests for further platforms test
359f01a 
- This commit adds EC2 functional tests for mldsa-native,
  similar to what we have in mlkem-native.

- Unlike other functional tests that run on GitHub runners, the EC2
  tests provide a bit more flexibility and control over the actual CPU
  architecture being tested.

- During this work, we found that the CI did not raise an error when
  specifying a config_variations value that does not exist.
  For example, In this commit, We passed several non-existent config
  headers from PR #607, which has not been merged yet. The CI silently
  accepted these missing configurations without any error.

- Since the configuration header from PR #607 is not yet merged,
  we temporarily comment out the config_variations parameter.
  Once PR #607 is ported, we will re-enable it.

- We also plan to add test ID validation,
  similar to mlkem-native PR #1291, to prevent false-positive test
  results caused by invalid configuration entries.

Signed-off-by: willieyz <[email protected]>
mkannwischer pushed a commit that referenced this pull request Nov 11, 2025
@willieyz @mkannwischer
Add ec2_functests for further platforms test
4d057b1 
- This commit adds EC2 functional tests for mldsa-native,
  similar to what we have in mlkem-native.

- Unlike other functional tests that run on GitHub runners, the EC2
  tests provide a bit more flexibility and control over the actual CPU
  architecture being tested.

- During this work, we found that the CI did not raise an error when
  specifying a config_variations value that does not exist.
  For example, In this commit, We passed several non-existent config
  headers from PR #607, which has not been merged yet. The CI silently
  accepted these missing configurations without any error.

- Since the configuration header from PR #607 is not yet merged,
  we temporarily comment out the config_variations parameter.
  Once PR #607 is ported, we will re-enable it.

- We also plan to add test ID validation,
  similar to mlkem-native PR #1291, to prevent false-positive test
  results caused by invalid configuration entries.

Signed-off-by: willieyz <[email protected]>
@mkannwischer mkannwischer mentioned this pull request Nov 12, 2025
Merged
mkannwischer added a commit that referenced this pull request Nov 12, 2025
@mkannwischer
native chknorm: 0/0xFFFFFFFF -> 0/1
de492ff 
Previously the native chknorm functions would follow the C implementation,
i.e., return 0 if all coefficients are within bound and 0xFFFFFFFF otherwise.
This leads to problems with the run-time dispatch in
#607
as we want to use -1 to signal that the current platforms lacks the required
capabilities to run the native code, and we should fall back to the C
implementation.

This commit changes the backend API to return 1 in the failure mode.
This will allow to implement the run-time dispatch.
mkannwischer added a commit that referenced this pull request Nov 12, 2025
@mkannwischer
native chknorm: 0/0xFFFFFFFF -> 0/1
0492095 
Previously the native chknorm functions would follow the C implementation,
i.e., return 0 if all coefficients are within bound and 0xFFFFFFFF otherwise.
This leads to problems with the run-time dispatch in
#607
as we want to use -1 to signal that the current platforms lacks the required
capabilities to run the native code, and we should fall back to the C
implementation.

This commit changes the backend API to return 1 in the failure mode.
This will allow to implement the run-time dispatch.

Signed-off-by: Matthias J. Kannwischer <[email protected]>
mkannwischer added a commit that referenced this pull request Nov 12, 2025
@mkannwischer
native chknorm: 0/0xFFFFFFFF -> 0/1
472d696 
Previously the native chknorm functions would follow the C implementation,
i.e., return 0 if all coefficients are within bound and 0xFFFFFFFF otherwise.
This leads to problems with the run-time dispatch in
#607
as we want to use -1 to signal that the current platforms lacks the required
capabilities to run the native code, and we should fall back to the C
implementation.

This commit changes the backend API to return 1 in the failure mode.
This will allow to implement the run-time dispatch.

Signed-off-by: Matthias J. Kannwischer <[email protected]>
mkannwischer added a commit that referenced this pull request Nov 12, 2025
@mkannwischer
native chknorm: 0/0xFFFFFFFF -> 0/1
bbfce1f 
Previously the native chknorm functions would follow the C implementation,
i.e., return 0 if all coefficients are within bound and 0xFFFFFFFF otherwise.
This leads to problems with the run-time dispatch in
#607
as we want to use -1 to signal that the current platforms lacks the required
capabilities to run the native code, and we should fall back to the C
implementation.

This commit changes the backend API to return 1 in the failure mode.
This will allow to implement the run-time dispatch.

Signed-off-by: Matthias J. Kannwischer <[email protected]>
hanno-becker pushed a commit that referenced this pull request Nov 12, 2025
@mkannwischer @hanno-becker
native chknorm: 0/0xFFFFFFFF -> 0/1
87000d3 
Previously the native chknorm functions would follow the C implementation,
i.e., return 0 if all coefficients are within bound and 0xFFFFFFFF otherwise.
This leads to problems with the run-time dispatch in
#607
as we want to use -1 to signal that the current platforms lacks the required
capabilities to run the native code, and we should fall back to the C
implementation.

This commit changes the backend API to return 1 in the failure mode.
This will allow to implement the run-time dispatch.

Signed-off-by: Matthias J. Kannwischer <[email protected]>
hanno-becker pushed a commit that referenced this pull request Nov 12, 2025
@mkannwischer @hanno-becker
native chknorm: 0/0xFFFFFFFF -> 0/1
8cfc313 
Previously the native chknorm functions would follow the C implementation,
i.e., return 0 if all coefficients are within bound and 0xFFFFFFFF otherwise.
This leads to problems with the run-time dispatch in
#607
as we want to use -1 to signal that the current platforms lacks the required
capabilities to run the native code, and we should fall back to the C
implementation.

This commit changes the backend API to return 1 in the failure mode.
This will allow to implement the run-time dispatch.

Signed-off-by: Matthias J. Kannwischer <[email protected]>
hanno-becker pushed a commit that referenced this pull request Nov 12, 2025
@mkannwischer @hanno-becker
native chknorm: 0/0xFFFFFFFF -> 0/1
804a2b8 
Previously the native chknorm functions would follow the C implementation,
i.e., return 0 if all coefficients are within bound and 0xFFFFFFFF otherwise.
This leads to problems with the run-time dispatch in
#607
as we want to use -1 to signal that the current platforms lacks the required
capabilities to run the native code, and we should fall back to the C
implementation.

This commit changes the backend API to return 1 in the failure mode.
This will allow to implement the run-time dispatch.

Signed-off-by: Matthias J. Kannwischer <[email protected]>
@willieyz
Add runtime dispatch (mld_ntt_native, mld_intt_native, mld_poly_permu…
a8551d0 
…te_bitrev_to_custom)

- Change mld_ntt_native() return type from void to int
- Add runtime capability checking with fallback support
- Implement dispatch logic in mld_poly_ntt() to try native first,
  fallback to C
- Add MLD_NATIVE_FUNC_SUCCESS/FALLBACK return codes
- Add mld_sys_check_capability() for system capability detection
- Add test configuration for AVX2, static ON/OFF, add to CI test.

Signed-off-by: willieyz <[email protected]>
@willieyz
Add runtime dispatch (mld_rej_uniform_native, mld_rej_uniform_eta2/et…
b51f45a 
…a4_native)

Signed-off-by: willieyz <[email protected]>
@willieyz
Add runtime dispatch (mld_poly_decompose_32/88_native)
e2db90b 
Signed-off-by: willieyz <[email protected]>
@willieyz
Add runtime dispatch (mld_poly_caddq_native)
5b860f8 
Signed-off-by: willieyz <[email protected]>
@willieyz
Add runtime dispatch (mld_poly_use_hint_32/88_native)
93b7f1a 
Signed-off-by: willieyz <[email protected]>
@willieyz
Add runtime dispatch (mld_poly_chknorm)
ecf6816 
Signed-off-by: willieyz <[email protected]>
@willieyz
Add runtime dispatch (mld_polyz_unpack_17/19_native)
32dfef3 
Signed-off-by: willieyz <[email protected]>
@willieyz
Add runtime dispatch (mld_poly_pointwise_montgomery_native)
234878d 
Signed-off-by: willieyz <[email protected]>
@willieyz
Add runtime dispatch (mld_polyvecl_pointwise_acc_montgomery_l4/l5/l7_…
65c0e23 
…native)

Signed-off-by: willieyz <[email protected]>
@willieyz
Add runtime dispatch (FIPS202)
dc8208e 
Signed-off-by: willieyz <[email protected]>
@willieyz willieyz mentioned this pull request Nov 13, 2025
Open
willieyz added a commit that referenced this pull request Nov 14, 2025
@willieyz
Example: custom backend, re-run autogen
3de7ad4 
- This commit temporary add MLD_NATIVE_FUNC_SUCCESS to exception, this macro will introduced after PR #607,  remove after it merged.
- This commit re-run autogen

Signed-off-by: willieyz <[email protected]>
willieyz added a commit that referenced this pull request Nov 14, 2025
@willieyz
Example: custom backend, re-run autogen
aa5f31c 
- This commit temporary add MLD_NATIVE_FUNC_SUCCESS to exception, this
  macro will introduced after PR #607,  remove after it merged.
- This commit re-run autogen

Signed-off-by: willieyz <[email protected]>
willieyz added a commit that referenced this pull request Nov 14, 2025
@willieyz
Example: custom backend, re-run autogen
26ed2ee 
- This commit temporary add MLD_NATIVE_FUNC_SUCCESS to exception, this
  macro will introduced after PR #607,  remove after it merged.
- This commit re-run autogen

Signed-off-by: willieyz <[email protected]>
willieyz added a commit that referenced this pull request Nov 14, 2025
@willieyz
Example: custom backend, re-run autogen
d4f46ac 
- This commit temporary add MLD_NATIVE_FUNC_SUCCESS to exception, this
  macro will introduced after PR #607,  remove after it merged.
- This commit re-run autogen

Signed-off-by: willieyz <[email protected]>
@mkannwischer mkannwischer mentioned this pull request Nov 17, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Port run-time dispatch from mlkem-native

2 participants

@willieyz