Skip to content

Conversation

PeterPtroc
Copy link

@PeterPtroc PeterPtroc commented Aug 26, 2025

Description of PR

  • Implements a CLMUL-based 16B fold + Barrett reduction algorithm, adapted from riscv-crc32-clmul.
  • True interleaved (round-robin) multi-block pipeline (1–3 blocks) to increase ILP.
  • Small buffers and tails fall back to the existing table-based software path.
  • Runtime gating:
    • Double-checked detection: “zbc” in /proc/cpuinfo AND a SIGILL-safe CLMUL probe.

How was this patch tested?

  • Build (native profile):
    • mvn -pl hadoop-common-project/hadoop-common -am -Pnative -DskipTests clean install
  • Run benchmark:
    • mvn -Pnative -DskipTests -Dexec.classpathScope=test -Dexec.mainClass=org.apache.hadoop.util.Crc32PerformanceTest

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 27m 44s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 44m 27s trunk passed
+1 💚 compile 18m 7s trunk passed
-1 ❌ mvnsite 1m 45s /branch-mvnsite-hadoop-common-project_hadoop-common.txt hadoop-common in trunk failed.
+1 💚 shadedclient 105m 59s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 10s the patch passed
+1 💚 compile 16m 23s the patch passed
+1 💚 cc 16m 23s the patch passed
+1 💚 golang 16m 23s the patch passed
+1 💚 javac 16m 23s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-1 ❌ mvnsite 1m 47s /patch-mvnsite-hadoop-common-project_hadoop-common.txt hadoop-common in the patch failed.
+1 💚 shadedclient 43m 12s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 24m 53s hadoop-common in the patch passed.
+1 💚 asflicense 1m 47s The patch does not generate ASF License warnings.
223m 28s
Subsystem Report/Notes
Docker ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7896/2/artifact/out/Dockerfile
GITHUB PR #7896
Optional Tests dupname asflicense compile cc mvnsite javac unit codespell detsecrets golang
uname Linux 4727479d09e6 5.15.0-144-generic #157-Ubuntu SMP Mon Jun 16 07:33:10 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 58baa6e
Default Java Red Hat, Inc.-1.8.0_312-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7896/2/testReport/
Max. process+thread count 3133 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7896/2/console
versions git=2.27.0 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 29m 24s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 46m 46s trunk passed
+1 💚 compile 17m 41s trunk passed
-1 ❌ mvnsite 1m 45s /branch-mvnsite-hadoop-common-project_hadoop-common.txt hadoop-common in trunk failed.
+1 💚 shadedclient 107m 55s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 12s the patch passed
+1 💚 compile 16m 38s the patch passed
+1 💚 cc 16m 38s the patch passed
+1 💚 golang 16m 38s the patch passed
+1 💚 javac 16m 38s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-1 ❌ mvnsite 1m 43s /patch-mvnsite-hadoop-common-project_hadoop-common.txt hadoop-common in the patch failed.
+1 💚 shadedclient 42m 59s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 24m 53s hadoop-common in the patch passed.
+1 💚 asflicense 1m 54s The patch does not generate ASF License warnings.
227m 14s
Subsystem Report/Notes
Docker ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7896/1/artifact/out/Dockerfile
GITHUB PR #7896
Optional Tests dupname asflicense compile cc mvnsite javac unit codespell detsecrets golang
uname Linux 262a84b27cca 5.15.0-144-generic #157-Ubuntu SMP Mon Jun 16 07:33:10 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 58baa6e
Default Java Red Hat, Inc.-1.8.0_312-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7896/1/testReport/
Max. process+thread count 3133 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7896/1/console
versions git=2.27.0 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants