HBASE-29787 Run Yetus General Checks as a GitHub Action (addendum) #7638

ndimiduk · 2026-01-15T16:44:44Z

Conditionally execute publishing to nightlies.

github-actions · 2026-01-15T16:50:33Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 18s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	yamllint	0m 0s		yamllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ master Compile Tests _
+1 💚	spotless	0m 52s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	spotless	0m 37s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 12s		The patch does not generate ASF License warnings.
		3m 18s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://nightlies.apache.org/hbase/HBase-PreCommit-GH-Actions-PR/PR-7638/60/yetus-general-check/output/Dockerfile
GITHUB PR	#7638
Optional Tests	dupname asflicense codespell detsecrets yamllint spotless
uname	Linux 3973db210c49 6.11.0-1018-azure #18~24.04.1-Ubuntu SMP Sat Jun 28 04:46:03 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `64dd0e7`
Max. process+thread count	42 (vs. ulimit of 30000)
modules	C: . U: .
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

ndimiduk · 2026-01-15T16:58:47Z

Context on https://issues.apache.org/jira/browse/INFRA-27571 . I'm not sure if this tweak will actually work without merging it, though maybe the PR from my external repo will teach us something.

ndimiduk · 2026-01-15T17:29:57Z

Hmm strange. Nothing at all from GHA on #7639

Apache9 · 2026-01-16T01:57:41Z

How do other projects which use GHA store the build result?

Apache9 · 2026-01-16T03:13:23Z

https://github.com/apache/dubbo/actions/runs/21034081969/job/60477885181?pr=16005

This project just uploads the artifacts to github.

https://github.com/apache/dubbo/blob/3.3/.github/workflows/build-and-test-pr.yml

name: "Upload test logs"
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: integration-test-logs-${{matrix.job_id}}-java${{matrix.java}}
          path: test/logs/*

Apache9 · 2026-01-16T03:18:14Z

OK, we also upload the yetus output to artifacts on github.

So what do we want to upload to nightlies? Can we also upload them to github artifacts?

ndimiduk · 2026-01-16T07:09:21Z

I started by uploading to GH artifacts, not using nightlies. The trouble is with the links out from the report on the comment -- they need a url for each plugin's output. GH artifacts does not support this. With multiple artifact files (we ship everything from the build) it only permits you to upload an archive and there's no way to deep link into the contents of that archive. So, no links. I built out nightlies support thinking that would be a solution.

To migrate to GHA, it appears that all failures need to be reported to the Checks API. We can use Annotations as well, but not all of our plugins support this (our spotless appears to be one such plugin). For all the plugins we want to run, we'll need to ensure that they report any details properly via annotation. Of course, a human can still download the entire test-patch archive bundle, but the need to know where to find it in the Actions run UI.

ndimiduk · 2026-01-16T07:59:19Z

Argh this means we don't have use of the hide-old-comments feature either. That will get annoying fast.

I'm going to remove reliance on secrets in this addendum. Let's looks more closely at what remains.

For hiding old comments, I guess we could implement a cron action. Let me add that and y'all can tell me what you think.

And let me see if it's a simple thing to add linecomment support to our spotless extension.

github-actions · 2026-01-16T08:18:08Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://github.com/apache/hbase/actions/runs/21060158924/ in case of problems.

github-actions · 2026-01-16T09:21:15Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 17s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+0 🆗	yamllint	0m 0s		yamllint was not available.
-1 ❌	@author	0m 0s	/results-author.txt	The patch appears to contain 1 @author tags which the community has agreed to not allow in code contributions.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+0 🆗	mvndep	0m 13s		Maven dependency ordering for branch
+1 💚	mvninstall	3m 20s		master passed
+1 💚	compile	9m 19s		master passed
+1 💚	checkstyle	2m 0s		master passed
+1 💚	spotbugs	8m 22s		master passed
+1 💚	spotless	0m 44s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 2s		the patch passed
+1 💚	compile	9m 12s		the patch passed
+1 💚	javac	9m 12s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 55s		the patch passed
+1 💚	shellcheck	0m 1s		No new issues.
+1 💚	spotbugs	8m 27s		the patch passed
+1 💚	hadoopcheck	9m 12s		Patch does not cause any errors with Hadoop 3.3.6 3.4.1.
-1 ❌	spotless	0m 7s	/patch-spotless.txt	patch has 34 errors when running spotless:check, run spotless:apply to fix.
			_ Other Tests _
-1 ❌	asflicense	0m 19s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		63m 10s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: /Dockerfile
GITHUB PR	#7638
Optional Tests	dupname asflicense codespell detsecrets shellcheck shelldocs spotless yamllint javac spotbugs checkstyle compile hadoopcheck hbaseanti
uname	Linux 0c435d04f0df 6.11.0-1018-azure #18~24.04.1-Ubuntu SMP Sat Jun 28 04:46:03 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `e3af448`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	161 (vs. ulimit of 30000)
modules	C: hbase-annotations . U: .
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3 shellcheck=0.8.0
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2026-01-16T16:06:34Z

I started by uploading to GH artifacts, not using nightlies. The trouble is with the links out from the report on the comment -- they need a url for each plugin's output. GH artifacts does not support this. With multiple artifact files (we ship everything from the build) it only permits you to upload an archive and there's no way to deep link into the contents of that archive. So, no links. I built out nightlies support thinking that would be a solution.

To migrate to GHA, it appears that all failures need to be reported to the Checks API. We can use Annotations as well, but not all of our plugins support this (our spotless appears to be one such plugin). For all the plugins we want to run, we'll need to ensure that they report any details properly via annotation. Of course, a human can still download the entire test-patch archive bundle, but the need to know where to find it in the Actions run UI.

OK, so the problem here is that, yetus has a --build-url-artifacts option which is used to as the prefix when rendering the yetus check result table, and for github actions, there is no such thing, and even if we upload the output files one by one to prevent them being zipped, we still can not have the url like "https://github.com/artifacts/spotless-output.txt"?

I guess we need to check how does yetus render the table...

github-actions · 2026-01-17T10:47:16Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://github.com/apache/hbase/actions/runs/21093050812/ in case of problems.

Apache-HBase · 2026-01-17T10:51:53Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7638/3/console in case of problems.

Apache-HBase · 2026-01-17T10:51:56Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7638/3/console in case of problems.

github-actions · 2026-01-17T11:49:32Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 25s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+0 🆗	yamllint	0m 0s		yamllint was not available.
-1 ❌	@author	0m 0s	/results-author.txt	The patch appears to contain 1 @author tags which the community has agreed to not allow in code contributions.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+0 🆗	mvndep	0m 12s		Maven dependency ordering for branch
+1 💚	mvninstall	3m 20s		master passed
+1 💚	compile	8m 57s		master passed
+1 💚	checkstyle	1m 53s		master passed
+1 💚	spotbugs	8m 18s		master passed
+1 💚	spotless	0m 46s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 1s		the patch passed
+1 💚	compile	8m 55s		the patch passed
+1 💚	javac	8m 55s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 57s		the patch passed
+1 💚	shellcheck	0m 1s		No new issues.
+1 💚	spotbugs	8m 16s		the patch passed
+1 💚	hadoopcheck	9m 13s		Patch does not cause any errors with Hadoop 3.3.6 3.4.1.
-1 ❌	spotless	0m 7s	/patch-spotless.txt	patch has 34 errors when running spotless:check, run spotless:apply to fix.
			_ Other Tests _
-1 ❌	asflicense	0m 19s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		62m 20s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: /Dockerfile
GITHUB PR	#7638
Optional Tests	dupname asflicense codespell detsecrets shellcheck shelldocs spotless yamllint javac spotbugs checkstyle compile hadoopcheck hbaseanti
uname	Linux 225f1f8d6c68 6.11.0-1018-azure #18~24.04.1-Ubuntu SMP Sat Jun 28 04:46:03 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f63c617`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	160 (vs. ulimit of 30000)
modules	C: hbase-annotations . U: .
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3 shellcheck=0.8.0
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

ndimiduk · 2026-01-17T11:55:06Z

I had to re-implement the function that generates the comment, but this one looks better at least.

Maybe this is all for nothing. The GHA isn't even writing a comment on #7639 . I think the compromise here is to extend Yetus to make each plugin's +1 as an explicit passing check.

@Apache9 @petersomogyi WDYT?

Apache-HBase · 2026-01-17T12:21:57Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 45s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+0 🆗	yamllint	0m 0s		yamllint was not available.
-1 ❌	@author	0m 0s	/results-author.txt	The patch appears to contain 1 @author tags which the community has agreed to not allow in code contributions.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+0 🆗	mvndep	0m 15s		Maven dependency ordering for branch
+1 💚	mvninstall	4m 25s		master passed
+1 💚	compile	11m 18s		master passed
+1 💚	checkstyle	2m 29s		master passed
+1 💚	spotbugs	11m 13s		master passed
+1 💚	spotless	1m 15s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s		Maven dependency ordering for patch
+1 💚	mvninstall	4m 53s		the patch passed
+1 💚	compile	11m 55s		the patch passed
+1 💚	javac	11m 55s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	2m 41s		the patch passed
+1 💚	shellcheck	0m 1s		No new issues.
+1 💚	spotbugs	13m 20s		the patch passed
+1 💚	hadoopcheck	15m 24s		Patch does not cause any errors with Hadoop 3.3.6 3.4.1.
-1 ❌	spotless	0m 17s	/patch-spotless.txt	patch has 34 errors when running spotless:check, run spotless:apply to fix.
			_ Other Tests _
-1 ❌	asflicense	0m 36s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		90m 46s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7638/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#7638
Optional Tests	dupname asflicense codespell detsecrets shellcheck shelldocs spotless yamllint javac spotbugs checkstyle compile hadoopcheck hbaseanti
uname	Linux 3e4c3578c247 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f63c617`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	190 (vs. ulimit of 30000)
modules	C: hbase-annotations . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7638/3/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3 shellcheck=0.8.0
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2026-01-17T15:30:00Z

For me I think the current solution is acceptable, anyway I can download the artifacts and check the result. It will be better if we can add a line in the footer of yetus comment to point to the artifacts with a url(Can be a separated issue).

And since now we can run general checks, it will be easier for us to also run the JDK17/11/8 hadoop2/3 checks too?

In this way, we can remove the pre commit jenkins job then.

Thanks.

Apache9 · 2026-01-17T15:31:25Z

hbase-annotations/src/test/java/org/apache/hadoop/hbase/testclassification/ClientTests.java

@@ -1,25 +1,9 @@
-/*


Remember to revert the changes to this file before merging :)

ndimiduk · 2026-01-17T15:33:08Z

I just left a note on the dev list to see if anyone else has other considerations. But yes, I think this is now working as designed -- no comment and only Checks. We can disable the Jenkins PR runner and start to explore converting other checks. Let me also remove this comment cleaning timer.

Apache-HBase · 2026-01-17T16:46:41Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 42s		Docker mode activated.
-0 ⚠️	yetus	0m 2s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+0 🆗	mvndep	0m 14s		Maven dependency ordering for branch
+1 💚	mvninstall	4m 22s		master passed
+1 💚	compile	3m 15s		master passed
+1 💚	javadoc	3m 5s		master passed
+1 💚	shadedjars	7m 22s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 12s		Maven dependency ordering for patch
+1 💚	mvninstall	4m 14s		the patch passed
+1 💚	compile	3m 6s		the patch passed
+1 💚	javac	3m 6s		the patch passed
+1 💚	javadoc	4m 3s		the patch passed
-1 ❌	shadedjars	0m 12s		patch has 10 errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 ❌	unit	316m 40s	/patch-unit-root.txt	root in the patch failed.
		355m 30s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7638/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#7638
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 369a1d6d2a15 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f63c617`
Default Java	Eclipse Adoptium-17.0.11+9
shadedjars	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7638/3/artifact/yetus-jdk17-hadoop3-check/output/patch-shadedjars.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7638/3/testReport/
Max. process+thread count	8317 (vs. ulimit of 30000)
modules	C: hbase-annotations . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7638/3/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

HBASE-29787 Run Yetus General Checks as a GitHub Action (addendum)

64dd0e7

Conditionally execute publishing to nightlies.

ndimiduk requested review from Apache9 and petersomogyi January 15, 2026 16:44

ndimiduk mentioned this pull request Jan 15, 2026

HBASE-29787 Run Yetus General Checks as a GitHub Action (addendum) #7639

Draft

This comment has been minimized.

Sign in to view

ndimiduk added 3 commits January 16, 2026 09:12

remove dependency on secrets

32e1164

add a workflow to periodically hide old comments

c9ecc65

add support for line comments to our spotless extension

a78a5ff

This comment has been minimized.

Sign in to view

ndimiduk added 2 commits January 17, 2026 11:45

override how generate the comment, only make links when we have a url

6b26e86

introduce some breakages

f63c617

ndimiduk force-pushed the 29787-addendum-master branch from e3af448 to f63c617 Compare January 17, 2026 10:46

Apache9 approved these changes Jan 17, 2026

View reviewed changes

HBASE-29787 Run Yetus General Checks as a GitHub Action (addendum) #7638

Are you sure you want to change the base?

HBASE-29787 Run Yetus General Checks as a GitHub Action (addendum) #7638

Conversation

ndimiduk commented Jan 15, 2026

Uh oh!

github-actions bot commented Jan 15, 2026

Uh oh!

ndimiduk commented Jan 15, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

ndimiduk commented Jan 15, 2026

Uh oh!

Apache9 commented Jan 16, 2026

Uh oh!

Apache9 commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Apache9 commented Jan 16, 2026

Uh oh!

ndimiduk commented Jan 16, 2026

Uh oh!

ndimiduk commented Jan 16, 2026

Uh oh!

github-actions bot commented Jan 16, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Jan 16, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

Apache9 commented Jan 16, 2026

Uh oh!

github-actions bot commented Jan 17, 2026

Uh oh!

Apache-HBase commented Jan 17, 2026

Uh oh!

Apache-HBase commented Jan 17, 2026

Uh oh!

github-actions bot commented Jan 17, 2026

Uh oh!

ndimiduk commented Jan 17, 2026

Uh oh!

Apache-HBase commented Jan 17, 2026

Uh oh!

Apache9 commented Jan 17, 2026

Uh oh!

Apache9 Jan 17, 2026

Choose a reason for hiding this comment

Uh oh!

ndimiduk commented Jan 17, 2026

Uh oh!

Apache-HBase commented Jan 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Apache9 commented Jan 16, 2026 •

edited

Loading