Skip to content

HBASE-29787 Run Yetus General Checks as a GitHub Action (addendum) #7639

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ndimiduk wants to merge 2 commits into from
+23 −69
Closed

HBASE-29787 Run Yetus General Checks as a GitHub Action (addendum) #7639

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
28dff30
HBASE-29787 Run Yetus General Checks as a GitHub Action (addendum)
ndimiduk Jan 15, 2026
751d6bf
pretty sure we're not using these permissions anymore
ndimiduk Jan 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 0 additions & 40 deletions .github/workflows/yetus-general-check.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,7 @@ name: Yetus General Check
types: [opened, synchronize, reopened]

permissions:
checks: write
contents: read
pull-requests: write
statuses: write

jobs:
Expand Down Expand Up @@ -95,44 +93,6 @@ jobs:
cd "${{ github.workspace }}"
bash src/dev-support/jenkins_precommit_github_yetus.sh

- name: Hide Old Comments
if: always()
env:
DEBUG: 'true'
run: |
# github-actions[bot] has a known user ID: 41898282
# Verify: curl -s "https://api.github.com/users/github-actions%5Bbot%5D" | jq .id
export DEBUG="${DEBUG:-false}"
export GITHUB_TOKEN="${{ secrets.GITHUB_TOKEN }}"
export BUILD_BOT_USER_ID="41898282"
export JOB_NAME="HBase-PreCommit-GH-Actions-PR"
export REPO="${{ github.repository }}"
bash src/dev-support/gh_hide_old_comments.sh ${{ github.event.pull_request.number }} || true

- name: Create Nightlies Directory Structure
# this version of rsync doesn't support --mkdirs so we do this step manually
if: always()
run: |
eval $(ssh-agent -s)
ssh-add - <<< "${{ secrets.NIGHTLIES_RSYNC_KEY }}"
ssh -p ${{ secrets.NIGHTLIES_RSYNC_PORT }} \
-o StrictHostKeyChecking=no \
${{ secrets.NIGHTLIES_RSYNC_USER }}@${{ secrets.NIGHTLIES_RSYNC_HOST }} \
"mkdir -p ${{ secrets.NIGHTLIES_RSYNC_PATH }}/hbase/HBase-PreCommit-GH-Actions-PR/PR-${{ github.event.pull_request.number }}/${{ github.run_number }}"
ssh-agent -k

- name: Publish to Nightlies
if: always()
uses: burnett01/rsync-deployments@0dc935cdecc5f5e571865e60d2a6cdc673704823
with:
switches: -avzr --no-o
path: yetus-general-check
remote_path: ${{ secrets.NIGHTLIES_RSYNC_PATH }}/hbase/HBase-PreCommit-GH-Actions-PR/PR-${{ github.event.pull_request.number }}/${{ github.run_number }}/
remote_host: ${{ secrets.NIGHTLIES_RSYNC_HOST }}
remote_port: ${{ secrets.NIGHTLIES_RSYNC_PORT }}
remote_user: ${{ secrets.NIGHTLIES_RSYNC_USER }}
remote_key: ${{ secrets.NIGHTLIES_RSYNC_KEY }}

- name: Publish Test Results
if: always()
uses: actions/upload-artifact@v4
Expand Down
42 changes: 15 additions & 27 deletions dev-support/hbase-personality.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,20 +88,6 @@ function personality_globals
# See HBASE-19902 for how we arrived at 20g.
#shellcheck disable=SC2034
DOCKERMEMLIMIT=20g

# Override BUILD_URL for GitHub Actions to point to nightlies
# instead of the GitHub Actions run page
# The robot sets BUILD_URL to the GHA run page, but we want nightlies URLs
if [[ "${GITHUB_ACTIONS}" == "true" ]] && [[ "${GITHUB_EVENT_NAME}" == "pull_request" ]]; then
local pr_number
# GITHUB_REF is a standard GitHub Actions environment variable
# shellcheck disable=SC2153
pr_number=$(echo "${GITHUB_REF}" | cut -f3 -d/)
#shellcheck disable=SC2034
BUILD_URL="https://nightlies.apache.org/hbase/HBase-PreCommit-GH-Actions-PR/PR-${pr_number}/${GITHUB_RUN_NUMBER}/"
#shellcheck disable=SC2034
BUILD_URL_ARTIFACTS="yetus-general-check/output"
fi
}

## @description Parse extra arguments required by personalities, if any.
Expand Down Expand Up @@ -849,19 +835,6 @@ function hbaseanti_patchfile

add_test_type spotless

## @description Build artifact URL for GitHub Actions
## @audience private
## @stability evolving
## @replaceable no
if ! declare -f githubactions_artifact_url >/dev/null; then
function githubactions_artifact_url
{
if [[ -n "${BUILD_URL}" ]] && [[ -n "${BUILD_URL_ARTIFACTS}" ]]; then
echo "${BUILD_URL}${BUILD_URL_ARTIFACTS}"
fi
}
fi

## @description spotless file filter
## @audience private
## @stability evolving
Expand All @@ -879,6 +852,7 @@ function spotless_rebuild
{
local repostatus=$1
local logfile="${PATCH_DIR}/${repostatus}-spotless.txt"
local linecommentsfile="${PATCH_DIR}/${repostatus}-spotless-linecomments.txt"

if ! verify_needed_test spotless; then
return 0
Expand All @@ -896,6 +870,20 @@ function spotless_rebuild

count=$(${GREP} -c '\[ERROR\]' "${logfile}")
if [[ ${count} -gt 0 ]]; then
# Generate file-level annotations for GitHub Actions
if [[ -n "${BUGLINECOMMENTS}" ]]; then
# Extract files with violations: lines like "[ERROR] src/path/to/file.java"
# with leading whitespace after [ERROR]
${GREP} '^\[ERROR\][[:space:]]\+[^[:space:]]' "${logfile}" \
| ${SED} 's/^\[ERROR\][[:space:]]*//g' \
| while read -r file; do
echo "${file}:1:Spotless formatting required, run mvn spotless:apply"
done > "${linecommentsfile}"
if [[ -s "${linecommentsfile}" ]]; then
bugsystem_linecomments_queue spotless "${linecommentsfile}"
fi
fi

add_vote_table_v2 -1 spotless \
"@@BASE@@/${repostatus}-spotless.txt" \
"${repostatus} has ${count} errors when running spotless:check, run spotless:apply to fix."
Expand Down
10 changes: 8 additions & 2 deletions dev-support/jenkins_precommit_github_yetus.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,11 @@ YETUS_ARGS+=("--console-report-file=${PATCHDIR}/console.txt")
YETUS_ARGS+=("--html-report-file=${PATCHDIR}/report.html")
# enable writing back to Github
YETUS_ARGS+=("--github-token=${GITHUB_PASSWORD}")
YETUS_ARGS+=("--github-write-comment")
# GitHub Actions fork PRs cannot write comments (GITHUB_TOKEN has no PR write permission)
# Jenkins can write comments via its own credentials
if [[ "${GITHUB_ACTIONS}" != "true" ]]; then
YETUS_ARGS+=("--github-write-comment")
fi
# auto-kill any surefire stragglers during unit test runs
YETUS_ARGS+=("--reapermode=kill")
# set relatively high limits for ASF machines
Expand All @@ -108,7 +112,9 @@ YETUS_ARGS+=("--spotbugs-strict-precheck")
# rsync these files back into the archive dir
YETUS_ARGS+=("--archive-list=${ARCHIVE_PATTERN_LIST}")
# URL for user-side presentation in reports and such to our artifacts
YETUS_ARGS+=("--build-url-artifacts=${BUILD_URL_ARTIFACTS}")
if [[ -n "${BUILD_URL_ARTIFACTS}" ]]; then
YETUS_ARGS+=("--build-url-artifacts=${BUILD_URL_ARTIFACTS}")
fi
# plugins to enable
YETUS_ARGS+=("--plugins=${PLUGINS},-findbugs")
# run in docker mode and specifically point to our
Expand Down