-
Notifications
You must be signed in to change notification settings - Fork 4k
Add BCR publishing automation with workflow and configuration #12555
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,195 @@ | ||
| # Bazel Central Registry (BCR) Publishing for grpc-java | ||
|
|
||
| This document explains how grpc-java releases are published to the [Bazel Central Registry (BCR)](https://github.com/bazelbuild/bazel-central-registry). | ||
|
|
||
| ## Overview | ||
|
|
||
| The BCR is the official registry for Bazel modules, allowing users to depend on grpc-java using Bazel's module system (bzlmod). When a new version of grpc-java is released, it is automatically published to the BCR via a GitHub Actions workflow. | ||
|
|
||
| ## How It Works | ||
|
|
||
| ### Automated Publishing Workflow | ||
|
|
||
| The publishing process is automated through the GitHub Actions workflow located at `.github/workflows/publish-to-bcr.yml`. This workflow: | ||
|
|
||
| 1. **Triggers automatically** when a new release is published on GitHub | ||
| 2. **Can be manually triggered** via workflow_dispatch for re-runs or troubleshooting | ||
| 3. **Uses the official publish-to-bcr action** from bazel-contrib | ||
| 4. **Generates attestations** for release provenance and security | ||
| 5. **Creates a pull request** directly to the official Bazel Central Registry (https://github.com/bazelbuild/bazel-central-registry) with the new version | ||
|
|
||
| ### BCR Configuration Files | ||
|
|
||
| The BCR configuration is stored in the `.bcr/` directory: | ||
|
|
||
| - **`metadata.template.json`**: Contains module metadata including homepage, maintainers, and repository information | ||
| - **`source.template.json`**: Specifies how to download the source archive (uses GitHub release tags) | ||
| - **`presubmit.yml`**: Defines CI tests that run in the BCR before accepting the new version | ||
|
|
||
| ### Presubmit Testing | ||
|
|
||
| Before a new version is accepted into the BCR, it must pass presubmit tests defined in `.bcr/presubmit.yml`. These tests: | ||
|
|
||
| - **Build key targets** from the grpc-java module to ensure basic functionality | ||
| - **Test the examples module** which demonstrates using grpc-java as a dependency | ||
| - **Run on multiple platforms**: Debian, macOS, Ubuntu, Windows | ||
| - **Test with multiple Bazel versions**: 7.x and 8.x | ||
|
|
||
| The examples directory serves as a test module that uses `local_path_override` to depend on the main grpc-java module, providing comprehensive integration testing. | ||
|
|
||
| ## Setup and Configuration | ||
|
|
||
| ### For Repository Maintainers | ||
|
|
||
| To enable BCR publishing for grpc-java, follow these steps: | ||
|
|
||
| #### 1. Create a Personal Access Token (PAT) | ||
|
|
||
| A GitHub Personal Access Token is required to create pull requests to the Bazel Central Registry. | ||
|
|
||
| **Creating a Classic PAT:** | ||
|
|
||
| 1. Go to GitHub Settings → Developer settings → Personal access tokens → Tokens (classic) | ||
| 2. Click "Generate new token (classic)" | ||
| 3. Give it a descriptive name like "grpc-java BCR Publishing" | ||
| 4. Select the following scopes: | ||
| - `repo` (Full control of private repositories) | ||
| - `workflow` (Update GitHub Action workflows) | ||
| 5. Set an appropriate expiration (consider using no expiration for automation) | ||
| 6. Click "Generate token" and **save the token securely** | ||
|
|
||
| **Note about Fine-grained PATs:** | ||
| Fine-grained tokens are not fully supported yet because they cannot open pull requests against public repositories (see [GitHub roadmap issue #600](https://github.com/github/roadmap/issues/600)). | ||
|
|
||
| #### 2. Configure Repository Secrets | ||
|
|
||
| 1. Go to the grpc-java repository Settings → Secrets and variables → Actions | ||
| 2. Click "New repository secret" | ||
| 3. Name: `BCR_PUBLISH_TOKEN` | ||
| 4. Value: Paste the PAT you created | ||
| 5. Click "Add secret" | ||
|
|
||
| #### 3. Verify Workflow Permissions | ||
|
|
||
| Ensure the workflow has the necessary permissions in the repository settings: | ||
|
|
||
| - Go to Settings → Actions → General → Workflow permissions | ||
| - Ensure "Read and write permissions" is selected | ||
| - Ensure "Allow GitHub Actions to create and approve pull requests" is checked | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure if this permission is granted and I don't have access to the repository options in Settings. Adding Eric as reviewer who has that access.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In no way do we want to give this write permissions to grpc-java. If that's required, then we will simply continue not publishing to BCR. That's not okay, as we try to give almost nothing write access to our repository. It would be fine to give it write permissions to a repo like https://github.com/grpc/bazel-central-registry |
||
|
|
||
| ## Publishing a New Release | ||
|
|
||
| ### Automatic Publishing | ||
|
|
||
| When you create a new release on GitHub: | ||
|
|
||
| 1. Go to the grpc-java repository | ||
| 2. Create a new release with a tag (e.g., `v1.76.0`) | ||
| 3. The `publish-to-bcr.yml` workflow will automatically trigger | ||
| 4. Monitor the workflow run in the Actions tab | ||
| 5. If successful, a PR will be created in the BCR repository | ||
|
|
||
| ### Manual Publishing | ||
|
|
||
| If you need to manually trigger the publishing workflow: | ||
|
|
||
| 1. Go to Actions → Publish to BCR | ||
| 2. Click "Run workflow" | ||
| 3. Enter the tag name (e.g., `v1.76.0`) | ||
| 4. Click "Run workflow" | ||
|
|
||
| ### Monitoring the Workflow | ||
|
|
||
| 1. Go to the Actions tab in the grpc-java repository | ||
| 2. Look for the "Publish to BCR" workflow run | ||
| 3. Check the logs for any errors or issues | ||
| 4. If successful, you'll see a link to the created BCR pull request | ||
|
|
||
| ## BCR Pull Request Review | ||
|
|
||
| After the workflow creates a PR in the BCR: | ||
|
|
||
| 1. The PR will be automatically assigned to the module maintainers | ||
| 2. BCR CI will run presubmit tests to validate the new version | ||
| 3. Maintainers should review and approve the PR | ||
| 4. Once approved and tests pass, the BCR team will merge the PR | ||
| 5. The new version becomes available in the BCR | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### Common Issues | ||
|
|
||
| **Workflow fails with authentication error:** | ||
| - Verify the `BCR_PUBLISH_TOKEN` secret is correctly set | ||
| - Ensure the PAT has not expired | ||
| - Check that the PAT has the required `repo` and `workflow` scopes | ||
|
|
||
| **Presubmit tests fail in BCR:** | ||
| - Review the test logs in the BCR PR | ||
| - Ensure the MODULE.bazel file is correctly formatted | ||
| - Verify all dependencies are available in BCR | ||
| - Test locally using the BCR's local testing instructions | ||
|
|
||
| **PR creation fails:** | ||
| - Verify the registry fork exists and is accessible | ||
| - Ensure the PAT has permissions for the fork repository | ||
| - Check that the workflow has correct permissions | ||
|
|
||
| **Version already exists:** | ||
| - BCR is add-only; you cannot replace existing versions | ||
| - If a fix is needed, it should be fixed upstream and a new version submitted | ||
| - If the fix is only in BCR patches (nothing to fix upstream), use the `.bcr.N` suffix format (e.g., `1.75.0.bcr.1`) | ||
|
|
||
| ### Getting Help | ||
|
|
||
| - Check the [BCR documentation](https://github.com/bazelbuild/bazel-central-registry/blob/main/docs/README.md) | ||
| - Review the [publish-to-bcr documentation](https://github.com/bazel-contrib/publish-to-bcr) | ||
| - Contact BCR maintainers at [email protected] | ||
| - File issues in the appropriate repository | ||
|
|
||
| ## Maintainers | ||
|
|
||
| The following people are listed as maintainers for grpc-java in the BCR and will be notified of new PRs: | ||
|
|
||
| - Eric Anderson (@ejona86) | ||
| - Abhishek Agrawal (@AgraVator) | ||
| - Kannan J (@kannanjgithub) | ||
| - MV Shiva (@shivaspeaks) | ||
|
|
||
| Maintainers have approval rights for BCR PRs related to grpc-java, even without direct write access to the BCR repository. | ||
|
|
||
| ## Patches | ||
|
|
||
| The BCR may include patches to the grpc-java module to ensure compatibility with the rest of the BCR ecosystem. These patches are typically applied by BCR maintainers and may include: | ||
|
|
||
| - Updating dependency versions to use BCR-compatible versions | ||
| - Adding missing dependencies required for Bazel module builds | ||
| - Adjusting build configurations for BCR compatibility | ||
|
|
||
| **Note:** If you need to include custom patches with your release, you can add them to the `.bcr/patches/` directory in the grpc-java repository. All patches must: | ||
| - Have the `.patch` extension | ||
| - Be in the `-p1` format (apply from the repository root) | ||
| - Be listed in the BCR entry's `source.json` file (automatically handled by the publish-to-bcr workflow) | ||
|
|
||
| The publish-to-bcr workflow will automatically include any patches found in `.bcr/patches/` when creating the BCR entry. | ||
|
|
||
| ## Module Structure | ||
|
|
||
| grpc-java is published as a single module to the BCR: | ||
|
|
||
| - **Main module**: The root `MODULE.bazel` defines the grpc-java module | ||
| - **Test module**: The `examples/` directory serves as a test module during presubmit | ||
|
|
||
| If additional modules need to be published from this repository in the future, they can be configured using the `moduleRoots` field in `.bcr/config.yml`. | ||
|
|
||
| ## References | ||
|
|
||
| - [Bazel Central Registry](https://github.com/bazelbuild/bazel-central-registry) | ||
| - [BCR Contribution Guidelines](https://github.com/bazelbuild/bazel-central-registry/blob/main/docs/README.md) | ||
| - [publish-to-bcr GitHub Action](https://github.com/bazel-contrib/publish-to-bcr) | ||
| - [Bzlmod User Guide](https://bazel.build/docs/bzlmod) | ||
| - [grpc-java in BCR](https://github.com/bazelbuild/bazel-central-registry/tree/main/modules/grpc-java) | ||
|
|
||
| ## License | ||
|
|
||
| This documentation and the BCR configuration files are part of grpc-java and are subject to the same Apache 2.0 license. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| { | ||
| "homepage": "https://github.com/grpc/grpc-java", | ||
| "maintainers": [ | ||
| { | ||
| "name": "Eric Anderson", | ||
| "email": "[email protected]", | ||
| "github": "ejona86", | ||
| "github_user_id": 2811396 | ||
| }, | ||
| { | ||
| "name": "Abhishek Agrawal", | ||
| "github": "AgraVator", | ||
| "github_user_id": 81427947 | ||
| }, | ||
| { | ||
| "name": "Kannan J", | ||
| "github": "kannanjgithub", | ||
| "github_user_id": 29600796 | ||
| }, | ||
| { | ||
| "name": "MV Shiva", | ||
| "github": "shivaspeaks", | ||
| "github_user_id": 56782215 | ||
| } | ||
| ], | ||
| "repository": [ | ||
| "github:grpc/grpc-java" | ||
| ], | ||
| "versions": [], | ||
| "yanked_versions": {} | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| # BCR presubmit configuration for grpc-java | ||
| # Tests the main module and the examples test module | ||
|
|
||
| # Test the main grpc-java module | ||
| matrix: | ||
| platform: ["debian10", "macos", "ubuntu2004", "windows"] | ||
| bazel: ["7.x", "8.x"] | ||
| tasks: | ||
| verify_targets: | ||
| platform: ${{ platform }} | ||
| bazel: ${{ bazel }} | ||
| build_targets: | ||
| - '@grpc-java//:java_grpc_library__external_repo_test' | ||
|
|
||
| # Test the examples module which uses grpc-java as a dependency | ||
| bcr_test_module: | ||
| module_path: "examples" | ||
| matrix: | ||
| platform: ["debian10", "macos", "ubuntu2004", "windows"] | ||
| bazel: ["7.x", "8.x"] | ||
| tasks: | ||
| run_test_module: | ||
| name: "Run examples test module" | ||
| platform: ${{ platform }} | ||
| bazel: ${{ bazel }} | ||
| build_targets: | ||
| - '//src/main/java/io/grpc/examples/helloworld:hello-world-client' | ||
| - '//src/main/java/io/grpc/examples/helloworld:hello-world-server' |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| { | ||
| "integrity": "", | ||
| "strip_prefix": "grpc-java-{VERSION}", | ||
| "url": "https://github.com/grpc/grpc-java/archive/refs/tags/v{VERSION}.tar.gz" | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| name: Publish to BCR | ||
|
|
||
| # This workflow publishes new releases to the Bazel Central Registry (BCR) | ||
| # It is triggered automatically when a new release is created | ||
|
|
||
| on: | ||
| # Automatically run when a release is published | ||
| release: | ||
| types: [published] | ||
|
|
||
| # Allow manual triggering for re-runs or troubleshooting | ||
| workflow_dispatch: | ||
| inputs: | ||
| tag_name: | ||
| description: 'Tag name to publish (e.g., v1.75.0)' | ||
| required: true | ||
| type: string | ||
|
|
||
| jobs: | ||
| publish: | ||
| name: Publish to Bazel Central Registry | ||
| # Use the reusable workflow from publish-to-bcr | ||
| uses: bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml@v6 | ||
| with: | ||
| tag_name: ${{ github.event_name == 'release' && github.event.release.tag_name || inputs.tag_name }} | ||
| # Repository to submit the PR to (using official BCR) | ||
| # You can also use a fork by specifying: your-org/bazel-central-registry | ||
| registry_fork: bazelbuild/bazel-central-registry | ||
| # Generate attestations for provenance | ||
| attest: true | ||
| # Required permissions for the workflow | ||
| permissions: | ||
| contents: write | ||
| # Required for generating attestations | ||
| id-token: write | ||
| attestations: write | ||
| secrets: | ||
| # Personal Access Token with 'repo' and 'workflow' permissions | ||
| # Must be stored as a repository secret named BCR_PUBLISH_TOKEN | ||
| publish_token: ${{ secrets.BCR_PUBLISH_TOKEN }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have added a new repository secret with this name.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't want to do that. Your account has permissions to lots of things we don't want this to have access to. We can try to purpose one of our robot accounts for this, although right now I think they have more access then we'd want as well. We need to determine a way to give it write access to only what it needs.