Merge pull request #14904 from jdaugherty/7.0.x

fix: issues found by M5 release verification

Author: jdaugherty

INSTALL

@@ -30,6 +30,33 @@ Project Structure
 ------------------------------
 The source of Grails is a multi-project Gradle build that uses composite multi-project builds via Gradle's includeBuild feature. The main project is `grails-core`, which contains the core framework code and all libraries that an end user would use in their application. The `grails-gradle` project contains code meant to run on the gradle build classpath.  `grails-forge` contains tooling related to App Generation.
 
+Building a Release Version
+------------------------------
+The source distribution will always be a release version of Grails - this means the version number will not contain the suffix `SNAPSHOT` and the release requirements will apply by default. One of those requirements is jar file signing. In order for forge to always correctly pull the `grails-core` project, a local publishing strategy is used - the jar files are published to a directory inside of the build directory of `grails-core`.  This publishing means building the jar files will always trigger the signing process. To successfully build the source distribution, either signing must be disabled or signing must be correctly configured.
+
+Disabling Signing
+------------------------------
+To disable signing, set the environment variable `GRAILS_PUBLISH_RELEASE` to `false`. This can be done by running the following command in the terminal:
+
+        export GRAILS_PUBLISH_RELEASE=false
+
+Enabling Signing
+------------------------------
+Grails makes use of the Grails Publish Gradle plugin to handle publishing artifacts. This plugin supports signing either via native tooling or via java libraries.
+
+To configure the native tooling, gpg must be installed with a key imported. Then the following must be specified:
+
+        export SIGNING_KEY=<your-gpg-key-id>
+        export SIGNING_PASSPHRASE=<your-gpg-key-passphrase>
+
+To configure the java tooling, the following must be specified:
+
+        export SIGNING_KEY=<your-gpg-key-id>
+        export SIGNING_KEYRING=<path-to-your-gpg-keyring-file>
+        export SIGNING_PASSPHRASE=<your-gpg-key-passphrase>
+
+Please note: if no GPG Key passphrase is set, then simply leave the environment variable `SIGNING_PASSPHRASE` unset.
+
 Building grails-core
 ------------------------------
 To build the libraries a Grails Application would use, run the following command at the source root:
@@ -58,10 +85,6 @@ To use the built libraries in a Grails Application, libraries must be published
         ./gradlew build publishToMavenLocal -PskipTests
         cd grails-forge && ./gradlew build publishToMavenLocal -PskipTests
 
-Please note that this will require signing with the GPG command since releases are mandated to be signed. To disable signing, set the environment variable to disable release detection:
-
-        export GRAILS_PUBLISH_RELEASE=false
-
 Creating an Application
 ------------------------------
 By default, the Grails CLI uses standard maven repos to retrieve upstream Grails libraries.  However, it's possible to "override" this behavior by setting the environment variable `GRAILS_REPO_URL` to an alternative location.  When wanting to use the libraries generated by a source distribution, set it to your local Maven repository, which is typically `~/.m2/repository`.  This can be done by running the following command in the terminal:

README.md

@@ -58,7 +58,8 @@ CLI applications exist to assist in Application generation. Instructions for the
 
 The Apache Grails Wrapper is a tiny distribution (25KB) that can manage larger sized CLIs for Grails. It consists of a
 `grailsw` shell script, a `grailsw.bat` batch script, and the jar file `grails-wrapper.jar`. It can be downloaded from
-the latest [GitHub Release](https://github.com/apache/grails-core/releases) page starting with 7.0.0-M4 & it is included
+the latest [GitHub Release](https://github.com/apache/grails-core/releases) page starting with version 7.0.0-M4 & it is
+included
 in any created project. The wrapper is used to either create an Apache Grails Application or to run commands in an
 existing Grails Application directory. The wrapper is generally meant to be forward compatible and downloads the Apache
 Grails CLIs to the directory `$HOME/.grails/wrapper`.
@@ -95,7 +96,7 @@ release page. You can use the command `./grails-forge-cli --help` to see what's
 ### SDKMAN
 
 If managing multiple, local copies of the Grails CLI, it is recommended to use [SDKMAN!](https://sdkman.io/). Assuming
-SDKMAN is installed, this command would install the 7.0.0-M4 version:
+SDKMAN is installed, this command would install the `7.0.0-M4` version:
 
      sdk install grails 7.0.0-M4
 

RELEASE.md

@@ -73,11 +73,13 @@ For Example:
     verify.sh v7.0.0-M4 /tmp/grails-verify
 ```
 
-### Download the Staged Artifacts
+Otherwise, you may follow the subsequent steps in this section to verify each step individually.
+
+### Manual Verification: Download the Staged Artifacts
 
 Use `etc/bin/download-release-artifacts.sh` to download the staged artifacts. This script will download the source distribution, wrapper binary distribution, and sdkman binary distribution. The distribution should come from [https://dist.apache.org/repos/dist/dev/incubator/grails/core/version](https://dist.apache.org/repos/dist/dev/incubator/grails/core).
 
-### Source Distribution Verification
+### Manual Verification: Source Distribution Verification
 
 The following are the source distribution artifacts:
    * `apache-grails-<version>-incubating-src.zip` - the source distribution
@@ -102,7 +104,7 @@ Extracts the zip file and verifies the contents:
    * Ensure the `PUBLISHED_ARTIFACTS` file is present so we know how to pull the various jar files.
    * Ensure the `CHECKSUMS` file is present so we can ensure those checksums match the staged artifacts.
 
-### Jar file Signature Verification (Nexus Staging Repositories)
+### Manual Verification: Jar file Signature Verification (Nexus Staging Repositories)
 
 As part of uploading to repository.apache.org the signatures are verified to match a KEY that has been distributed, but RAO does not verify that the jar files are built with a key trusted by the Grails project.
 
@@ -113,7 +115,7 @@ Example:
     ./etc/bin/verify-jar-artifacts.sh v7.0.0-M4 <grailsdownloadlocation>
 ```
 
-### Reproducible Jar File
+### Manual Verification: Reproducible Jar Files
 After all jar files are verified to be signed by a valid Grails key, we need to build a local copy to ensure the file was built with the right code base. The `very-reproducible.sh` script handles this check, but if the bootstrap needs to be manually bootstrapped, perform the following step: 
 
     gradle -p gradle bootstrap
@@ -124,13 +126,17 @@ If there are any jar file differences, confirm they are relevant by following th
 1. Extract the differing jar file using the `etc/bin/extract-build-artifact.sh <jarfilepath from diff.txt>`
 2. In IntelliJ, under `etc/bin/results` there will now be a `firstArtifact` & `secondArtifact` folder. Select them both, right click, and select `Compared Directories`  
 
-### Binary Distribution Verification
+Please note that Grails is officially built on Linux so if there are differences they may be due to the OS platform.
+There is a dockerfile checked into to assist building in an environment like GitHub actions. Please see the section
+`Appendix: Verification from a Container` for more information.
+
+### Manual Verification: Binary Distribution Verification
 
 Grails has 2 binary distributions:
    * `grailsw` - the Grails wrapper, which is a script that downloads the necessary jars to run Grails. This will exist inside of the generated applications, but can be optionally downloaded as a standalone binary distribution.
    * `grails` - the delegating CLI, which is a script that delegates to the Grails Forge CLI. This is the `sdkman` distribution.
 
-#### Verify Grails Wrapper Binary Distribution
+#### Manual Verification: Verify Grails Wrapper Binary Distribution
 
 The following are the Grails Wrapper distribution artifacts:
 * `apache-grails-wrapper-<version>-incubating-bin.zip` - the wrapper distribution
@@ -152,7 +158,7 @@ Verifies the wrapper distribution signature via the command:
 Extracts the zip file and verifies the contents:
 * Ensure the `LICENSE` & `NOTICE` files are present to ensure license compliance.
 
-#### Verify Grails Delegating CLI Binary Distribution
+#### Manual Verification: Verify Grails Delegating CLI Binary Distribution
 
 The following are the Grails distribution artifacts:
 * `apache-grails-<version>-incubating-bin.zip` - the cli distribution that will be uploaded to sdkman
@@ -176,12 +182,17 @@ Extracts the zip file and verifies the contents:
 
 ## 3. Verifying the CLIs are Functional
 
-The CLI distribution consists of various CLI's: `grailsw` (wrapper), `grails` (delegating), `grails-forge-cli`, and `grails-shell-cli`. Each CLI needs tested to ensure it's functional prior to release:
+The CLI distribution consists of various CLI's: `grailsw` (wrapper), `grails` (delegating), `grails-forge-cli`, and
+`grails-shell-cli`. Each CLI needs tested to ensure it's functional prior to release. The `verify.sh` script will output
+example commands to perform this verification. However, it if manually verifying, these are the minimum suggested steps:
 
-* testing `grailsw`:
+* Testing `grailsw`:
     * set GRAILS_REPO_URL to the staging repository (https://repository.apache.org/content/groups/staging)
     * run `grailsw` and ensure it downloads the correct jars to `.grails` (verify the checksums of the jars)
-* testing `grails-shell-cli`:
+  * Please note that the Grails Wrapper (`grailsw`) writes to the `.grails` directory in the user's home directory. If
+    not run in an isolated environment, it may attempt to use already downloaded artifacts. If the wrapper fails, try
+    removing the `.grails` directory to confirm it isn't a transient state issue.
+* Testing `grails-shell-cli`:
     * create a basic app:
     ```bash
     grails-shell-cli create-app test
@@ -310,18 +321,13 @@ Setup the key for validity:
 
 # Appendix: Verification from a Container
 
-The Grails image is officially built on linux in a GitHub action using an Ubuntu container. To run a linux container locally, you can use the following command:
+The Grails image is officially built on linux in a GitHub action using an Ubuntu container. To run a linux container
+locally, you can use the following command (substitute `<git-tag-of-release` with the tag name):
 
 ```bash
     docker build -t grails:testing -f etc/bin/Dockerfile . && docker run -it --rm -v $(pwd):/home/groovy/project -p 8080:8080 grails:testing bash
     cd grails-verify
-    verify.sh v7.0.0-M4 .
-    cd grails 
-    gradlew wrapper
-    cd grails-gradle 
-    gradlew wrapper
-    cd ../..
-    verify-reproducible.sh .
+    verify.sh <git-tag-of-release> .
 ```
 
 Please note that the argument `-p 8080:8080` is used to expose the port 8080 of the container to the host machine's port 8080 (fromContainerPort:toHostPort). This allows you to access any running Grails application in the container from your host. If you have another application on port 8080, you can change the port mapping to avoid conflicts, e.g., `-p 8080:8081`. 

gradle/publish-root-config.gradle

@@ -140,15 +140,8 @@ subprojects {
     }
 }
 
-project.afterEvaluate {
-    tasks.register('publishAllPublicationsToTestCaseMavenRepoRepository').configure {
-        dependsOn(
-                subprojects
-                        .findAll { it.name in publishedProjects }
-                        .collect { it.tasks.named('publishAllPublicationsToTestCaseMavenRepoRepository') }
-        )
-    }
-}
+// subprojects will register dependencies via the publish plugin
+tasks.register('publishAllPublicationsToTestCaseMavenRepoRepository')
 
 def aggregatePublishedArtifacts = tasks.register('aggregatePublishedArtifacts')
 

grails-gradle/plugins/src/main/groovy/org/grails/gradle/plugin/publishing/GrailsPublishGradlePlugin.groovy

@@ -22,6 +22,7 @@ import grails.util.GrailsNameUtils
 import groovy.namespace.QName
 import io.github.gradlenexus.publishplugin.InitializeNexusStagingRepository
 import io.github.gradlenexus.publishplugin.NexusPublishPlugin
+import org.gradle.api.GradleException
 import org.gradle.api.Plugin
 import org.gradle.api.Project
 import org.gradle.api.Task
@@ -178,12 +179,12 @@ Note: if project properties are used, the properties must be defined prior to ap
         projectPluginManager.apply(MavenPublishPlugin)
 
         boolean localSigning = false
+        String signingKeyId = project.findProperty('signing.keyId') ?: System.getenv('SIGNING_PASSPHRASE')
         if (isRelease) {
-            String signingKeyId = project.findProperty('signing.keyId') ?: System.getenv('SIGNING_KEY')
             extraPropertiesExtension.setProperty('signing.keyId', signingKeyId)
             String secringFile = project.findProperty('signing.secretKeyRingFile') ?: System.getenv('SIGNING_KEYRING')
             if (!secringFile) {
-                project.logger.info("No keyring file has been specified. Assuming the use of local gpgCommand instead.")
+                project.logger.lifecycle('No keyring file (SIGNING_KEYRING) has been specified. Assuming the use of local gpgCommand to sign instead.')
                 localSigning = true
                 extraPropertiesExtension.setProperty('signing.gnupg.keyName', signingKeyId)
             } else {
@@ -282,6 +283,17 @@ Note: if project properties are used, the properties must be defined prior to ap
                             }
                         }
                     }
+                } else {
+                    // This is a local publish. Add the test case repository if it's defined on the extension.
+                    def testRepoPath = gpe.testRepositoryPath.getOrNull()
+                    if (testRepoPath) {
+                        repositories {
+                            maven {
+                                name = 'TestCaseMavenRepo'
+                                url = testRepoPath
+                            }
+                        }
+                    }
                 }
 
                 publications {
@@ -426,12 +438,23 @@ Note: if project properties are used, the properties must be defined prior to ap
                 // The sign task does not properly setup dependencies, see https://github.com/gradle/gradle/issues/26091
                 project.tasks.withType(Sign).configureEach {
                     it.dependsOn(project.tasks.withType(Jar))
+                    it.doFirst {
+                        if (!signingKeyId) {
+                            throw new GradleException('A signing key is required to sign a release. Set GRAILS_PUBLISH_RELEASE=false to bypass signing.')
+                        }
+                    }
                 }
                 project.tasks.withType(PublishToMavenRepository).configureEach {
                     it.mustRunAfter(project.tasks.withType(Sign))
                 }
             }
 
+            if (project.rootProject.tasks.names.contains('publishAllPublicationsToTestCaseMavenRepoRepository')) {
+                project.rootProject.tasks.named('publishAllPublicationsToTestCaseMavenRepoRepository').configure {
+                    it.dependsOn(project.tasks.named('publishAllPublicationsToTestCaseMavenRepoRepository'))
+                }
+            }
+
             addInstallTaskAliases(project)
         }
     }