Skip to content

Add full configuration cache support#13

Open
Luna712 wants to merge 5 commits intorecloudstream:masterfrom
Luna712:full-configcache
Open

Add full configuration cache support#13
Luna712 wants to merge 5 commits intorecloudstream:masterfrom
Luna712:full-configcache

Conversation

@Luna712
Copy link
Copy Markdown
Contributor

@Luna712 Luna712 commented Apr 11, 2026

Finally completes configuration cache compatibility for all remaining unsupported tasks: compileResources, make, cleanCache, and deployWithAdb, and fixes numerous issues with the makePluginsJson task with it.

compileResources

The task resolved the Android SDK directory (for aapt2 and android.jar) at execution time by querying LibraryExtensionCompat directly inside exec(). These paths are now declared as @InputFile properties (aaptExecutable, androidJar) and wired up at configuration time in Tasks.kt.

This also has massive performance benefits even without configuration cache because by wiring them into the task it prevents wiring the entire directory which for the SDK Directory can have tens of thousands of files in it which resulted in huge input size, and thus takes a lot longer than should be necessary. This can also be achieved by simply removing the android-actions/setup-android from building plugins also which is mostly unnecessary, and prevents having such a large directory passed to it, but it is still something good to properly handle as well.

make / GenerateManifestTask (new)

The root cause blocking configuration cache on make was that the plugin manifest was being built at execution time via Project.makeManifest(), which accessed project state lazily. This is fixed by introducing a dedicated GenerateManifestTask that produces the manifest JSON at configuration time using properly declared @Input/@OutputFile properties.

WriteCacheEntryTask (new)

Plugin entry metadata (file size, SHA-256 hashes, repo URLs, etc.) was previously computed in a doLast block on the make task and stored on the internal extension object, which is incompatible with configuration cache. This logic is extracted into a new WriteCacheEntryTask that runs after make, computes hashes and sizes from its declared @InputFile references, and writes a plugin-entry.json. The makePluginsJson task now achieves full configuration cache support by depending on writeCacheEntry and reading from these per-plugin entry files, rather than pulling data from the extension at execution time.

cleanCache, deployWithAdb, and generateSources

All three tasks previously resolved inputs lazily at execution time. They now receive all inputs (jarFile, adbPath, pluginFile) as configuration-time properties wired via providers.


With Project.makeManifest() and Project.makePluginEntry() both eliminated, and SHA-256 hashing now self-contained within WriteCacheEntryTask, Utils.kt no longer has any remaining responsibilities and has been deleted entirely. This PR also entirely replaces createProgressLogger() methods in DownloadUtils with service injection, which is fully configuration-cache safe and modern practices.

Finally, ensureJarCompatibility is also now automatically ran after compilePluginJar to ensure the JAR file is valid and to prevent the need for another separate task running.

All necessary tasks for plugin builds can now be done with just a single task (makePluginsJson) and nothing else. When running locally you can run make to avoid unnecessary plugins.json work, or also deployWithAdb will run make and everything else that is necessary automatically as well. Tasks are now are more tightly coupled and the task graph now follows things more cleanly to prevent race conditions and other things.


This is a little off-topic but kinda related as this PR sets up the foundation to support this as well. I just wanted to put this here so I don't forget. I also have some things planned for the future after this. To unify more logic all extensions use inside this plugin. I have a working PoC for this already, including the following:

fun Project.cloudstream(configuration: CloudstreamExtension.() -> Unit) = extensions.getByName<CloudstreamExtension>("cloudstream").configuration()

fun Project.android(configuration: LibraryExtension.() -> Unit) = extensions.getByName<LibraryExtension>("android").configuration()

tasks.register<Delete>("clean") {
    delete(rootProject.layout.buildDirectory)
}

But for the final task, a new task name would be used to avoid conflicts and maintain backwards compatibility. BaseExtension version of Project.android wouldn't be added, so that it could just be fully removed when migrating to AGP 9.


NOTE: This may still need more testing, however once the basic use cases are fully tested we could probably merge this, and depend on users reporting potential issues they find in more advanced usages.

UPDATE: i have fully tested the primary tasks (everything that runs to build plugins (make, makePluginsJson, etc...) and it all works. I have also briefly tested, though may need a little more testing, cleanCache and generateSources. deployWithAdb has not been tested.

UPDATE 2: I also just realized this fixes compatibility with Kotlin 2.1.0-2.3.0, fixing numerous errors Type argument is not within its bounds: must be subtype of 'Any'. with now disallowed nullable Property declarations, which are now proper @Optional declarations. It was supposedly an error in 2.1.0, but was inconsistent until 2.3.0 where they fixed it.

This adds full configuration cache support for the last two remaining tasks, `compileResources`, and `make`.

This took me like a year of trying *a lot* to finally figure out how to fix the `make` task for it. I finally figured out that the manifest generation was being done at execution time,  not configuration time. This adds a new task for generating manifests in order to do it at configuration time and support configuration cache. That also means the Project helper, `makeManifest()` is no longer necessary/used and is now removed.
@Luna712
Copy link
Copy Markdown
Contributor Author

Luna712 commented Apr 11, 2026

This seems to drastically speed up the build from about 6 minutes to about 2 minutes as well even without configuration cache. I am thinking as true configuration cache support will be a bit more difficult, we could mark it as not truly supporting configuration cache and merge this in the meantime while I work on true support.

Thoughts @fire-light42 ?

@Luna712
Copy link
Copy Markdown
Contributor Author

Luna712 commented Apr 13, 2026

This seems to drastically speed up the build from about 6 minutes to about 2 minutes as well even without configuration cache. I am thinking as true configuration cache support will be a bit more difficult, we could mark it as not truly supporting configuration cache and merge this in the meantime while I work on true support.

Thoughts @fire-light42 ?

This is now solved! This PR should now have support in every single area/task now.

@Luna712 Luna712 marked this pull request as ready for review April 13, 2026 00:15
@fire-light42
Copy link
Copy Markdown
Contributor

Looks good, but a more comprehensive review is required and will take some time.

@Luna712
Copy link
Copy Markdown
Contributor Author

Luna712 commented Apr 13, 2026

Looks good, but a more comprehensive review is required and will take some time.

@fire-light42

Understandable, appreciate that. Just to note though I have now tested on the largest repos with many plugins and it worked. I also compared plugins.json for both including file size and it fully matching. Functionality seems to work. I havent fully tested deployWithAdb yet though. Look forward to the review though, this isnt quite urgent yet but will be required with Gradle 10 which releases in a few months anyway.

I wish I could have split this PR more, but while there are some tiny cleanup to some lines that arent fully related most of this depends on the rest of it so I had to do just one larger PR this time unfortunately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants