Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: skip deep dependency comparison for Gradle projects #9817

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

jjohannes
Copy link
Contributor

This is unnecessary and leads to a long runtime (and possible an infinite loop/recursion) for large dependency graphs as observed in #9763.

Copy link

codecov bot commented Jan 23, 2025

Codecov Report

Attention: Patch coverage is 33.33333% with 4 lines in your changes missing coverage. Please review.

Project coverage is 68.19%. Comparing base (48b638f) to head (7229394).

Files with missing lines Patch % Lines
...el/src/main/kotlin/utils/DependencyGraphBuilder.kt 20.00% 2 Missing and 2 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #9817      +/-   ##
============================================
- Coverage     68.21%   68.19%   -0.02%     
  Complexity     1293     1293              
============================================
  Files           250      250              
  Lines          8849     8853       +4     
  Branches        920      921       +1     
============================================
+ Hits           6036     6037       +1     
- Misses         2424     2426       +2     
- Partials        389      390       +1     
Flag Coverage Δ
funTest-docker 65.14% <ø> (ø)
funTest-non-docker 33.35% <0.00%> (-0.03%) ⬇️
test-ubuntu-24.04 35.97% <33.33%> (-0.01%) ⬇️
test-windows-2022 35.95% <33.33%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@sschuberth sschuberth force-pushed the gradle-graph-buillder-quick-fix branch from a3c19f7 to 86d09b3 Compare January 23, 2025 17:32
@sschuberth sschuberth marked this pull request as ready for review January 23, 2025 17:32
@sschuberth sschuberth requested a review from a team as a code owner January 23, 2025 17:32
@sschuberth sschuberth enabled auto-merge (rebase) January 23, 2025 17:32
sschuberth
sschuberth previously approved these changes Jan 23, 2025
@sschuberth
Copy link
Member

There are some test differences to inspect.

@sschuberth
Copy link
Member

There are some test differences to inspect.

For whatever reason, the change causes a lot of the following differences in the dependency tree:

image

It basically seems that cycles are now cut too early or so.

@sschuberth
Copy link
Member

It basically seems that cycles are now cut too early or so.

Or actually, it seems that runtime dependencies are omitted now. Take a look at e.g. lifecycle-livedata-core:2.5.0:

image

core-common:2.1.0 and core-runtime:2.1.0 are exactly the omitted direct dependencies.

@sschuberth sschuberth force-pushed the gradle-graph-buillder-quick-fix branch from 86d09b3 to 05b57b9 Compare January 23, 2025 22:08
val dependencies2 = dependencies.associateBy { dependencyHandler.identifierFor(it) }
if (!dependencies2.keys.containsAll(dependencies1)) return false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, @oheger-bosch do you recall why the original code uses an "uni-directional" containsAll() here? In other words, why did we continue here if dependencies2 was a superset of dependencies1, instead of also returning false early?

This makes the diff, if any, to expected results more readable.

Signed-off-by: Sebastian Schuberth <[email protected]>
auto-merge was automatically disabled February 4, 2025 13:54

Head branch was pushed to by a user without write access

@jjohannes jjohannes force-pushed the gradle-graph-buillder-quick-fix branch from 05b57b9 to bbd5289 Compare February 4, 2025 13:54
@jjohannes
Copy link
Contributor Author

@sschuberth in order to make this solution workable, the selected variants need to be tracked. The same component may have a different set of dependencies, depending on which variant was selected. In most cases, it is only one, but Gradle also supports to select multiple in one scope under certain conditions.

I added variants as an additional optional field of the Identifier data class to check if the tests pass with this. It likely works, but you have to check is such an addition is desired.

@jjohannes jjohannes force-pushed the gradle-graph-buillder-quick-fix branch from bbd5289 to ae58a8c Compare February 4, 2025 14:41
This can be avoided as it leads to a long runtime (and possible an
infinite loop / recursion) for large dependency graphs. In order
to be able to skip the deep comparison, the selected variants
need to be tracked in the Identifier. In the DependencyGraphBuilder
all 'scopes' (runtimeClasspath, compileClasspath, ...) are thrown
together. Gradle may have selected different variants of the components
in different 'scopes'.

Signed-off-by: Jendrik Johannes <[email protected]>
@jjohannes jjohannes force-pushed the gradle-graph-buillder-quick-fix branch from ae58a8c to 7229394 Compare February 4, 2025 15:05
@jjohannes
Copy link
Contributor Author

Wit the change in this PR, the problem described in the description of #9763 no longer occurs.

@mnonnenmacher
Copy link
Member

I added variants as an additional optional field of the Identifier data class to check if the tests pass with this. It likely works, but you have to check is such an addition is desired.

Changing the Identifier class is indeed problematic as it is used in so many places, so even if no tests are failing there could be unintended side effects. We have discussed to switch to PURL (#8982) which should also make it easier to add such custom properties, but that's a huge refactoring.

However, I wonder if even you even use the new property anywhere in this change? I couldn't find it.

@jjohannes
Copy link
Contributor Author

Changing the Identifier class is indeed problematic...

Yes I was afraid that it may not be a simple thing to do. Although, the change only adds a new field (variants) that defaults to emptySet(). Therefore, it should/would be non breaking. But I can understand, if you still see it as too risky.

However, I wonder if even you even use the new property anywhere in this change? I couldn't find it.

I is implicitly used by dependencies1 == dependencies2.keys as it adds a new value that is used when the data objects are compared.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants