frontend/dockerui: share context load across bake targets#6729
Open
vanackere wants to merge 1 commit intomoby:masterfrom
Open
frontend/dockerui: share context load across bake targets#6729vanackere wants to merge 1 commit intomoby:masterfrom
vanackere wants to merge 1 commit intomoby:masterfrom
Conversation
When buildx runs a multi-target bake with the same local context, it can pass that context as a shared session with local-sessionid:context. The dockerfile frontend still computes FollowPaths per target from the context files reachable by that target and passes the filter to MainContext.
Small reproducer:
# Dockerfile
FROM alpine AS app
RUN --mount=type=bind,source=app/cfg,target=/cfg cat /cfg
FROM alpine AS tools
RUN --mount=type=bind,source=tools/cfg,target=/cfg cat /cfg
# docker-bake.hcl
target "app" { context = "."; target = "app" }
target "tools" { context = "."; target = "tools" }
group "default" { targets = ["app", "tools"] }
With app/cfg and tools/cfg present, a bake of both targets can show two separate context-load vertices before this change:
=> [app internal] load build context
=> [tools internal] load build context
The targets compute different FollowPaths values, so the marshaled llb.Local("context") ops differ even though they use the same shared local session.
For shared-session main contexts, clear FollowPaths after applying the caller-provided local options. Non-shared builds keep the existing per-target FollowPaths optimization.
Add a dockerui unit test covering the option behavior: non-shared sessions preserve FollowPaths, while shared sessions clear it and produce identical context source ops for divergent caller filters.
Author
|
@tonistiigi this commit replaces #6725 (this one was definitely wrong, sorry for that I messed-up while trying to make a minimal case to reproduce my issue) I made a small reproducer case here if it helps: https://github.com/vanackere/misc/tree/main/docker-debug/context-load |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When buildx runs a multi-target bake with the same local context, it can pass that context as a shared session with local-sessionid:context. The dockerfile frontend still computes FollowPaths per target from the context files reachable by that target and passes the filter to MainContext.
Small reproducer:
With app/cfg and tools/cfg present, a bake of both targets can show two separate context-load vertices before this change:
The targets compute different FollowPaths values, so the marshaled llb.Local("context") ops differ even though they use the same shared local session.
For shared-session main contexts, clear FollowPaths after applying the caller-provided local options. Non-shared builds keep the existing per-target FollowPaths optimization.
Add a dockerui unit test covering the option behavior: non-shared sessions preserve FollowPaths, while shared sessions clear it and produce identical context source ops for divergent caller filters.
On my real-world - and quite complex - multi-stage Dockerfile this patch significantly improves the build time (times with cold/empty build cache):
Before:
After: