Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aot: move jl_insert_backedges to Julia side #56499

Merged
merged 3 commits into from
Jan 15, 2025

Conversation

aviatesk
Copy link
Member

@aviatesk aviatesk commented Nov 8, 2024

With #56447, the dependency between jl_insert_backedges and method insertion has been eliminated, allowing jl_insert_backedges to be performed after loading. As a result, it is now possible to move jl_insert_backedges to the Julia side.

Currently this commit simply moves the implementation without adding any new features.

@aviatesk aviatesk requested a review from vtjnash November 8, 2024 08:47
@aviatesk aviatesk force-pushed the avi/move-staticdata_utils-julia branch from 4959411 to 2eafa61 Compare November 8, 2024 10:49
@gbaraldi
Copy link
Member

gbaraldi commented Nov 8, 2024

Does this have any performance diff?

@aviatesk
Copy link
Member Author

aviatesk commented Nov 8, 2024

Moving the implementation to Julia seems to result in a slight increase in allocations, but considering the potential for future improvements, it doesn’t seem like a significant issue

julia> @time using Debugger
  0.233097 seconds (626.92 k allocations: 33.375 MiB, 26.87% compilation time: 100% of which was recompilation) # master
  0.228869 seconds (627.05 k allocations: 33.421 MiB, 27.19% compilation time: 100% of which was recompilation)

@aviatesk aviatesk force-pushed the avi/move-staticdata_utils-julia branch from 2eafa61 to 718aea8 Compare November 9, 2024 11:02
@aviatesk aviatesk force-pushed the avi/move-staticdata_utils-julia branch 4 times, most recently from 907d058 to 6b69260 Compare November 22, 2024 08:56
@aviatesk aviatesk force-pushed the avi/move-staticdata_utils-julia branch 2 times, most recently from 7b54cf5 to 87836bf Compare January 6, 2025 17:37
base/loading.jl Outdated Show resolved Hide resolved
base/loading.jl Outdated Show resolved Hide resolved
base/loading.jl Outdated Show resolved Hide resolved
Copy link
Member

@vtjnash vtjnash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aviatesk aviatesk force-pushed the avi/move-staticdata_utils-julia branch from 109748f to 929d53c Compare January 7, 2025 06:48
aviatesk added a commit that referenced this pull request Jan 8, 2025
If #56499 is merged, the functionality of
`jl_reinit_ccallable` will become incomplete. However, it is
questionable whether `jl_reinit_ccallable` is truly necessary.
It seems to have been added in #44527, but all test cases
appear to pass without it. Therefore, it might be safe to remove it
altogether.
aviatesk added a commit that referenced this pull request Jan 8, 2025
If #56499 is merged, the functionality of
`jl_reinit_ccallable` will become incomplete. However, it is
questionable whether `jl_reinit_ccallable` is truly necessary.
It seems to have been added in #44527, but all test cases
appear to pass without it. Therefore, it might be safe to remove it
altogether.
@aviatesk aviatesk force-pushed the avi/move-staticdata_utils-julia branch from 929d53c to 8ed7f7f Compare January 8, 2025 06:49
Keno added a commit that referenced this pull request Jan 9, 2025
Our binding partion invalidation code scans the original source
for any GlobalRefs that need to be invalidated. However, this
is not the only source of access to binding partitions. Various
intrinsics (in particular the `*global` ones) also operate on
bindings. Since these are not manifest in the source, we instead
use the existing `edges` mechanism to give them forward edges.

This PR only includes the basic info type, and validation in the
replacement case. There's a bit more work to do there, but I'm
waiting on #56499 for that part, precompilation in particular.
Keno added a commit that referenced this pull request Jan 9, 2025
Our binding partion invalidation code scans the original source
for any GlobalRefs that need to be invalidated. However, this
is not the only source of access to binding partitions. Various
intrinsics (in particular the `*global` ones) also operate on
bindings. Since these are not manifest in the source, we instead
use the existing `edges` mechanism to give them forward edges.

This PR only includes the basic info type, and validation in the
replacement case. There's a bit more work to do there, but I'm
waiting on #56499 for that part, precompilation in particular.
Keno added a commit that referenced this pull request Jan 10, 2025
Our binding partion invalidation code scans the original source
for any GlobalRefs that need to be invalidated. However, this
is not the only source of access to binding partitions. Various
intrinsics (in particular the `*global` ones) also operate on
bindings. Since these are not manifest in the source, we instead
use the existing `edges` mechanism to give them forward edges.

This PR only includes the basic info type, and validation in the
replacement case. There's a bit more work to do there, but I'm
waiting on #56499 for that part, precompilation in particular.
Keno added a commit that referenced this pull request Jan 10, 2025
Our binding partion invalidation code scans the original source for any
GlobalRefs that need to be invalidated. However, this is not the only
source of access to binding partitions. Various intrinsics (in
particular the `*global` ones) also operate on bindings. Since these are
not manifest in the source, we instead use the existing `edges`
mechanism to give them forward edges.

This PR only includes the basic info type, and validation in the
replacement case. There's a bit more work to do there, but I'm waiting
on #56499 for that part, precompilation in particular.
@aviatesk aviatesk force-pushed the avi/move-staticdata_utils-julia branch 2 times, most recently from e1c3ce6 to 9340251 Compare January 13, 2025 10:51
@aviatesk aviatesk changed the base branch from master to avi/rm-reinit_ccallable January 13, 2025 14:16
@aviatesk aviatesk force-pushed the avi/move-staticdata_utils-julia branch from 9340251 to b5d0227 Compare January 13, 2025 14:30
@aviatesk
Copy link
Member Author

This should be ready to go.

@aviatesk aviatesk force-pushed the avi/move-staticdata_utils-julia branch from b5d0227 to d9d83a6 Compare January 14, 2025 06:33
@aviatesk aviatesk changed the base branch from avi/rm-reinit_ccallable to master January 14, 2025 06:33
@aviatesk
Copy link
Member Author

This PR is currently depending on and thus blocked by #56987. However, I believe it is not strictly necessary to merge #56987 before this PR. For the sake of the ongoing development of partitioned bindings, I have made this PR standalone and ready for quick merging.

@aviatesk aviatesk added the merge me PR is reviewed. Merge when all tests are passing label Jan 14, 2025
@aviatesk aviatesk force-pushed the avi/move-staticdata_utils-julia branch 2 times, most recently from 0b2575d to 2e7b0b2 Compare January 14, 2025 10:53
base/staticdata.jl Outdated Show resolved Hide resolved
base/staticdata.jl Outdated Show resolved Hide resolved
@@ -0,0 +1,296 @@
# This file is a part of Julia. License is MIT: https://julialang.org/license

module StaticData
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This eventually should be moved inside the Compiler module and locked to the Compiler world age, since broadly speaking this algorithm is just a slightly different re-implementation of that entire code. I guess that isn't blocking though for now, since this is a private module name

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move it to Compiler in a separate PR.

aviatesk and others added 3 commits January 15, 2025 13:45
With #56447, the dependency between `jl_insert_backedges`
and method insertion has been eliminated, allowing `jl_insert_backedges`
to be performed after loading. As a result, it is now possible to move
`jl_insert_backedges` to the Julia side.

Currently this commit simply moves the implementation without adding
any new features.
Co-authored-by: Jameson Nash <[email protected]>
@aviatesk aviatesk force-pushed the avi/move-staticdata_utils-julia branch from d427164 to 6e8bd6b Compare January 15, 2025 04:47
@aviatesk aviatesk merged commit ef04055 into master Jan 15, 2025
5 of 7 checks passed
@aviatesk aviatesk deleted the avi/move-staticdata_utils-julia branch January 15, 2025 09:13
@inkydragon inkydragon removed the merge me PR is reviewed. Merge when all tests are passing label Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants