Skip to content

Conversation

@topolarity
Copy link
Contributor

These implementations are basic, but I tried to follow the patterns in the other parts of the Parallel module.

My real motivation is to be able to move the Distributed implementations into an extension, so that Graphs.jl does not depend on Distributed.

@simonschoelly
Copy link
Member

What is the issue with using Distrubuted? As far as I know it is a standard library.

@oscardssmith
Copy link
Member

Multithreading typically has lower overhead since it's shared memory. Distributed is usually only worthwhile if you are trying to distribute across multiple machines.

@oscardssmith
Copy link
Member

Also, Distributed is a stdlib, but stdlibs are still dependencies, and as we have started moving stlibs out of the sysimage, they are becoming more like regular packages.

@topolarity topolarity force-pushed the ct/missing-threaded-impls branch from b5b55a0 to c109686 Compare May 14, 2025 13:14
@Krastanov
Copy link
Member

Would this require also bumping the minimum requirement to julia=1.9?

I am generally in favor of moving more dependencies to optional dependencies. It makes compilation times much more pleasant as well.

@topolarity topolarity force-pushed the ct/missing-threaded-impls branch 2 times, most recently from 2e9b9f3 to a7db672 Compare May 14, 2025 14:19
@codecov
Copy link

codecov bot commented May 14, 2025

Codecov Report

Attention: Patch coverage is 94.33962% with 3 lines in your changes missing coverage. Please review.

Project coverage is 97.38%. Comparing base (6130332) to head (d8a8730).
Report is 12 commits behind head on master.

Files with missing lines Patch % Lines
src/Parallel/distance.jl 94.73% 1 Missing ⚠️
src/Parallel/shortestpaths/dijkstra.jl 94.44% 1 Missing ⚠️
src/Parallel/traversals/greedy_color.jl 93.75% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #429      +/-   ##
==========================================
+ Coverage   97.31%   97.38%   +0.07%     
==========================================
  Files         117      120       +3     
  Lines        6956     7007      +51     
==========================================
+ Hits         6769     6824      +55     
+ Misses        187      183       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@topolarity topolarity force-pushed the ct/missing-threaded-impls branch from fc4a1b9 to cbda59e Compare May 14, 2025 16:49
@topolarity
Copy link
Contributor Author

Would this require also bumping the minimum requirement to julia=1.9?

There are compatibility pathways where we only switch to an extension on 1.9+ and keep the hard dependency for earlier versions

I think this PR should be ready for review now - it is passing tests for me locally

Copy link
Member

@Krastanov Krastanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I am a bit new to the maintainer team, and this does introduce a new public feature (the parallel kwarg), so I will wait a bit for dissent from folks with more seniority.

@topolarity
Copy link
Contributor Author

topolarity commented May 14, 2025

a new public feature (the parallel kwarg)

FWIW, that kwarg is already there on a lot of other methods, so it's not a new design choice exactly - it was just never provided for these particular operations (because they only had a parallel=:distributed implementation)

@simonschoelly
Copy link
Member

LGTM, but I am a bit new to the maintainer team, and this does introduce a new public feature (the parallel kwarg), so I will wait a bit for dissent from folks with more seniority.

There is some new multi threaded code here - so we probably should take a bit of time to review it - as it can lead to notoriously difficult to spot bugs.

Copy link
Member

@simonschoelly simonschoelly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks mostly good to me. I don't see any race condition

I have added some comments. They are mostly suggestions. In addition:

  • We probably should write some documentation for the new parallel argument. Although I admit that this module is fairly undocumented anyway - and as we don't export anything yet we don't have to be so rigorous.

@simonschoelly simonschoelly added the enhancement New feature or request label May 23, 2025
@topolarity topolarity force-pushed the ct/missing-threaded-impls branch 2 times, most recently from a99a063 to 5575b3a Compare June 6, 2025 11:11
These implementations are extremely basic, but they try to follow the
patterns in the other parts of the Parallel module. My real motivation
is to be able to move the Distributed implementations into an extension,
so that Graphs.jl does not depend on Distributed.
@topolarity topolarity force-pushed the ct/missing-threaded-impls branch from 5575b3a to d8a8730 Compare June 6, 2025 11:13
Base.Threads.@threads for i in 1:vlen
d = Graphs.dijkstra_shortest_paths(g, vs[i], distmx)
eccs[i] = maximum(d.dists)
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit weird that the end is not covered by the tests - the code inside the loop is, right? If it is indeed then we might just merge it anyways.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That has been a common issue for me with macro-transformed begin-end blocks. I have never bothered delving into why it is happening though. Macro-heavy libraries like ResumableFunctions.jl and ConcurrentSim.jl suffer from that (and from other more severe macro-covereage issues).

@Krastanov
Copy link
Member

The PR Pre-Commit Bot failure is a problem with the github action, not this PR. (Probably due to this coming from a fork -- I will look into fixing the action)

@topolarity
Copy link
Contributor Author

Anything else needed from me here?

@topolarity
Copy link
Contributor Author

(polite) bump - anything still blocking merge?

@Krastanov
Copy link
Member

Thanks, @topolarity ! Sorry for the slow response, we are a bit low on volunteers right now and Simon is the only current maintainer with actual deep knowledge of the codebase.

Looking through the code, I do not see any issues, all of Simon's comments have been addressed, and any documentation requirements or API restructuring needs are reported as separate posts on the issue tracker. This seems to be complete. I will rerun the tests and merge after they pass.

@Krastanov Krastanov closed this Jul 2, 2025
@Krastanov Krastanov reopened this Jul 2, 2025
@Krastanov
Copy link
Member

The patch codecov is failing, but the overall project coverage is increasing, so it seems to be good on that front too. Merging. Thank you for fixing this up!

@Krastanov Krastanov merged commit 624c20a into JuliaGraphs:master Jul 2, 2025
21 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants