Skip to content

Collaboration, transfer, and adding iterators #38

Open
@timholy

Description

@timholy

JuliaLang/julia#31563 (comment), posted by @AriMKatz, drew my attention to this package. I agree that there appear to be a subset of shared goals between this package and https://github.com/timholy/ArrayIteration.jl. This is a proposal to collaborate so we don't duplicate effort.

To summarize ArrayIteration (because the README is ancient and talks about some stuff that long ago made it into Base): if you had to boil it down to a single goal, it's to provide an efficient fallback for A*B for matrices A and B with arbitrary sparsity patterns and representation. The general problem is that if you have to specialize on both typeof(A) and typeof(B), you have to write O(N^2) methods where N is the number of array types. (And wrappers like Adjoint make the problem worse because it's essentially O(4N^2) or O(9N^2).) ArrayIteration takes the stance that you can solve the problem with O(N) methods if you instead create a fancy iterator for each matrix that allows synchronization with another iterator. You sync the row iterator of A to the column iterator of B and thereby mostly just visit entries that are nonzero in both.

ArrayIteration is basically waiting for JuliaLang/julia#34126 (so we don't pay the allocation penalty for wrappers) and probably JuliaLang/julia#34847 before I trying bringing it back to life. The state of the art is the teh/stored2 branch which was almost functional, before the performance problems due to wrapper creation put it on hold. (The README, however, was not updated to reflect the state of that branch.)

I'd propose we all get together in a package hosted at JuliaArrays, which seems like the natural home for generic array stuff. We could start by transferring this one, since I'm guessing it's working whereas ArrayIteration is stalled out. If anyone wants to just steal the code in ArrayIteration that's fine too, but I don't think keeping it under the JuliaDiffEq umbrella makes sense if it's supposed to be generic array handling. We could also keep them two packages, but in that case we'd want to make sure they're independent but interoperable.

Also CCing @vchuravy whose https://github.com/JuliaGPU/KernelAbstractions.jl was also mentioned. I am less certain there are true shared goals there, but just in case.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions