-
-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issue with OffsetArrays #186
Comments
Should hopefully be fixed by #175, but what is the end goal? Would be lovely to have a larger test-case, I suspect the OffsetArrays will shake out a couple of assumption we have about axes vs indices |
I have a problem that I'd usually use C++ and MPI for which involves many nodes loading data and holding that data in memory for the duration of the program. The data collectively represents a large domain. Local computation is performed and halos (the edge cells of the local matrices) are exchanged to keep things synchronized. Julia doesn't seem well-suited to this paradigm from a library perspective, though the language itself seems fine. So I've been experimenting with ways to achieve this behaviour. |
Yes that sounds like a usecase I would love to eventually support. We have a project right now that will require doing something similar. |
Note that with the conversion method now being defined in julia> @everywhere using DistributedArrays
julia> @everywhere using OffsetArrays
julia> r1 = @spawnat 2 OffsetArray(zeros(6,6), 0:5, 0:5)
Future(2, 1, 93, nothing)
julia> r2 = @spawnat 3 OffsetArray(zeros(6,6), 0:5, 0:5)
Future(3, 1, 94, nothing)
julia> r3 = @spawnat 4 OffsetArray(zeros(6,6), 0:5, 0:5)
Future(4, 1, 95, nothing)
julia> r4 = @spawnat 5 OffsetArray(zeros(6,6), 0:5, 0:5)
Future(5, 1, 96, nothing)
julia> ras = [r1 r2; r3 r4]
2×2 Matrix{Future}:
Future(2, 1, 93, nothing) Future(3, 1, 94, nothing)
Future(4, 1, 95, nothing) Future(5, 1, 96, nothing)
julia> D = DArray(ras);
julia> D |> typeof
DArray{Float64, 2, OffsetMatrix{Float64, Matrix{Float64}}} However the indices are not correct as julia> D |>axes
(Base.OneTo(12), Base.OneTo(12)) |
I was getting nervous about the communication costs of being imprecise with communications outside of the localpart of the array, so I figured I'd build a halo exchange system by combining OffsetArrays and DistributedArrays.
I intended for that to look like the following, where each array has a 4x4 block of purely local data and a surrounding 1-cell halo to cache remote data pulled using DistributedArrays.
Unfortunately, when I try to run this, I get:
which doesn't seem like an issue with OffsetArrays because using SparseArrays in an analogous way produces no problems:
Sadly, I'm not familiar enough with Julia to know why the interaction of OffsetArrays and DistributedArrays is problematic.
I've got:
The text was updated successfully, but these errors were encountered: