When one of the inputs of the kernel is not column-major, the threading model never fires and everything silently runs in single-threaded mode.
The pipeline is fuse - order - block - kernel, which means that fusing is tested before things get put into canonical order. As a result, something like permutedims(A, (2, 1)) .= permutedims(B, (2, 1)) fails to fuse the dimensions even though this is just a regular copy.
This leaves some performance on the table for some cases in single-threaded mode, but it also makes it such that the multithreading never fires.
Probably the easiest fix is to first order, and only then fuse.
MWE:
using Strided
f(x) = x * exp(-2x) + sin(x * x)
function bench(n, niter)
A = rand(n, n) # column-major, strides (1, n)
B = similar(A)
At = StridedView(PermutedDimsArray(A, (2, 1))) # permuted, strides (n, 1)
Bt = StridedView(PermutedDimsArray(B, (2, 1)))
# warm up
@strided B .= f.(A)
@strided Bt .= f.(At)
t_col = @elapsed for _ in 1:niter; @strided B .= f.(A); end
t_row = @elapsed for _ in 1:niter; @strided Bt .= f.(At); end
println("threads : ", Threads.nthreads())
println("elements : ", n * n)
println("col-major : ", 1e3 * t_col / niter, " ms")
println("row-major : ", 1e3 * t_row / niter, " ms")
end
julia> bench(1000, 20)
threads : 8
elements : 1000000
col-major : 4.6721882 ms
row-major : 10.9288698 ms
See also https://github.com/tensor4all/strided-rs/blob/d759d2a57a3ea47b2f1f7ec431ac80aedeb867c9/docs/strided_jl_threading_bug.md?plain=1#L4
When one of the inputs of the kernel is not column-major, the threading model never fires and everything silently runs in single-threaded mode.
The pipeline is fuse - order - block - kernel, which means that fusing is tested before things get put into canonical order. As a result, something like
permutedims(A, (2, 1)) .= permutedims(B, (2, 1))fails to fuse the dimensions even though this is just a regular copy.This leaves some performance on the table for some cases in single-threaded mode, but it also makes it such that the multithreading never fires.
Probably the easiest fix is to first
order, and only thenfuse.MWE:
See also https://github.com/tensor4all/strided-rs/blob/d759d2a57a3ea47b2f1f7ec431ac80aedeb867c9/docs/strided_jl_threading_bug.md?plain=1#L4