Skip to content

perf: threading unused for permuted contiguous views #72

Description

@lkdvos

When one of the inputs of the kernel is not column-major, the threading model never fires and everything silently runs in single-threaded mode.

The pipeline is fuse - order - block - kernel, which means that fusing is tested before things get put into canonical order. As a result, something like permutedims(A, (2, 1)) .= permutedims(B, (2, 1)) fails to fuse the dimensions even though this is just a regular copy.
This leaves some performance on the table for some cases in single-threaded mode, but it also makes it such that the multithreading never fires.

Probably the easiest fix is to first order, and only then fuse.

MWE:

using Strided
f(x) = x * exp(-2x) + sin(x * x)

function bench(n, niter)
       A  = rand(n, n)                                 # column-major, strides (1, n)
       B  = similar(A)
       At = StridedView(PermutedDimsArray(A, (2, 1)))  # permuted, strides (n, 1)
       Bt = StridedView(PermutedDimsArray(B, (2, 1)))

           # warm up
       @strided B  .= f.(A)
       @strided Bt .= f.(At)

       t_col = @elapsed for _ in 1:niter; @strided B  .= f.(A);  end
       t_row = @elapsed for _ in 1:niter; @strided Bt .= f.(At); end

       println("threads   : ", Threads.nthreads())   
       println("elements  : ", n * n)
       println("col-major : ", 1e3 * t_col / niter, " ms")
       println("row-major : ", 1e3 * t_row / niter, " ms")
end

julia> bench(1000, 20)
threads   : 8
elements  : 1000000
col-major : 4.6721882 ms
row-major : 10.9288698 ms

See also https://github.com/tensor4all/strided-rs/blob/d759d2a57a3ea47b2f1f7ec431ac80aedeb867c9/docs/strided_jl_threading_bug.md?plain=1#L4

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions