✔ Most straightforward way to add threads to a `map` call. · helpdesk (published)

Stream: helpdesk (published)

Topic: ✔ Most straightforward way to add threads to a `map` call.

Júlio Hoffimann (Dec 26 2023 at 19:23):

Suppose I have a function f(x) that takes constant time, and a map as follows:

function task(xs)
  map(xs) do x
    f(x)
  end
end

What is the most straightforward way to perform the computation with multiple threads in Julia v1.9?

jar (Dec 26 2023 at 19:29):

Use ThreadsX.map

Júlio Hoffimann (Dec 26 2023 at 19:29):

(without a package)

Júlio Hoffimann (Dec 26 2023 at 19:30):

I can use Transducers.tcollect, but was wondering if I could avoid the dependency.

jar (Dec 26 2023 at 19:30):

I'm sure you can do it with @sync @spawn or whatever, I just don't see the point

Júlio Hoffimann (Dec 26 2023 at 19:31):

The only reason is to avoid an extra dependency, but I may end up adding Transducers.jl to the dep list at the end of the day.

Mason Protter (Dec 26 2023 at 19:36):

do you know the output type of f(::typeof(x))?

Júlio Hoffimann (Dec 26 2023 at 19:42):

Yes, I have this a priori. In this context, x is a point in $R^3$ , represented by Point struct, and the result is a Float64

Mason Protter (Dec 26 2023 at 19:51):

Probably the easiest thing would just be do to

julia> using Base.Threads: nthreads, @spawn, @sync

julia> function tmap(f, ::Type{R}, xs::Array; tasks_per_thread::Int=2) where {R}
           out = Array{R}(undef, size(xs)...)
           chunk_size = max(1, length(xs) ÷ (tasks_per_thread * nthreads()))
           @sync for chunk ∈ Iterators.partition(enumerate(xs), chunk_size)
               @spawn for (i, x) ∈ chunk
                   out[i] = f(x)
               end
           end
           out
       end