I have a multi-threaded loop running some simulations each iteration. I'd like to be able to save the results intermediately, something like
using Base.Threads
using JLD2
@Threads.threads for i in 1:100
simulation_data = run_simulation(i)
# something blocking here, a mutex or atomic block??
jldopen("output.jld2","a") do f
write(f, "hdf/path/to/data/$i", simulation_data)
end
end
Is something like this possible? I couldn't find anything other than atomic operations on numeric types in the docs.
In my particular application, it's not too bad to wait until after the loop and do all the IO single-threaded, but I thought I'd ask anyway.
yes, you'd need to hold a lock around that bit of IO code
Last updated: Nov 06 2024 at 04:40 UTC