Say I have a loop like:
for i in 1:100000
for j in 1:100000
i <= j && continue # avoid double counting
# do some actual work
end
end
where the condition i <= j
is used to skip the inner loop iterations. Is there any performance decrease to define this skip condition with dispatch?
skipfun(::MyType1) = (i, j) -> i <= j
skipfun(::MyType2) = (i, j) -> false
If I replace the literal i <= j
in the loop by a call to these lambda functions, should I expect slow down or the compiler is good enough to cleanup the code? How would you investigate it further with existing Julia tools?
Are you capturing i and j from outer scope? That could be a problem
(deleted)
On the other side, why not:
skipfun(::MyType1, i, j) = i <= j
skipfun(::MyType2, i, j) = false
to be used with, within the loop:
skipfun(MyType1(), i, j) && continue
That certainly has no signficant overhead relative to the conditional.
Or, if the conditional is already a problem (in the case it is unnecessary), define one iterator for each type, like:
julia> struct A end
julia> struct B end
julia> iter(::A,n,m) = Iterators.product(1:n,1:m)
iter (generic function with 2 methods)
julia> iter(::B,n,m) = Iterators.filter(pair -> pair[1] < pair[2], Iterators.product(1:n,1:m))
iter (generic function with 2 methods)
julia> for pair in iter(A(), 3, 3)
@show pair
end
pair = (1, 1)
pair = (2, 1)
pair = (3, 1)
pair = (1, 2)
pair = (2, 2)
pair = (3, 2)
pair = (1, 3)
pair = (2, 3)
pair = (3, 3)
julia> for pair in iter(B(), 3, 3)
@show pair
end
pair = (1, 2)
pair = (1, 3)
pair = (2, 3)
Thank you all for the tips. I will proceed with the custom iterator approach, it is the most self-contained :+1:
I wonder how I could inspect the actual code generated by the compiler and see that the iterators are compiled away?
I don't think they are compiled away in any sense. The result should be very similar to the actual loops without and with the conditional. It may be more efficient in the case of the conditional to write an iterator that only runs over the upper diagonal elements instead of filtering.
(I wouldn't bet filter is smart enough to do that)
In case you want to give it a try, here is the code in question:
The skip
function is retrieved with dispatch from two different algorithm types.
I commented there. In that situation, first, I would just dispatch to differnet skip
mehods depending on the type, without using a closure.
Last updated: Nov 06 2024 at 04:40 UTC