Stream: helpdesk (published)

Topic: mutating array of structs

view this post on Zulip Frank Van Der Meulen (Jan 19 2022 at 18:43):

Suppose I have the struct

struct M
    a
    b
    c
end

and

x = [M(1,2,3), M(4,5,6), M(7,8,9)]

Now I wish to loop over each element of x and for each element adjust some fields. For example, if 

ind = [:a, :b]
vals =[10, 300]

the desired output would be 

[M(10,300,3), M(10,300,6), M(10,300,9)]

The following does the job, but I wonder if this is the righ tway to go...

using Setfield
for i in eachindex(x)
    xi = x[i]
    @set! xi.a=10
    @set! xi.b=300
    x[i] = xi
end
x

Any suggestions?

view this post on Zulip Mason Protter (Jan 19 2022 at 18:52):

That's how I'd do it, yeah.

view this post on Zulip Mason Protter (Jan 19 2022 at 18:55):

I assume you did it for ease of example, but just in case you don't know, leaving the fields of your struct untyped can carry a hefty performance price.

view this post on Zulip Frank Van Der Meulen (Jan 19 2022 at 18:59):

Yes, that was for each of example indeed. Ok, good to hear.

Now in this code I explicitly wrote like xi.a=10, and similarly xi.b=100, but I would rather loop over ind. I don't know how to do that.

view this post on Zulip Andrey Oskin (Jan 19 2022 at 19:00):

I wonder, maybe it's possible to use map!?

view this post on Zulip Mason Protter (Jan 19 2022 at 19:01):

yeah,

map!(x) do xi
    @set! xi.a=10
    @set! xi.b=300
end

should work

view this post on Zulip Expanding Man (Jan 19 2022 at 19:02):

I think you're looking for

    for (a, n) ∈ zip(vals, ind)
        setproperty!(xi, n, a)
    end

view this post on Zulip Mason Protter (Jan 19 2022 at 19:02):

Setproperty won't work for immutable structs

view this post on Zulip Expanding Man (Jan 19 2022 at 19:03):

Sorry, I just noticed that this was immutable, yeah you need to use whatever is the equivalent from Setfield

view this post on Zulip Expanding Man (Jan 19 2022 at 19:03):

or do it by explicitly creating new structs

view this post on Zulip Mason Protter (Jan 19 2022 at 19:03):

Here is what @set! does:

julia> @macroexpand @set! xi.a = 10
quote
    #= /home/mason/.julia/packages/Setfield/NshXm/src/sugar.jl:191 =#
    var"#54###lens#274" = (identity)((Setfield.compose)((Setfield.PropertyLens){:a}()))
    #= /home/mason/.julia/packages/Setfield/NshXm/src/sugar.jl:192 =#
    xi = (Setfield.set)(xi, var"#54###lens#274", 10)
end

so you can just copy and adapt that

view this post on Zulip Mason Protter (Jan 19 2022 at 19:04):

julia> map!(x, x) do xi
           for (prop, val) ∈ (:a => 10, :b => 300)
               l = Setfield.PropertyLens{prop}()
               xi = Setfield.set(xi, l, val)
           end
           xi
       end
3-element Vector{M}:
 M(10, 300, 3)
 M(10, 300, 6)
 M(10, 300, 9)

view this post on Zulip Expanding Man (Jan 19 2022 at 19:04):

If I were doing this I would probably structure it as a map! over a function that takes an M and returns another M.

view this post on Zulip Frank Van Der Meulen (Jan 19 2022 at 19:08):

Mason Protter said:

julia> map!(x, x) do xi
           for (prop, val) ∈ (:a => 10, :b => 300)
               l = Setfield.PropertyLens{prop}()
               xi = Setfield.set(xi, l, val)
           end
           xi
       end
3-element Vector{M}:
 M(10, 300, 3)
 M(10, 300, 6)
 M(10, 300, 9)

Yes, that looks like what I need.

view this post on Zulip Mason Protter (Jan 19 2022 at 19:09):

One thing I'll also note is that if the property being modified isn't known as a compile time constant, this will also incur some hefty performance penalties.

view this post on Zulip Mason Protter (Jan 19 2022 at 19:09):

So you really want your ind to not be a vector, but a Tuple and you ideally want it to be constant propagated into the function.

view this post on Zulip Takafumi Arakaki (tkf) (Jan 19 2022 at 19:13):

There's ConstructionBase.setproperties if you want to update multiple fields in a struct at once https://juliaobjects.github.io/ConstructionBase.jl/dev/#ConstructionBase.setproperties

Setfields.jl and Accessors.jl both use setproperties internally

view this post on Zulip Takafumi Arakaki (tkf) (Jan 19 2022 at 19:13):

Also, if you don't have any GC-managed objects in the struct (= it's Base.datatype_pointerfree), you can also do an evil pointer hack to mutate the field directly... :) Ref: https://github.com/tkf/RecordArrays.jl

view this post on Zulip Mason Protter (Jan 19 2022 at 19:15):

One other thing: If M is an isbits struct, then you have another that might interest you. 

julia> struct N
           a::Int
           b::Int
           c::Int
       end

julia> y = [N(1,2,3), N(4,5,6), N(7, 8, 9)];

julia> Y = reinterpret(reshape, Int, y)
3×3 reinterpret(reshape, Int64, ::Vector{N}) with eltype Int64:
 1  4  7
 2  5  8
 3  6  9

julia> Y[1, :] .= 10;

julia> Y[2, :] .= 300;

julia> Y
3×3 reinterpret(reshape, Int64, ::Vector{N}) with eltype Int64:
  10   10   10
 300  300  300
   3    6    9

julia> y
3-element Vector{N}:
 N(10, 300, 3)
 N(10, 300, 6)
 N(10, 300, 9)

view this post on Zulip Frank Van Der Meulen (Jan 19 2022 at 19:16):

No problem to make it a tuple. I simplified the actual problem a bit. So not sure about the isbits solution.

view this post on Zulip Frank Van Der Meulen (Jan 20 2022 at 08:38):

The actual problem where I wish to use this is slightly more complicated, and there the solution using ConstructionBase does not seem to work, I tried to adapt @Mason Protter code snippet using Setfield.PropertyLens, but failed.

So what is the more complicated problem? The structs M are in fact a field of another struct, say Q.

using ConstructionBase
struct Q
    z
end

The following fails

x = [Q(M(1,2,3)), Q(M(4,5,6)), Q(M(7,8,9))]. # wish to adjust fields a and b of M in all elements of the vector x
tup = (a=10,b=300)
for i in eachindex(x)
   xi =  setproperties(x[i].z, tup)
   x[i] =xi
end
x

ERROR: LoadError: MethodError: Cannot `convert` an object of type M to an object of type Q
Closest candidates are:
  convert(::Type{T}, ::RObject{S}) where {T, S<:Sxp} at ~/.julia/packages/RCall/iMDW2/src/convert/base.jl:1
  convert(::Type{T}, ::T) where T at /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/base/essentials.jl:218
  Q(::Any) at ~/.julia/dev/Bffg_sde/srcold/testscript.jl:217
Stacktrace:
 [1] setindex!(A::Vector{Q}, x::M, i1::Int64)
   @ Base ./array.jl:903
 [2] top-level scope
   @ ~/.julia/dev/Bffg_sde/srcold/testscript.jl:223
in expression starting at /Users/frankvandermeulen/.julia/dev/Bffg_sde/srcold/testscript.jl:221

view this post on Zulip Frank Van Der Meulen (Jan 20 2022 at 09:39):

Nevermind, should have done

for i in eachindex(x)
    x[i] = Q(setproperties(x[i].z, tup))
end

view this post on Zulip Filippos Christou (Mar 26 2022 at 18:34):

That's a little bit weird for me. How is it possible to mutate an unmutable struct ? Does this @set! macro instantiates a whole new struct which replaces the old one ?

view this post on Zulip Mason Protter (Mar 28 2022 at 18:25):

Yes, that's correct.

julia> using Setfield

julia> let nt = (;a = 1, b=2, c=3)
           @macroexpand @set! nt.b = -1
       end
quote
    #= /Users/mason/.julia/packages/Setfield/AS2xF/src/sugar.jl:196 =#
    var"#62###lens#291" = (identity)((Setfield.compose)((Setfield.PropertyLens){:b}()))
    #= /Users/mason/.julia/packages/Setfield/AS2xF/src/sugar.jl:197 =#
    nt = (Setfield.set)(nt, var"#62###lens#291", -1)
end

view this post on Zulip Mason Protter (Mar 28 2022 at 18:26):

So it's basically just making a copy of the struct with one field changed and all others the same, and then rebinding the old variable name to that. It's a quite elegant trick

Last updated: Oct 02 2023 at 04:34 UTC