Stream: helpdesk (published)

Topic: Modeling efficiency - sparse states and heterogeneous types

Alec (Nov 18 2025 at 22:09):

I am working on a problem where I would like parts of a model to emit small immutable results, but they aggregate into something with a lot more fields. The states depend on a collection of heterogeneous actor types. My two questions:

1. Say I've got a 'dense' set of state deltas but each of my underlying model components generally uses a subset (< half) of the overall states. Is this an efficient way to do this immutably? I think the alternative is to define the components as mutable and then have each agent sequentially modify them. I'm looking for efficiency due to creating millions of these components in a simulation.
2. The collection I'm operation on has heterogeneous types. I explored SumTypes and enum alternatives, but since I dispatch on the type (Actors below) those approaches didn't seem to work for me. Is there a recommended pattern instead? E.g. a Dict Dict(Actor1 => [...Actor1s...], Actor2 => [...Actor2s...],...)?

basic example:

import Base: +

# state components really represent the *change* in system state
abstract type AbstractComponent end

Base.@kwdef struct ComponentA <: AbstractComponent
   a::Float64=0.0
   b::Float64=0.0
   c::Float64=0.0
   d::Float64=0.0
   e::Float64=0.0
   f::Float64=0.0
   g::Float64=0.0
end

Base.@kwdef struct ComponentB <: AbstractComponent
   h::Float64=0.0
   i::Float64=0.0
   j::Float64=0.0
   k::Float64=0.0
   l::Float64=0.0
   m::Float64=0.0
   n::Float64=0.0
end

Base.@kwdef struct ComponentC <: AbstractComponent
   o::Float64=0.0
   p::Float64=0.0
   q::Float64=0.0
   r::Float64=0.0
   s::Float64=0.0
   t::Float64=0.0
   u::Float64=0.0
   v::Float64=0.0
   w::Float64=0.0
end

struct ModelState
    compA::ComponentA
    compB::ComponentB
    compC::ComponentC
end

Base.:+(a::T, b::T) where {T<:AbstractComponent} = T([(getfield(a, f) + getfield(b, f)) for f in fieldnames(T)]...)
Base.:+(a::T, b::T) where {T<:ModelState} = T([(getfield(a, f) + getfield(b, f)) for f in fieldnames(T)]...)


abstract type AbstractActor end
struct Actor1 end
struct Actor2 end
struct Actor3 end

evolve(::Actor1) = ModelState(ComponentA(c = rand(),d = rand()),ComponentB(i = rand(),n = rand()), ComponentC(o = rand(),v = rand()))
evolve(::Actor2) = ModelState(ComponentA(a = rand(),f = rand()),ComponentB(l = rand(),m = rand()), ComponentC(o = rand(),w = rand()))
evolve(::Actor3) = ModelState(ComponentA(a = rand(),b = rand()),ComponentB(l = rand(),j = rand()), ComponentC(s = rand(),w = rand()))

heterogeneous_actors = [Actor1(), Actor2(), Actor3()]

state_deltas = evolve.(heterogeneous_actors)

# the evolution of the system state through time
states = cumsum(state_deltas)

I have the components and agents in separate because in my real code I dispatch behavior (e.g. components enforce some relationships between fields, agents actually have more complex behavior, custom show methods for the different types). In the real problem, I have 10'000s of actors and 1000's of time steps.

Gunnar Farnebäck (Nov 19 2025 at 06:49):

Is https://discourse.julialang.org/t/ann-ark-jl-archetype-based-entity-component-system-ecs-for-games-and-simulations/133851 of any relevance to this problem?

Alec (Nov 19 2025 at 16:20):

@Gunnar Farnebäck Thanks, this is very interesting. I've spent some time looking at this and reading up on ECS systems. From what I can tell, Ark.jl/ECS systems benefit modeling when you are trying to control/update components depending on their group membership or properties (accomplished via Query). E.g. only update entities with a position property. I think my use case/objective is a bit different.

My actual use case has every entity get updated each tilmestep. The related output (I called it component) just is kind of sparse. My actual use case is related to JuliaActuary -relating the MWE example above to what I'm actually trying to do is model a block of business for an insurance company:

• Components are financial statements where I am trying to enforce some properties (akin to double entry accounting)
• Actors are assets/liabilities
• Every tilmestep something happens to both assets and liabilities (e.g. a bond pays a coupon, a policy liability pays a premium or has a claim) so in all of the components of the financial statement what each actor (asset/liability) touches is fairly small subset but summing up everything gets you a complete, populated set of financial statements.

Gunnar Farnebäck (Nov 19 2025 at 16:28):

I have no more insights to share since I've never worked with these kinds of problems. It just seemed like something that could be in the same domain.

cschen (Nov 19 2025 at 17:02):

Given that you only update a couple of fields at a time, I’d suggest keep them immutable and use Accessors to efficiently generate new objects.

Regarding the heterogenous actors why didn’t sum types work (Or any of the alternatives)? It’s kind of exactly what you need in this case: A closed union such that the compiler can union split efficiently. (By the way it could also help with the abstract components probably)

Are the updated fields in the components always the same for each actor?

Alec (Nov 24 2025 at 04:55):

For sum types, I wanted users to be able to declare their own subtypes and inherit methods if possible, otherwise define their own. AFAICT sum types doesn’t allow for this.

Yes, the components are always the same for each actor.

Mason Protter (Nov 24 2025 at 11:06):

The way I handle stuff like this in GraphDynamics.jl is to partition the list of actors by their type. E.g. suppose you had

heterogeneous_actors = [Actor1(a=1), Actor2(h=1), Actor1(b=1), Actor3(o=1), Actor2(i=1), Actor3(p=1), Actor1(c=1)]

then as a setup-step before the actual silulation, I would do

function partition_actors(actors)
    # Contain all the type instability into this step!
    types = Tuple(unique((typeof(actor) for actor in actors)))
    map(types) do T
        filter(actor -> actor isa T, actors)
    end
end

actors_partitioned = partition_actors(heterogeneous_actors)

#+Results
([Actor1(), Actor1()], [Actor3(), Actor3(), Actor3(), Actor3(), Actor3(), Actor3()], [Actor2(), Actor2()])

Now you have an object you can work with in a type-stable, efficient manner later on.

Mason Protter (Nov 24 2025 at 11:08):

This approach might be overkill for your needs though. I did it that way because the equivalent of the states and components were also unique per-actor, and I also needed to support arbitrary, but type stable communication between each different actor

Mason Protter (Nov 24 2025 at 11:17):

If you want a Sum-Types-like approach, but where users can still make their own types and you don't want to have to manually enumerate all possible types up-front, you could do it like this:

struct ActorWrapper{PossibleTypes}
    actor::PossibleTypes
end
unwrap(aw::ActorWrapper) = aw.actor
function wrap_actors(actors)
     # Contain all the type instability into this step!
    possible_types = Union{unique((typeof(actor) for actor in actors))...}
    ActorWrapper{possible_types}.(actors)
end

actors_wrapped = wrap_actors(heterogeneous_actors)

state_deltas = evolve.(unwrap.(actors_wrapped))
states = cumsum(state_deltas)

I suspect that's good enough for your use-case here since all the ModelStates are of the same type.

Mason Protter (Nov 24 2025 at 11:39):

That said, I tried some benchmarks with various sizes, and with adding more and more different actor types, and I found that while both of my suggestions sped up the evolve.(...) step, those speedups were irrelevant because the cumsum operation took orders of magnitude more time.

Last updated: Nov 27 2025 at 04:44 UTC