skip GC and manage allocation for object manually · helpdesk (published)

Is there a way to manually take care of the allocation and deallocation of some object, making GC not be aware of it?

The use case is the following: I have a program that depends on a lower-level C++ library, but that library generates some data, which on the Julia side is wrapped in a struct, that I have to clean very often, otherwise the memory blows up. However, if I call GC manually all the time, the program becomes way too slow. I would like to "manually GC" only that specific object. Since it is an object that wraps an underlying C++ data structure, I cannot just overwrite the data.

(at a more lower level the issue is that the C++ code is leaking memory, but the bug is being tracked downstream but while it is not solved I would like to take care of the issue on my side).

Gunnar Farnebäck (Jun 24 2025 at 08:25):

I have a hard time figuring out how your data is represented and why wrapping C++ data causes memory problems on the Julia side. Is it possible to describe this with reasonably short code?

Leandro Martínez (Jun 24 2025 at 11:36):

Yeah, its somewhat hard to explain because we are trying to figure out a workaround for a bug we do not have full control of, which has its own issues but relates to other side-effects.

But, in summary, I have a situation where I have to read a huge data file, sequentially (the positions of atoms in a simulation). There is a package, Chemfiles.jl, which wraps a C++ implementation of the reading of various formats for this file.

for iframe in 1:nframes
     positions = read_frame(trajectory_file)
     # do cool stuff with the positions
end

at each iteration of the loop the positions object must be cleared, because they are huge and storing more than a few of those in memory crashes the computation.

The garbage collector is not being effective in this process. Sometimes memory builds up and the computation crashes. Because of that, I have in my code something like after reading each frame:

            if Sys.free_memory() / Sys.total_memory() < 0.1
                GC.gc()
            end

where I call GC.gc() whenever the free memory is smaller than a threshold. This alleviates the issue, and crashes are less frequent. The issue (at this point) is that if I call GC.gc() too often (by using a large threshold), the code becomes much slower. Something from taking 2 minutes to taking 30 minutes, only because of calling the garbage collector at every step.

And there is where I thought that a workaround could be to manually managing the memory of the positions object independently of the GC. That manually management would be easy except because the positions object is a wrapper of an underlying C++ object created by the Chemfiles library.

Any insight is appreciated. In any case explaning the issue helps me figuring out what might be done.

Mason Protter (Jun 24 2025 at 12:44):

Mason Protter (Jun 24 2025 at 12:46):

One thing I've done before for handling mutable data separately from Julia's GC is this:

module PtrStructs

struct PtrStruct{T}
    ptr::Ptr{T}
    function PtrStruct(p::Ptr{T}) where {T}
        if isbitstype(T)
            new{T}(p)
        else
            error("PtrStruct only works with isbits types.")
        end
    end
end
Base.pointer(p::PtrStruct) = getfield(p, :ptr)

function Base.getindex(p::PtrStruct{T}) where {T}
    unsafe_load(pointer(p))
end
function Base.setindex!(p::PtrStruct{T}, x) where {T}
    unsafe_store!(pointer(p), convert(T, x)::T)
end

@generated function Base.getproperty(p::PtrStruct{T}, s::Symbol) where {T}
    foldl(1:fieldcount(T), init=:(invalid_propertyname_error(T, s))) do ex, i
        name = fieldname(T, i)
        offset = fieldoffset(T, i)
        type = fieldtype(T, i)
        Expr(
            :if,
            :(s === $(QuoteNode(name))),
            :(unsafe_load(Ptr{$type}(pointer(p) + $offset))),
            ex
        )
    end
end

@generated function Base.setproperty!(p::PtrStruct{T}, s::Symbol, x) where {T}
    foldl(1:fieldcount(T), init=:(invalid_propertyname_error(T, s))) do ex, i
        name = fieldname(T, i)
        offset = fieldoffset(T, i)
        type = fieldtype(T, i)
        Expr(
            :if,
            :(s === $(QuoteNode(name))),
            :(unsafe_store!(Ptr{$type}(pointer(p) + $offset), convert($type, x)::$type)),
            ex
        )
    end
end

@noinline invalid_propertyname_error(::Type{T}, s::Symbol) where {T} = throw(ArgumentError("Invalid propertyname $s for object of type $T"))

end # module PtrStructs

Mason Protter (Jun 24 2025 at 12:47):

and then you can basically bring your own pointer, make your own objects with setproperty! and then free the pointer later when you're done with it. i.e. old fashioned memory management with all it's upsides and downsides

Gunnar Farnebäck (Jun 24 2025 at 14:52):

C++ memory is indeed wrapped by a mutable struct which automatically frees the memory from a finalizer when it is garbage collected. But Julia doesn't see that memory, so it doesn't put any pressure on the GC to collect garbage.

Gunnar Farnebäck (Jun 24 2025 at 15:01):

If it is safe to call lib.chfl_free twice, you could possibly work around this by searching your objects for CxxPointers and manually freeing them before letting your objects go out of scope.

Gunnar Farnebäck (Jun 24 2025 at 15:13):

help?> Base.finalize
  finalize(x)

  Immediately run finalizers registered for object x.

Leandro Martínez (Jun 24 2025 at 16:38):

Uhm, that's interesting. Actually my function reads the data and copies it to a Julia data structure right away, and the objects get out of scope in a very localized way. I added the finalize() calls after returning from the function. Is that what you mean by that? (I'm not sure if this makes sense, and by testing it didn't really seem to work, at first sight):

function nextframe!(trajectory::ChemFile{T}) where {T}

    st = stream(trajectory)

    frame = Chemfiles.read(st)
    positions = Chemfiles.positions(frame)
    uc0 = Chemfiles.UnitCell(frame)
    ucm = Chemfiles.matrix(uc0)
    trajectory.unitcell .= transpose(SMatrix{3,3}(ucm))

    # Save coordinates of solute and solvent in trajectory arrays (of course this could be avoided,
    # but the code in general is more clear aftwerwards by doing this)
    for i in eachindex(trajectory.x_solute)
        trajectory.x_solute[i] = T(
            positions[1, trajectory.solute.indices[i]],
            positions[2, trajectory.solute.indices[i]],
            positions[3, trajectory.solute.indices[i]],
        )
    end
    for i in eachindex(trajectory.x_solvent)
        trajectory.x_solvent[i] = T(
            positions[1, trajectory.solvent.indices[i]],
            positions[2, trajectory.solvent.indices[i]],
            positions[3, trajectory.solvent.indices[i]],
        )
    end
    Base.finalize(uc0)
    Base.finalize(ucm)
    Base.finalize(positions)
    Base.finalize(frame)

    return trajectory
end

(In this code ChemFile is a standard Julia type manage myself, with all Julia structures except the trajectory "stream" (stream::Stream{<:Chemfiles.Trajectory}).

Leandro Martínez (Jun 24 2025 at 16:55):

In parallel, to solve the underlying issue in the Chemfiles.jl package, what would be the proper way to wrap a C++ object such that Julia does see its size? I guess this is a common scenario.

Neven Sajko (Jun 24 2025 at 21:31):

I don't think finalize is a very useful function, certainly not in the example code immediately above. finalize just runs the registered finalizers. I think this is useless if these objects (uc0, ucm, etc.) are unreachable at the end of the function, because the finalizers get run anyway when the objects get garbage collected.

Leandro Martínez (Jun 24 2025 at 21:44):

Is there a way to garbage collect those specific objects before leaving the function?

Leandro Martínez (Jun 24 2025 at 21:50):

Luthaf (Jun 24 2025 at 23:03):

One possible reason why finalize does not help is that we are keeping a reference to the frame inside the positions array (otherwise julia could access freed memory). Maybe first setting positions to nothing would help. Does anyone knows if finalize runs the finalizers regardless of the reference count of the object?

I'm also not 100% sure the issue is that the GC does not see memory pressure, because I tried to replace the C++ allocators with jl_malloc and jl_free, which AFAIU should be enough for the GC to know more about the actual memory pressure, but also did not help in my testing.

Luthaf (Jun 24 2025 at 23:06):

I'm not sure I follow you here: from my understanding Julia GC (contrary to e.g. Python's GC which only uses reference counting) does not immediately collect non-reachable objects, and there can be a large delay between the object being unreachable and the object being collected. But finalize not helping is also strange, so I might be missing something here …

Leandro Martínez (Jun 25 2025 at 00:33):

FWIW, I just tried setting all objects to nothing before calling finalize on each one in the code above, but didn't see any improvement.

(also, I remember being able to reproduce the issue in the Python interface of Chemfiles, but that might have nothing to do with the issue here)

Leandro Martínez (Jun 25 2025 at 00:35):

Also I do not think that GC is completely unaware of the memory of each positions object. It does free memory, and calling it more frequently alleviates the issue. But for some reason something ends up leaking, but it is not the full memory of the object, for sure.

Neven Sajko (Jun 25 2025 at 13:12):

Neven Sajko (Jun 25 2025 at 13:16):

Taking a step back here: as far as I understand, it's still not known for certain what is leaking? Perhaps inspecting heap snapshots might be a good first step?

Neven Sajko (Jun 25 2025 at 14:03):

Even though the heap snapshots only show what's tracked by GC, maybe it could be useful.

Stream: helpdesk (published)

Topic: skip GC and manage allocation for object manually

Leandro Martínez (Jun 24 2025 at 00:19):