Rasmus Henningsson (Apr 24 2025 at 13:00): Rasmus Henningsson (Apr 24 2025 at 13:00):If I have a Ptr to an immutable struct, is there an easy way to access a single field?
Example:
struct Foo
x::Int
y::Float64
end
foo = fill(Foo(1,2))
p = pointer(foo)
julia> typeof(p)
Ptr{Foo}
It's easy to do this:
julia> unsafe_load(p)
Foo(1, 2.0)
But what can I do to load p->y directly, without reading the entire struct (which might be large)?
Timothy (Apr 24 2025 at 13:09):How open to hacky ideas are you?
Timothy (Apr 24 2025 at 13:11):Because I have a definition of unsafe_getfield in mind, but it's more than a few lines and not particularly clean
Rasmus Henningsson (Apr 24 2025 at 13:28):I'm open to hacky ideas if it's needed.
(But I would prefer a non-hacky solution if it's possible.)
It seems to me that unsafe_getfieldshould be in Base.
Mason Protter (Apr 24 2025 at 13:34):I've done this before. The tricky part is efficiently getting the Symbol -> fieldnumber mapping at compile time.
function unsafe_loadfield(p::Ptr{T}, i::Int) where {T}
FT = fieldtype(T, i)
offset = fieldoffset(T, i)
pfield = Ptr{FT}(p) + offset
unsafe_load(pfield)
end
@generated function unsafe_loadfield(p::Ptr{T}, s::Symbol) where {T}
names = fieldnames(T)
exprs = map(eachindex(names)) do i
quote
if s == $(QuoteNode(names[i]))
return unsafe_loadfield(p, $i)
end
end
end
Expr(:block, exprs..., :(error("Symbol $s is not a field of type $T")))
end
Oy! I got halfway through writing that :stuck_out_tongue_closed_eyes:
Mason Protter (Apr 24 2025 at 13:36):julia> unsafe_loadfield(p, :x)
1
julia> unsafe_loadfield(p, :y)
2.0
julia> @btime unsafe_loadfield($p, :y)
1.187 ns (0 allocations: 0 bytes)
2.0
julia> unsafe_loadfield(p, :z)
ERROR: Symbol z is not a field of type Foo
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:35
[2] macro expansion
@ ./REPL[25]:6 [inlined]
[3] unsafe_loadfield(p::Ptr{Foo}, s::Symbol)
@ Main ./REPL[25]:1
[4] top-level scope
@ REPL[29]:1
Nice! Thanks!
Notification Bot (Apr 24 2025 at 13:38):Rasmus Henningsson has marked this topic as resolved.
Mosè Giordano (Apr 24 2025 at 21:22):Rasmus Henningsson said:
If I have a
Ptrto an immutable struct, is there an easy way to access a single field?
How do you have a pointer to an immutable struct, which isn't possible? What you showed is a pointer to a vector whose elements are instances of an immutable struct.
Mason Protter (Apr 25 2025 at 04:00):No, it's the pointer of a vector. The pointer he showed points to an immutable struct.
Sundar R (Apr 25 2025 at 04:13):foo = fill(Foo(1,2))
p = pointer(foo)
pointer(array [, index])
Get the native address of an array or string, optionally at a given location
index.
It's a pointer to the "native address of" the vector, which happens to point to the immutable struct here. Does the language guarantee that this will be the case - that pointers to zero dimensional arrays will always be pointers to their element? One could imagine that not being the case (though in practice this might not ever happen).
Sundar R (Apr 25 2025 at 04:14):The docs don't explicitly say it, but the "optionally at a given index" part seems to imply that it will get the address of the element at that index - so maybe it's slightly less hacky if we get the pointer as pointer(foo, 1) instead.
Rasmus Henningsson (Apr 25 2025 at 07:02):My actual use case is that I get the pointer to the struct in a callback from C. So it's a C struct which has a Julia struct equivalent (which is defined in a package, not by me).
Sukera (Apr 25 2025 at 09:41):structs allocated from C should be a mutable struct in Julia, to preserve object identity based on the memory address
Sukera (Apr 25 2025 at 09:42):so if you want to write changes back so that C can see them, you're going to have to keep the Ptr around and only work with that object through that Ptr
Mason Protter (Apr 25 2025 at 10:05):The two things you say there are kinda in conflict with eachother though. You can't wrap an external pointer with a mutable struct, so even if you do load the pointer, and convert its contents to a mutable struct, you still need to separately keep the pointer around the whole time and synchronize any changes you made to it
Mason Protter (Apr 25 2025 at 10:06):In these situations I find it's usually best to not use a mutable struct and just work on the pointer directly instead.
Sukera (Apr 25 2025 at 10:09):right, the second message is referring to the situation where the struct is immutable
Sukera (Apr 25 2025 at 10:10):you're right that you need to keep the pointer around in either situation though, this kind of interop is a bit messy
Mason Protter (Apr 25 2025 at 10:12):I have some utils laying around for basically wrapping a pointer and treating it as a mutable struct, turning getproperty/setproperty! calls into unsafe_load/ unsafe_store!s. I should maybe publish that at some point
Rasmus Henningsson (Apr 25 2025 at 12:06):@Sukera Yes I agree that the interop is messy. I don't think there's any great solution here, since C and Julia do some things differently. But I'm fine with that and can keep my interaction with the pointers to a minimum in isolated code, so it doesn't get very ugly.
Mason Protter said:
I have some utils laying around for basically wrapping a pointer and treating it as a mutable struct, turning
getproperty/setproperty!calls intounsafe_load/unsafe_store!s. I should maybe publish that at some point
That might be the solution for my use case, and seems very nice in general. The solution you posted above (together with a corresponding unsafe_storefield!) works very well for me at the moment though.
Mason Protter (Apr 25 2025 at 13:14):I just found out there actually is a Base.fieldindex function, so the @generated function I wrote is a bit unnecessary. It can just be
function unsafe_loadfield(p::Ptr{T}, s::Symbol) where {T}
i = Base.fieldindex(T, s, true)
unsafe_loadfield(p, i)
end
(opened a PR to mark it public here: https://github.com/JuliaLang/julia/pull/58228)
Mason Protter (Apr 25 2025 at 13:38):ugh, sorry, no you should not use fieldindex here as it seems to block constant propagation somehow here and make the function very slow
Mason Protter (Apr 25 2025 at 13:38):It's weird because fieldindex is fast, but putting it in the body of unsafe_loadfield is slow somehow.
Mason Protter (Apr 25 2025 at 13:39):so yeah, you're better off with the @generated function.
Rasmus Henningsson (Apr 25 2025 at 13:53):Thanks for figuring all these things out!
Last updated: May 17 2025 at 04:42 UTC