Stream: helpdesk (published)

Topic: ✔ `Type{Type{` showing up in `@generated` function

Mason Protter (Jan 25 2022 at 23:15):

Not sure if this is a bug, but here is a MWE:

Keys(::Type{NamedTuple{names, data}}) where {names, data} = names
@generated function foo(x)
    Core.println(Keys(x))
    nothing
end

foo(NamedTuple{(:a, :b)})

#+RESULTS:
MethodError: no method matching Keys(::Type{Type{NamedTuple{(:a, :b)}}})

Anyone know what might be happening here? Should I open an issue?

Mason Protter (Jan 25 2022 at 23:17):

I find it very bizarre that Type{Type{NamedTuple{(:a, :b)}}} got constructed, that seems like a bug.

Mason Protter (Jan 25 2022 at 23:21):

versioninfo()

#+RESULTS:
: Julia Version 1.7.1
: Commit ac5cc99908* (2021-12-22 19:35 UTC)
: Platform Info:
:   OS: Linux (x86_64-pc-linux-gnu)
:   CPU: AMD Ryzen 5 2600 Six-Core Processor
:   WORD_SIZE: 64
:   LIBM: libopenlibm
:   LLVM: libLLVM-12.0.1 (ORCJIT, znver1)
: Environment:
:   JULIA_NUM_THREADS = 6

Dilum Aluthge (Jan 25 2022 at 23:24):

That seems like the correct behavior, right?

Dilum Aluthge (Jan 25 2022 at 23:28):

If you call foo(a), then "inside" the generated function, x = typeof(a), right?

Dilum Aluthge (Jan 25 2022 at 23:32):

You called foo(a) where a = NamedTuple, and thus typeof(a) = Type{NamedTuple}.

So inside the generated function, we have x = typeof(a) = Type{NamedTuple}, right?

So, now, inside the generated function, you call Keys(x).

Dilum Aluthge (Jan 25 2022 at 23:34):

If x = Type{NamedTuple}, and you call Keys(x), then what should the type signature of that Keys method be?

It should be Keys(::Type{Type{NamedTuple}}) = ..., right?

Mason Protter (Jan 25 2022 at 23:35):

I guess I've just never experienced that before when passing types to generated functions. I thought that Type just existed so that we didn't get DataType, so I thought we didn't double wrap things in Type

Dilum Aluthge (Jan 25 2022 at 23:37):

Well, we need to be able to distinguish between e.g. foo(NamedTuple()) and foo(NamedTuple), right?

Mason Protter (Jan 25 2022 at 23:37):

That's not the same thing

Takafumi Arakaki (tkf) (Jan 25 2022 at 23:38):

It'd be confusing if a @generated function changes its behavior when the argument is a type or not. So, I'd say the current behavior of @generated is correct and "ergonomic."

Mason Protter (Jan 25 2022 at 23:38):

We generally don't want to distinguish between Type{T} and T when T is a type.

Takafumi Arakaki (tkf) (Jan 25 2022 at 23:39):

If you know the argument is a type, you can use @generated foo(::T) where {T} .... Then T is the same for generator code and the runtime code.

Dilum Aluthge (Jan 25 2022 at 23:40):

julia> NamedTuple isa Type{NamedTuple}
true

julia> Type{NamedTuple} isa NamedTuple
false

Mason Protter (Jan 25 2022 at 23:40):

Huh. Today I learned that Core.Typeof(Type{T}) gives Type{Type{T}}. For some reason I thought I had remembered that we didn't do this

Mason Protter (Jan 25 2022 at 23:42):

Takafumi Arakaki (tkf) said:

If you know the argument is a type, you can use @generated foo(::T) where {T} .... Then T is the same for generator code and the runtime code.

Yeah, that's what I normally do, but the use case where I ran into this, I had a variable number of heterogeneous arguments so that wasn't an option.

Takafumi Arakaki (tkf) (Jan 25 2022 at 23:42):

It is important to note that T and Type{T} are different.

julia> 1 isa Int
true

julia> 1 isa Type{Int}
false

Type{T} is a type of T

julia> Int isa Type{Int}
true

julia> Int isa DataType
true

julia> Type{Int} <: DataType
true

Mason Protter (Jan 25 2022 at 23:42):

Yeah, makes sense, thanks for the clarification

Takafumi Arakaki (tkf) (Jan 25 2022 at 23:43):

Type{T} is useful because it is a singleton representation of "type of T"

Mason Protter (Jan 25 2022 at 23:43):

I guess I'll need to splat the arguments, then have an inner method that I pass the arguments to as a tuple.

Takafumi Arakaki (tkf) (Jan 25 2022 at 23:48):

Ahh... yes, I remember that vararg is a bit weird inside @generated. I'd probably define a function to unwrap the Type

julia> @generated f(xs...) = (global XS = xs; nothing);

julia> f(Int, Nothing)

julia> XS
(Type{Int64}, Type{Nothing})

julia> instanceof(::Type{Type{T}}) where {T} = T;

julia> map(instanceof, XS)
(Int64, Nothing)

Mason Protter (Jan 25 2022 at 23:55):

Yeah, good idea. I'll just build that into Keys.

Mason Protter (Jan 25 2022 at 23:57):

I was fiddling again today with something @Brian Chen asked about yesterday, and the part I left as an 'exercise to the reader' was a bit more frustrating than I had anticipated, so here it is in case you want it Brian. A vararg complement to Base.structdiff that isn't slow at runtime:

Keys(::Type{<:NamedTuple{names}}) where {names} = names
Keys(::Type{Type{T}}) where {T} = Keys(T)

@generated function structintersect(a::NamedTuple{an}, rest::Union{NamedTuple, Type{<:NamedTuple}}...) where {an}
    names = Tuple(intersect(an, Keys.(rest)...))
    data  = Expr(:tuple, (:(getproperty(a, $(QuoteNode(name)))) for name in names)...)
    :(NamedTuple{$names}($data))
end

structintersect((;a=1, b=2, c=3, d=4), (;b=1, c=3, d=4), NamedTuple{(:b, :d)})

#+RESULTS:
: (b = 2, d = 4)

Notification Bot (Jan 26 2022 at 00:12):

Mason Protter has marked this topic as resolved.

Mason Protter (Jan 26 2022 at 00:41):

Think I should bother trying to open a PR to Base with this? I suspect Base.structdiff only exists because they have a need to use it internally

Takafumi Arakaki (tkf) (Jan 26 2022 at 01:39):

It would have been nice if we had a uniform dict/set API so that we could motivate this a bit better as a prerequisite of intersect-on-NamedTuple

Takafumi Arakaki (tkf) (Jan 26 2022 at 01:40):

Not that I'm against it or anything. I just don't know if it's OK or not

Mason Protter (Jan 26 2022 at 02:07):

That’s my feeling also.

Last updated: Oct 02 2023 at 04:34 UTC