Stream: helpdesk (published)

Topic: Woes with `esc` and non-recursive `macroexpand`


view this post on Zulip Rafael Fourquet (May 06 2021 at 10:14):

Given a recursive macro @m containing esc, macroexpand(Main, :(@m x), recursive=true) works as expected, but not macroexpand(Main, macroexpand(Main, :(@m x), recursive=false)), which is not cleaned up of :escape expressions.
The simplest is to give an example:

julia> macro m(x, rec=true)
           if !rec
               quote
                   Z = 1
                   $(esc(x))
               end
           else
               quote
                   @m begin
                       Z = 2
                       $(esc(x))
                   end false
               end
           end
       end
@m (macro with 2 methods)

julia> ex = :(Z = 0; @m Z);

julia> macroexpand(Main, ex)
quote
    Z = 0
    #= REPL[2]:1 =#
    begin
        #= REPL[1]:9 =#
        begin
            #= REPL[1]:4 =#
            var"#1#Z" = 1
            #= REPL[1]:5 =#
            begin
                #= REPL[1]:10 =#
                var"#1#Z" = 2
                #= REPL[1]:11 =#
                Z
            end
        end
    end
end

julia> macroexpand(Main, ex, recursive=false)
quote
    Z = 0
    #= REPL[2]:1 =#
    begin
        #= REPL[1]:9 =#
        #= REPL[1]:9 =# @m begin
                #= REPL[1]:10 =#
                Z = 2
                #= REPL[1]:11 =#
                $(Expr(:escape, :Z))
            end false
    end
end

julia> macroexpand(Main, macroexpand(Main, ex, recursive=false))
quote
    Z = 0
    #= REPL[2]:1 =#
    begin
        #= REPL[1]:9 =#
        begin
            #= REPL[1]:4 =#
            var"#5#Z" = 1
            #= REPL[1]:5 =#
            begin
                #= REPL[1]:10 =#
                Z = 2
                #= REPL[1]:11 =#
                $(Expr(:escape, :Z))
            end
        end
    end
end

julia> eval(macroexpand(Main, macroexpand(Main, ex, recursive=false)))
ERROR: syntax: invalid syntax (escape (outerref Z))
Stacktrace:
[...]

Note the :escape present in the last expansion, but also the fact that there is a non-gensymed Z = 2, and in particular that I then can't simply unwrap manually the remaining :escape expression.

I have to use macroexpand with recursive=false because I also need to process some macro invocations more specifically.
Any idea if the behavior above is expected, and if there is a war around for my use case?

view this post on Zulip Rafael Fourquet (May 06 2021 at 10:26):

Actually there is a closed issue stating about "sequential partial macro expansion" and "full macro expansion at once" that

No, they are not currently supposed to produce the same result. (Jameson)

That's unfortunate, but still wondering whether there would be a workaround...

view this post on Zulip Takafumi Arakaki (tkf) (May 06 2021 at 22:36):

Not sure if it works for your use-case, but a possible workaround is a manual hygiene

julia> macro m2(x, rec=true)
           @gensym Z
           if !rec
               quote
                   $Z = 1
                   $x
               end
           else
               quote
                   $(@__MODULE__).@m2 begin
                       $Z = 2
                       $x
                   end false
               end
           end |> esc
       end
@m2 (macro with 2 methods)

julia> ex2 = :(Z = 0; @m2 Z);

julia> eval(macroexpand(Main, macroexpand(Main, ex2, recursive=false)))
0

Note that $Z is not shared between macro invocations and @m2 has a subtle difference to @m:

julia> macroexpand(Main, ex)
quote
    Z = 0
    #= REPL[6]:1 =#
    begin
        #= REPL[5]:9 =#
        begin
            #= REPL[5]:4 =#
            var"#4#Z" = 1
            #= REPL[5]:5 =#
            begin
                #= REPL[5]:10 =#
                var"#4#Z" = 2
                #= REPL[5]:11 =#
                Z
            end
        end
    end
end

julia> macroexpand(Main, ex2)
quote
    Z = 0
    #= REPL[2]:1 =#
    begin
        #= REPL[1]:10 =#
        begin
            #= REPL[1]:5 =#
            var"##Z#269" = 1
            #= REPL[1]:6 =#
            begin
                #= REPL[1]:11 =#
                var"##Z#268" = 2
                #= REPL[1]:12 =#
                Z
            end
        end
    end
end

Observe that macroexpand(Main, ex) has var"#4#Z" = 1 and var"#4#Z" = 2 that assign to an identical variable while macroexpand(Main, ex2) hasvar"##Z#269" = 1 and var"##Z#268" = 2 that assign to different variables.

To share gensym'ed variable between macro invocations, I think you'd need something like this:

julia> macro static_gensym(x::Symbol)
           s = gensym(x)
           esc(:($x = $(QuoteNode(s))))
       end
@static_gensym (macro with 3 methods)

julia> macro m3(x, rec=true)
           @static_gensym Z
           if !rec
               quote
                   $Z = 1
                   $x
               end
           else
               quote
                   $(@__MODULE__).@m3 begin
                       $Z = 2
                       $x
                   end false
               end
           end |> esc
       end
@m3 (macro with 2 methods)

julia> macroexpand(Main, ex3)
quote
    Z = 0
    #= REPL[12]:1 =#
    begin
        #= REPL[16]:10 =#
        begin
            #= REPL[16]:5 =#
            var"##Z#265" = 1
            #= REPL[16]:6 =#
            begin
                #= REPL[16]:11 =#
                var"##Z#265" = 2
                #= REPL[16]:12 =#
                Z
            end
        end
    end
end

view this post on Zulip Rafael Fourquet (May 10 2021 at 12:51):

Thanks, this is definitely interesting! Although this doesn't really work for my use case, as macro(s) @m is supposed to be written by users, which can't be expected to dive into these subtleties. I think I will just document for now that esc is not supported in their macros.


Last updated: Nov 06 2024 at 04:40 UTC