Stream: helpdesk (published)

Topic: Zygote.jl breaking precompilation in several packages

Júlio Hoffimann (Jan 20 2025 at 11:53):

In a fresh environment I can no longer install packages I maintain, and it seems that recent changes in Zygote/extensions were made a patch when they should've been breaking?

image.png

Could you reproduce the issue in a fresh environment with ]add GeoStats?

Júlio Hoffimann (Jan 20 2025 at 11:54):

In this case the precompilation never ends due to some unfinished state in Zygote extensions.

Júlio Hoffimann (Jan 20 2025 at 12:03):

The problem is even worse than I thought. Any package that depends on Zygote.jl is failing to compile:

] add CoordRefSystems

Mosè Giordano (Jan 20 2025 at 12:18):

Delete zygote compile cache in compiled/v1.whatever/Zygote

Júlio Hoffimann (Jan 20 2025 at 12:22):

Will give it a try. Wonder what is the cause of the issue though. Probably affecting many end-users out there.

Júlio Hoffimann (Jan 20 2025 at 12:23):

Mosè Giordano said:

Delete zygote compile cache in compiled/v1.whatever/Zygote

The problem remains.

Sundar R (Jan 20 2025 at 12:58):

Can confirm on my Julia 1.11.1 on Windows, ] add CoordRefSystems results in:

┌ Zygote
│  WARNING: Method definition unthunk_tangent(ChainRulesCore.AbstractThunk) in module ZygoteRules at C:\Users\Sundar\.julia\packages\ZygoteRules\KF1B6\src\adjoint.jl:40 overwritten in module Zygote at C:\Users\Sundar\.julia\packages\Zygote\59YyM\src\compiler\chainrules.jl:4.
│  ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
└

Júlio Hoffimann (Jan 20 2025 at 13:02):

Opened an issue on Zygote. It is not good that compilation failures can propagate to downstream projects like that.

Júlio Hoffimann (Jan 20 2025 at 13:02):

Could CI work avoid similar issues in the future?

Sundar R (Jan 20 2025 at 13:29):

This seems to be a bad interaction between Zygote and ZygoteRules, from what I can tell. Adding a CI test where ZygoteRules is added to the environment seems sensible, since it's a closely related package anyway and this interaction between packages is bound to come up a lot (like it has here).

Sundar R (Jan 20 2025 at 13:33):

It looks like ZygoteRules does have that integration test in its CI, and it has been failing since the last commit to ZygoteRules 7 hours ago.

Mosè Giordano (Jan 20 2025 at 14:44):

Júlio Hoffimann said:

It is not good that compilation failures can propagate to downstream projects like that.

I mean, if you can't precompile a package it's kind of normal you can't precompile other packages which need it

Júlio Hoffimann (Jan 20 2025 at 14:50):

Yes, so the issue is allowing a patch release to propagate with compilation failure?

Last updated: Jan 29 2025 at 04:38 UTC