Stream: helpdesk (published)

Topic: Repeated precompilation on using


view this post on Zulip Alexander Sikorski (Mar 22 2024 at 22:39):

I just went through 15 minutes of precompiling around 400 Packages. (]precompile after cleaning .julia/compiled)
Now when using myPackage (from its activated environment) it just goes through precompilation of 35 packages again.
Why is this happening?

view this post on Zulip Fredrik Ekre (Mar 22 2024 at 22:40):

Where/what did you precompile the first time?

view this post on Zulip Alexander Sikorski (Mar 22 2024 at 22:42):

Fredrik Ekre said:

Where/what did you precompile the first time?

From the packages environment. (I am sane enough not to have 400 packages in my Base environment :D)

view this post on Zulip Fredrik Ekre (Mar 22 2024 at 22:43):

Not sure, would need a reproducer.

view this post on Zulip Alexander Sikorski (Mar 22 2024 at 22:46):

https://github.com/axsk/ISOKANN.jl

view this post on Zulip Alexander Sikorski (Mar 22 2024 at 22:48):

I am trying to get it usinged for far more then an hour by now, just looking at precompilation all the time :(

view this post on Zulip Alexander Sikorski (Mar 22 2024 at 22:49):

Currently stuck at

julia> @time @time_imports using ISOKANN
Precompiling ISOKANN
  32 dependencies successfully precompiled in 781 seconds. 395 already precompiled.
[ Info: Precompiling ISOKANN [512544ac-929d-4ba3-aa9d-55e9313a4b47]

view this post on Zulip Alexander Sikorski (Mar 22 2024 at 22:50):

I don't even know how much time I spent on the last trial only to get an error because I had two includes in the wrong order :/

view this post on Zulip Alexander Sikorski (Mar 22 2024 at 22:50):

Would really appreciate if someone could have a look at this and help me

view this post on Zulip Alexander Sikorski (Mar 22 2024 at 22:52):

julia> versioninfo()
Julia Version 1.10.2
Commit bd47eca2c8a (2024-03-01 10:14 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (x86_64-apple-darwin22.4.0)
  CPU: 4 × Intel(R) Core(TM) i5-7267U CPU @ 3.10GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, skylake)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Environment:
  JULIA_REVISE_INCLUDE = 1

view this post on Zulip Fredrik Ekre (Mar 22 2024 at 22:57):

That env doesn't even instantiate for me (segfault in CUDA instantiation). Try stripping away deps until you just have enough to repro perhaps.

view this post on Zulip Alexander Sikorski (Mar 22 2024 at 22:58):

Which takes me around 30 mins for every trial.. (still stuck with precompiling...)

view this post on Zulip Alexander Sikorski (Mar 22 2024 at 23:12):

The result of @time using ISOKANN
1823.583727 seconds (22.52 M allocations: 1.451 GiB, 0.08% gc time, 0.20% compilation time: 78% of which was recompilation)

view this post on Zulip Alexander Sikorski (Mar 22 2024 at 23:14):

Calling it a day :(

view this post on Zulip Mason Protter (Mar 22 2024 at 23:44):

what happens if you do julia --startup=no and then using the package?

view this post on Zulip Mason Protter (Mar 22 2024 at 23:50):

For what it's worth, I get an error during precompilation:

julia> using ISOKANN
Precompiling ISOKANN
[ Info: Precompiling ISOKANN [512544ac-929d-4ba3-aa9d-55e9313a4b47]
ERROR: LoadError: UndefVarError: `Iso2` not defined
Stacktrace:
 [1] top-level scope
   @ /tmp/ISOKANN.jl/src/plots.jl:123
 [2] include(mod::Module, _path::String)
   @ Base ./Base.jl:495
 [3] include(x::String)
   @ ISOKANN /tmp/ISOKANN.jl/src/ISOKANN.jl:3
 [4] top-level scope
   @ /tmp/ISOKANN.jl/src/ISOKANN.jl:71
 [5] include
   @ ./Base.jl:495 [inlined]
 [6] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
   @ Base ./loading.jl:2222
 [7] top-level scope
   @ stdin:3
in expression starting at /tmp/ISOKANN.jl/src/plots.jl:123
in expression starting at /tmp/ISOKANN.jl/src/ISOKANN.jl:3
in expression starting at stdin:3
ERROR: Failed to precompile ISOKANN [512544ac-929d-4ba3-aa9d-55e9313a4b47] to "/home/mason/.julia/compiled/v1.10/ISOKANN/jl_JZRjCz".
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:2468
  [3] compilecache
    @ ./loading.jl:2340 [inlined]
  [4] (::Base.var"#968#969"{Base.PkgId})()
    @ Base ./loading.jl:1974
  [5] mkpidlock(f::Base.var"#968#969"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
    @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:93
  [6] #mkpidlock#6
    @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:88 [inlined]
  [7] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
    @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:111
  [8] #invokelatest#2
    @ ./essentials.jl:894 [inlined]
  [9] invokelatest
    @ ./essentials.jl:889 [inlined]
 [10] maybe_cachefile_lock(f::Base.var"#968#969"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
    @ Base ./loading.jl:2983
 [11] maybe_cachefile_lock
    @ ./loading.jl:2980 [inlined]
 [12] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1970
 [13] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1812
 [14] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [15] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [16] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1803
 [17] macro expansion
    @ ./loading.jl:1790 [inlined]
 [18] macro expansion
    @ ./lock.jl:267 [inlined]
 [19] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1753
 [20] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [21] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [22] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1746

view this post on Zulip Alexander Sikorski (Mar 23 2024 at 03:59):

@Mason Protter Thank you for trying. Seems I have had forgotten to actually commit the fix to the missing Iso2 (wrong include order). Anyways, did you get to that error fast? Just to get that error message took me ages..

view this post on Zulip Alexander Sikorski (Mar 23 2024 at 04:00):

I am now trying without the startup files. Compiling 200 packages anew

view this post on Zulip Alexander Sikorski (Mar 23 2024 at 04:22):

Now it takes 20 seconds after starting Julia.
Even if starting it with the startup file :O

view this post on Zulip Alexander Sikorski (Mar 23 2024 at 04:25):

inverse pauli effect.
thank you for looking into it @Mason Protter @Fredrik Ekre


Last updated: Nov 22 2024 at 04:41 UTC