Alexander Sikorski (Mar 22 2024 at 22:39): Alexander Sikorski (Mar 22 2024 at 22:39):I just went through 15 minutes of precompiling around 400 Packages. (]precompile after cleaning .julia/compiled)
Now when using myPackage (from its activated environment) it just goes through precompilation of 35 packages again.
Why is this happening?
Fredrik Ekre (Mar 22 2024 at 22:40):Where/what did you precompile the first time?
Alexander Sikorski (Mar 22 2024 at 22:42):Fredrik Ekre said:
Where/what did you precompile the first time?
From the packages environment. (I am sane enough not to have 400 packages in my Base environment :D)
Fredrik Ekre (Mar 22 2024 at 22:43):Not sure, would need a reproducer.
Alexander Sikorski (Mar 22 2024 at 22:46):https://github.com/axsk/ISOKANN.jl
Alexander Sikorski (Mar 22 2024 at 22:48):I am trying to get it usinged for far more then an hour by now, just looking at precompilation all the time :(
Alexander Sikorski (Mar 22 2024 at 22:49):Currently stuck at
julia> @time @time_imports using ISOKANN
Precompiling ISOKANN
32 dependencies successfully precompiled in 781 seconds. 395 already precompiled.
[ Info: Precompiling ISOKANN [512544ac-929d-4ba3-aa9d-55e9313a4b47]
I don't even know how much time I spent on the last trial only to get an error because I had two includes in the wrong order :/
Alexander Sikorski (Mar 22 2024 at 22:50):Would really appreciate if someone could have a look at this and help me
Alexander Sikorski (Mar 22 2024 at 22:52):julia> versioninfo()
Julia Version 1.10.2
Commit bd47eca2c8a (2024-03-01 10:14 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: macOS (x86_64-apple-darwin22.4.0)
CPU: 4 × Intel(R) Core(TM) i5-7267U CPU @ 3.10GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-15.0.7 (ORCJIT, skylake)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Environment:
JULIA_REVISE_INCLUDE = 1
That env doesn't even instantiate for me (segfault in CUDA instantiation). Try stripping away deps until you just have enough to repro perhaps.
Alexander Sikorski (Mar 22 2024 at 22:58):Which takes me around 30 mins for every trial.. (still stuck with precompiling...)
Alexander Sikorski (Mar 22 2024 at 23:12):The result of @time using ISOKANN
1823.583727 seconds (22.52 M allocations: 1.451 GiB, 0.08% gc time, 0.20% compilation time: 78% of which was recompilation)
Alexander Sikorski (Mar 22 2024 at 23:14):Calling it a day :(
Mason Protter (Mar 22 2024 at 23:44):what happens if you do julia --startup=no and then using the package?
Mason Protter (Mar 22 2024 at 23:50):For what it's worth, I get an error during precompilation:
julia> using ISOKANN
Precompiling ISOKANN
[ Info: Precompiling ISOKANN [512544ac-929d-4ba3-aa9d-55e9313a4b47]
ERROR: LoadError: UndefVarError: `Iso2` not defined
Stacktrace:
[1] top-level scope
@ /tmp/ISOKANN.jl/src/plots.jl:123
[2] include(mod::Module, _path::String)
@ Base ./Base.jl:495
[3] include(x::String)
@ ISOKANN /tmp/ISOKANN.jl/src/ISOKANN.jl:3
[4] top-level scope
@ /tmp/ISOKANN.jl/src/ISOKANN.jl:71
[5] include
@ ./Base.jl:495 [inlined]
[6] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
@ Base ./loading.jl:2222
[7] top-level scope
@ stdin:3
in expression starting at /tmp/ISOKANN.jl/src/plots.jl:123
in expression starting at /tmp/ISOKANN.jl/src/ISOKANN.jl:3
in expression starting at stdin:3
ERROR: Failed to precompile ISOKANN [512544ac-929d-4ba3-aa9d-55e9313a4b47] to "/home/mason/.julia/compiled/v1.10/ISOKANN/jl_JZRjCz".
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:35
[2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
@ Base ./loading.jl:2468
[3] compilecache
@ ./loading.jl:2340 [inlined]
[4] (::Base.var"#968#969"{Base.PkgId})()
@ Base ./loading.jl:1974
[5] mkpidlock(f::Base.var"#968#969"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
@ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:93
[6] #mkpidlock#6
@ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:88 [inlined]
[7] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
@ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:111
[8] #invokelatest#2
@ ./essentials.jl:894 [inlined]
[9] invokelatest
@ ./essentials.jl:889 [inlined]
[10] maybe_cachefile_lock(f::Base.var"#968#969"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
@ Base ./loading.jl:2983
[11] maybe_cachefile_lock
@ ./loading.jl:2980 [inlined]
[12] _require(pkg::Base.PkgId, env::String)
@ Base ./loading.jl:1970
[13] __require_prelocked(uuidkey::Base.PkgId, env::String)
@ Base ./loading.jl:1812
[14] #invoke_in_world#3
@ ./essentials.jl:926 [inlined]
[15] invoke_in_world
@ ./essentials.jl:923 [inlined]
[16] _require_prelocked(uuidkey::Base.PkgId, env::String)
@ Base ./loading.jl:1803
[17] macro expansion
@ ./loading.jl:1790 [inlined]
[18] macro expansion
@ ./lock.jl:267 [inlined]
[19] __require(into::Module, mod::Symbol)
@ Base ./loading.jl:1753
[20] #invoke_in_world#3
@ ./essentials.jl:926 [inlined]
[21] invoke_in_world
@ ./essentials.jl:923 [inlined]
[22] require(into::Module, mod::Symbol)
@ Base ./loading.jl:1746
@Mason Protter Thank you for trying. Seems I have had forgotten to actually commit the fix to the missing Iso2 (wrong include order). Anyways, did you get to that error fast? Just to get that error message took me ages..
Alexander Sikorski (Mar 23 2024 at 04:00):I am now trying without the startup files. Compiling 200 packages anew
Alexander Sikorski (Mar 23 2024 at 04:22):Now it takes 20 seconds after starting Julia.
Even if starting it with the startup file :O
Alexander Sikorski (Mar 23 2024 at 04:25):inverse pauli effect.
thank you for looking into it @Mason Protter @Fredrik Ekre
Last updated: Nov 06 2024 at 04:40 UTC