Starting a new thread because this isn't about USE_BINARYBUILDER
, just the standard method isn't working.
Workflow: clone https://github.com/JuliaLang/julia
, cd to the directory, type make
. Error:
curl: (22) The requested URL returned error: 502
curl: Failed to extract a sensible file name from the URL to use for storage
curl: (3) URL using bad/illegal format or missing URL
make[1]: *** [/path/to/julia/stdlib/srccache/Pkg-.tar.gz] Error 3
make: *** [julia-stdlib] Error 2
echo $LD_LIBRARY_PATH
and echo $DYLD_LIBRARY_PATH
return nothing, that's the only OS-specific tip I've been able to glean. That seems like an unlikely place for a curl error to arise anyway.
Is this something I should consider filing an issue over? Or is there some known problem I'm inadvertently triggering? I didn't see any obvious related issues in the tracker, and it seems to me that if building Julia was just entirely broken, or entirely broken on macOS, other people would notice. I'm stumped on this one.
Do you have xcode installed?
Although that's failing on retrieving Pkg stdlib, not downloading libraries
The make stack trace looks incomplete?
I do have xcode installed and up to date, yes.
here's everything I get:
➜ make (base)
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 285 100 285 0 0 197 0 0:00:01 0:00:01 --:--:-- 197
100 361 100 361 0 0 219 0 0:00:01 0:00:01 --:--:-- 219
100 513 100 513 0 0 208 0 0:00:02 0:00:02 --:--:-- 0
100 817 100 817 0 0 305 0 0:00:02 0:00:02 --:--:-- 305
100 1425 100 1425 0 0 494 0 0:00:02 0:00:02 --:--:-- 494
100 2641 100 2641 0 0 848 0 0:00:03 0:00:03 --:--:-- 848
100 5073 100 5073 0 0 1522 0 0:00:03 0:00:03 --:--:-- 4954k
0 157 0 0 0 0 0 0 --:--:-- 0:00:04 --:--:-- 0
curl: (22) The requested URL returned error: 502
curl: Failed to extract a sensible file name from the URL to use for storage
curl: (3) URL using bad/illegal format or missing URL
make[1]: *** [/path/to/julia/stdlib/srccache/Pkg-.tar.gz] Error 3
make: *** [julia-stdlib] Error 2
With light editing of the directory.
make VERBOSE=1 should be more verbose
The output with VERBOSE=1
is actually identical, sorry, I should have said that earlier
oh wait, I take that back, there's one extra line.
➜ make VERBOSE=1 (base)
/path/to/julia/deps/tools/jldownload /path/to/julia/stdlib/srccache/Pkg-.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 285 100 285 0 0 283 0 0:00:01 0:00:01 --:--:-- 283
100 361 100 361 0 0 299 0 0:00:01 0:00:01 --:--:-- 299
100 513 100 513 0 0 364 0 0:00:01 0:00:01 --:--:-- 364
100 817 100 817 0 0 506 0 0:00:01 0:00:01 --:--:-- 506
100 1425 100 1425 0 0 784 0 0:00:01 0:00:01 --:--:-- 784
100 2641 100 2641 0 0 1307 0 0:00:02 0:00:02 --:--:-- 0
100 5073 100 5073 0 0 2273 0 0:00:02 0:00:02 --:--:-- 2273
0 157 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0
curl: (22) The requested URL returned error: 502
curl: Failed to extract a sensible file name from the URL to use for storage
curl: (3) URL using bad/illegal format or missing URL
make[1]: *** [/path/to/julia/stdlib/srccache/Pkg-.tar.gz] Error 3
make: *** [julia-stdlib] Error 2
make -C stdlib VERBOSE=1
Identical output
Pkg-.tar.gz
looks suspicious. There should be a hash after the -
. The URL for this would be something with that hash trailing I think, I guess if that is missing curl gives that error
Yes, I wasn't sure what was supposed to be after Pkg-
but that looked obviously wrong.
It should be read from https://github.com/JuliaLang/julia/blob/ba34d899edcbed48bddfdcf59c09057f3c97b663/stdlib/Pkg.version#L2
my version of that file is identical to the link. I'd be rather surprised if that wasn't true, but wanted to confirm it here
the srccache directory is completely empty, which is expected since filling it is the operation that failed (?)
As of the latest commit to master (3e6ff3d37161932f52) the build fails in the same way.
So I've been doing the following daily: trash the julia repo, clone it, and run make VERBOSE=1
. It fails the same way each time, I think this is going on a week
Is this the point where I should file an issue? The problem must be idiosyncratic in some sense, there's no way that builds have simply been broken for everyone on macOS for this long. But I ran out of ideas about what could be wrong a long time ago.
I have macOS 12.6 as well and I don't have any issues
Find out where that URL is coming from? Debugging makefiles isn't terrible, you can put all the echo commands you want to trace things
Yeah it beggars belief that this is a universal problem. Which is why I've been sticking to helpdesk, the issue tracker isn't for idiosyncratic stuff
I've both written and debugged makefiles in the past, and would, in fact, describe debugging them as "terrible". but yeah at this point I don't have much choice. VERBOSE=0
is still pretty terse
$ make -C stdlib print-PKG_SHA1
prints what?
PKG_SHA1=6dd0e7c9e99d578aa5477e2c78c91a161ce4c357
Strange, it finds that, but omits in from the URL
sourced from Pkg.version
PKG_BRANCH = master
PKG_SHA1 = 6dd0e7c9e99d578aa5477e2c78c91a161ce4c357
PKG_GIT_URL := https://github.com/JuliaLang/Pkg.jl.git
PKG_TAR_URL = https://api.github.com/repos/JuliaLang/Pkg.jl/tarball/$1
I've haven't managed to suss out where jldownload is trying to read this stuff from
This is in deps/tools/git-external.mk
.
Screen-Shot-2024-02-15-at-6.14.59-PM.png
Imma keep it real w. y'all, this is line noise to me
make -C stdlib print-PKG_SRC_FILE
?
oh ho: PKG_SRC_FILE=
no results found in VSCode search for PKG_SRC_FILE
either, and it found PKG_SHA1
It is defined here: https://github.com/JuliaLang/julia/blob/e460d357d48d121f55e8007d35f6848e9983cfac/deps/tools/git-external.mk#L61
Is PKG_SRC_DIR
also empty then?
PKG_SRC_DIR
is also empty, yep
Strange, that should just interpolate PKG_SHA1
which you said was defined: https://github.com/JuliaLang/julia/blob/e460d357d48d121f55e8007d35f6848e9983cfac/deps/tools/git-external.mk#L60
Whats your make --version
?
3.81
I'll try it with gmake and see if that gets things going
I had assumed that would be mentioned in the macos-specific build instructions, it didn't occur to me to try it
weirdly that doesn't even start:
➜ gmake VERBOSE=0 (base)
Makefile:53: *** empty variable name. Stop.
gmake: *** [Makefile:79: julia-stdlib] Error 2
here's where gmake quits from a fresh clone:
echo 'libwhich-81e9723c0273d78493dc8c8ed570f68d9ce7e89e staged-uninstaller' > /Users/atman/orb/julia/usr/manifest/libwhich
gmake[1]: Leaving directory '/Users/atman/orb/julia/deps'
gmake[1]: Entering directory '/Users/atman/orb/julia/stdlib'
Makefile:53: *** empty variable name. Stop.
gmake[1]: Leaving directory '/Users/atman/orb/julia/stdlib'
gmake: *** [Makefile:79: julia-stdlib] Error 2
Whats the version of gmake then? I am using
$ make --version
GNU Make 4.3
4.4.1
Commenting so that I can find the conversation later.
Have you tried make distcleanall
?
I updated the Github issue with a few other things to try:
https://github.com/JuliaLang/julia/issues/53444#issuecomment-1969363676
Just gave make distcleanall
a shot, same error. which at this point doesn't surprise me, something isn't populating PKG_SRC_DIR
and PKG_SRC_FILE
and on any given day that I try a build, it's from a fresh clone of the repo. Thanks for helping me troubleshoot this
Last updated: Nov 06 2024 at 04:40 UTC