mnemnion (Feb 13 2024 at 17:25): mnemnion (Feb 13 2024 at 17:25):Starting a new thread because this isn't about USE_BINARYBUILDER, just the standard method isn't working.
Workflow: clone https://github.com/JuliaLang/julia, cd to the directory, type make. Error:
curl: (22) The requested URL returned error: 502
curl: Failed to extract a sensible file name from the URL to use for storage
curl: (3) URL using bad/illegal format or missing URL
make[1]: *** [/path/to/julia/stdlib/srccache/Pkg-.tar.gz] Error 3
make: *** [julia-stdlib] Error 2
echo $LD_LIBRARY_PATH and echo $DYLD_LIBRARY_PATH return nothing, that's the only OS-specific tip I've been able to glean. That seems like an unlikely place for a curl error to arise anyway.
mnemnion (Feb 13 2024 at 17:27):Is this something I should consider filing an issue over? Or is there some known problem I'm inadvertently triggering? I didn't see any obvious related issues in the tracker, and it seems to me that if building Julia was just entirely broken, or entirely broken on macOS, other people would notice. I'm stumped on this one.
Mosè Giordano (Feb 13 2024 at 19:32):Do you have xcode installed?
Mosè Giordano (Feb 13 2024 at 19:33):Although that's failing on retrieving Pkg stdlib, not downloading libraries
Mosè Giordano (Feb 13 2024 at 19:33):The make stack trace looks incomplete?
mnemnion (Feb 13 2024 at 19:33):I do have xcode installed and up to date, yes.
mnemnion (Feb 13 2024 at 19:35):here's everything I get:
➜ make (base)
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 285 100 285 0 0 197 0 0:00:01 0:00:01 --:--:-- 197
100 361 100 361 0 0 219 0 0:00:01 0:00:01 --:--:-- 219
100 513 100 513 0 0 208 0 0:00:02 0:00:02 --:--:-- 0
100 817 100 817 0 0 305 0 0:00:02 0:00:02 --:--:-- 305
100 1425 100 1425 0 0 494 0 0:00:02 0:00:02 --:--:-- 494
100 2641 100 2641 0 0 848 0 0:00:03 0:00:03 --:--:-- 848
100 5073 100 5073 0 0 1522 0 0:00:03 0:00:03 --:--:-- 4954k
0 157 0 0 0 0 0 0 --:--:-- 0:00:04 --:--:-- 0
curl: (22) The requested URL returned error: 502
curl: Failed to extract a sensible file name from the URL to use for storage
curl: (3) URL using bad/illegal format or missing URL
make[1]: *** [/path/to/julia/stdlib/srccache/Pkg-.tar.gz] Error 3
make: *** [julia-stdlib] Error 2
With light editing of the directory.
Mosè Giordano (Feb 13 2024 at 20:37):make VERBOSE=1 should be more verbose
mnemnion (Feb 13 2024 at 20:45):The output with VERBOSE=1 is actually identical, sorry, I should have said that earlier
mnemnion (Feb 13 2024 at 20:46):oh wait, I take that back, there's one extra line.
➜ make VERBOSE=1 (base)
/path/to/julia/deps/tools/jldownload /path/to/julia/stdlib/srccache/Pkg-.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 285 100 285 0 0 283 0 0:00:01 0:00:01 --:--:-- 283
100 361 100 361 0 0 299 0 0:00:01 0:00:01 --:--:-- 299
100 513 100 513 0 0 364 0 0:00:01 0:00:01 --:--:-- 364
100 817 100 817 0 0 506 0 0:00:01 0:00:01 --:--:-- 506
100 1425 100 1425 0 0 784 0 0:00:01 0:00:01 --:--:-- 784
100 2641 100 2641 0 0 1307 0 0:00:02 0:00:02 --:--:-- 0
100 5073 100 5073 0 0 2273 0 0:00:02 0:00:02 --:--:-- 2273
0 157 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0
curl: (22) The requested URL returned error: 502
curl: Failed to extract a sensible file name from the URL to use for storage
curl: (3) URL using bad/illegal format or missing URL
make[1]: *** [/path/to/julia/stdlib/srccache/Pkg-.tar.gz] Error 3
make: *** [julia-stdlib] Error 2
make -C stdlib VERBOSE=1
Identical output
Fredrik Ekre (Feb 13 2024 at 23:01):Pkg-.tar.gz looks suspicious. There should be a hash after the -. The URL for this would be something with that hash trailing I think, I guess if that is missing curl gives that error
mnemnion (Feb 13 2024 at 23:02):Yes, I wasn't sure what was supposed to be after Pkg- but that looked obviously wrong.
Fredrik Ekre (Feb 13 2024 at 23:05):It should be read from https://github.com/JuliaLang/julia/blob/ba34d899edcbed48bddfdcf59c09057f3c97b663/stdlib/Pkg.version#L2
mnemnion (Feb 13 2024 at 23:14):my version of that file is identical to the link. I'd be rather surprised if that wasn't true, but wanted to confirm it here
mnemnion (Feb 13 2024 at 23:15):the srccache directory is completely empty, which is expected since filling it is the operation that failed (?)
mnemnion (Feb 14 2024 at 22:37):As of the latest commit to master (3e6ff3d37161932f52) the build fails in the same way.
mnemnion (Feb 15 2024 at 19:45):So I've been doing the following daily: trash the julia repo, clone it, and run make VERBOSE=1. It fails the same way each time, I think this is going on a week
mnemnion (Feb 15 2024 at 19:46):Is this the point where I should file an issue? The problem must be idiosyncratic in some sense, there's no way that builds have simply been broken for everyone on macOS for this long. But I ran out of ideas about what could be wrong a long time ago.
Mosè Giordano (Feb 15 2024 at 22:44):I have macOS 12.6 as well and I don't have any issues
Mosè Giordano (Feb 15 2024 at 22:45):Find out where that URL is coming from? Debugging makefiles isn't terrible, you can put all the echo commands you want to trace things
mnemnion (Feb 15 2024 at 22:46):Yeah it beggars belief that this is a universal problem. Which is why I've been sticking to helpdesk, the issue tracker isn't for idiosyncratic stuff
mnemnion (Feb 15 2024 at 22:53):I've both written and debugged makefiles in the past, and would, in fact, describe debugging them as "terrible". but yeah at this point I don't have much choice. VERBOSE=0 is still pretty terse
Fredrik Ekre (Feb 15 2024 at 23:09):$ make -C stdlib print-PKG_SHA1 prints what?
mnemnion (Feb 15 2024 at 23:09):PKG_SHA1=6dd0e7c9e99d578aa5477e2c78c91a161ce4c357
Fredrik Ekre (Feb 15 2024 at 23:10):Strange, it finds that, but omits in from the URL
mnemnion (Feb 15 2024 at 23:12):sourced from Pkg.version
PKG_BRANCH = master
PKG_SHA1 = 6dd0e7c9e99d578aa5477e2c78c91a161ce4c357
PKG_GIT_URL := https://github.com/JuliaLang/Pkg.jl.git
PKG_TAR_URL = https://api.github.com/repos/JuliaLang/Pkg.jl/tarball/$1
I've haven't managed to suss out where jldownload is trying to read this stuff from
Fredrik Ekre (Feb 15 2024 at 23:12):This is in deps/tools/git-external.mk.
mnemnion (Feb 15 2024 at 23:15):Screen-Shot-2024-02-15-at-6.14.59-PM.png
Imma keep it real w. y'all, this is line noise to me
Fredrik Ekre (Feb 15 2024 at 23:15):make -C stdlib print-PKG_SRC_FILE?
mnemnion (Feb 15 2024 at 23:16):oh ho: PKG_SRC_FILE=
mnemnion (Feb 15 2024 at 23:18):no results found in VSCode search for PKG_SRC_FILE either, and it found PKG_SHA1
Fredrik Ekre (Feb 15 2024 at 23:18):It is defined here: https://github.com/JuliaLang/julia/blob/e460d357d48d121f55e8007d35f6848e9983cfac/deps/tools/git-external.mk#L61
Fredrik Ekre (Feb 15 2024 at 23:19):Is PKG_SRC_DIR also empty then?
mnemnion (Feb 15 2024 at 23:20):PKG_SRC_DIR is also empty, yep
Fredrik Ekre (Feb 15 2024 at 23:22):Strange, that should just interpolate PKG_SHA1 which you said was defined: https://github.com/JuliaLang/julia/blob/e460d357d48d121f55e8007d35f6848e9983cfac/deps/tools/git-external.mk#L60
Fredrik Ekre (Feb 15 2024 at 23:23):Whats your make --version?
mnemnion (Feb 15 2024 at 23:23):3.81
mnemnion (Feb 15 2024 at 23:24):I'll try it with gmake and see if that gets things going
mnemnion (Feb 15 2024 at 23:25):I had assumed that would be mentioned in the macos-specific build instructions, it didn't occur to me to try it
mnemnion (Feb 15 2024 at 23:26):weirdly that doesn't even start:
➜ gmake VERBOSE=0 (base)
Makefile:53: *** empty variable name. Stop.
gmake: *** [Makefile:79: julia-stdlib] Error 2
here's where gmake quits from a fresh clone:
echo 'libwhich-81e9723c0273d78493dc8c8ed570f68d9ce7e89e staged-uninstaller' > /Users/atman/orb/julia/usr/manifest/libwhich
gmake[1]: Leaving directory '/Users/atman/orb/julia/deps'
gmake[1]: Entering directory '/Users/atman/orb/julia/stdlib'
Makefile:53: *** empty variable name. Stop.
gmake[1]: Leaving directory '/Users/atman/orb/julia/stdlib'
gmake: *** [Makefile:79: julia-stdlib] Error 2
Whats the version of gmake then? I am using
$ make --version
GNU Make 4.3
4.4.1
Fredrik Ekre (Feb 15 2024 at 23:56):
Mark Kittisopikul (Feb 28 2024 at 19:52):Commenting so that I can find the conversation later.
Mark Kittisopikul (Feb 28 2024 at 19:56):Have you tried make distcleanall ?
Mark Kittisopikul (Feb 29 2024 at 00:33):I updated the Github issue with a few other things to try:
https://github.com/JuliaLang/julia/issues/53444#issuecomment-1969363676
mnemnion (Mar 05 2024 at 02:30):Just gave make distcleanall a shot, same error. which at this point doesn't surprise me, something isn't populating PKG_SRC_DIR and PKG_SRC_FILE and on any given day that I try a build, it's from a fresh clone of the repo. Thanks for helping me troubleshoot this
Last updated: Nov 22 2024 at 04:41 UTC