Stream: helpdesk (published)

Topic: Building Julia on macOS 12.6.3 fails


view this post on Zulip mnemnion (Feb 13 2024 at 17:25):

Starting a new thread because this isn't about USE_BINARYBUILDER, just the standard method isn't working.

Workflow: clone https://github.com/JuliaLang/julia, cd to the directory, type make. Error:

curl: (22) The requested URL returned error: 502
curl: Failed to extract a sensible file name from the URL to use for storage
curl: (3) URL using bad/illegal format or missing URL
make[1]: *** [/path/to/julia/stdlib/srccache/Pkg-.tar.gz] Error 3
make: *** [julia-stdlib] Error 2

echo $LD_LIBRARY_PATH and echo $DYLD_LIBRARY_PATH return nothing, that's the only OS-specific tip I've been able to glean. That seems like an unlikely place for a curl error to arise anyway.

view this post on Zulip mnemnion (Feb 13 2024 at 17:27):

Is this something I should consider filing an issue over? Or is there some known problem I'm inadvertently triggering? I didn't see any obvious related issues in the tracker, and it seems to me that if building Julia was just entirely broken, or entirely broken on macOS, other people would notice. I'm stumped on this one.

view this post on Zulip Mosè Giordano (Feb 13 2024 at 19:32):

Do you have xcode installed?

view this post on Zulip Mosè Giordano (Feb 13 2024 at 19:33):

Although that's failing on retrieving Pkg stdlib, not downloading libraries

view this post on Zulip Mosè Giordano (Feb 13 2024 at 19:33):

The make stack trace looks incomplete?

view this post on Zulip mnemnion (Feb 13 2024 at 19:33):

I do have xcode installed and up to date, yes.

view this post on Zulip mnemnion (Feb 13 2024 at 19:35):

here's everything I get:

  make                                                                                                                                      (base)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   285  100   285    0     0    197      0  0:00:01  0:00:01 --:--:--   197
100   361  100   361    0     0    219      0  0:00:01  0:00:01 --:--:--   219
100   513  100   513    0     0    208      0  0:00:02  0:00:02 --:--:--     0
100   817  100   817    0     0    305      0  0:00:02  0:00:02 --:--:--   305
100  1425  100  1425    0     0    494      0  0:00:02  0:00:02 --:--:--   494
100  2641  100  2641    0     0    848      0  0:00:03  0:00:03 --:--:--   848
100  5073  100  5073    0     0   1522      0  0:00:03  0:00:03 --:--:-- 4954k
  0   157    0     0    0     0      0      0 --:--:--  0:00:04 --:--:--     0
curl: (22) The requested URL returned error: 502
curl: Failed to extract a sensible file name from the URL to use for storage
curl: (3) URL using bad/illegal format or missing URL
make[1]: *** [/path/to/julia/stdlib/srccache/Pkg-.tar.gz] Error 3
make: *** [julia-stdlib] Error 2

With light editing of the directory.

view this post on Zulip Mosè Giordano (Feb 13 2024 at 20:37):

make VERBOSE=1 should be more verbose

view this post on Zulip mnemnion (Feb 13 2024 at 20:45):

The output with VERBOSE=1 is actually identical, sorry, I should have said that earlier

view this post on Zulip mnemnion (Feb 13 2024 at 20:46):

oh wait, I take that back, there's one extra line.

  make VERBOSE=1                                                                                                                            (base)
/path/to/julia/deps/tools/jldownload /path/to/julia/stdlib/srccache/Pkg-.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   285  100   285    0     0    283      0  0:00:01  0:00:01 --:--:--   283
100   361  100   361    0     0    299      0  0:00:01  0:00:01 --:--:--   299
100   513  100   513    0     0    364      0  0:00:01  0:00:01 --:--:--   364
100   817  100   817    0     0    506      0  0:00:01  0:00:01 --:--:--   506
100  1425  100  1425    0     0    784      0  0:00:01  0:00:01 --:--:--   784
100  2641  100  2641    0     0   1307      0  0:00:02  0:00:02 --:--:--     0
100  5073  100  5073    0     0   2273      0  0:00:02  0:00:02 --:--:--  2273
  0   157    0     0    0     0      0      0 --:--:--  0:00:03 --:--:--     0
curl: (22) The requested URL returned error: 502
curl: Failed to extract a sensible file name from the URL to use for storage
curl: (3) URL using bad/illegal format or missing URL
make[1]: *** [/path/to/julia/stdlib/srccache/Pkg-.tar.gz] Error 3
make: *** [julia-stdlib] Error 2

view this post on Zulip Mosè Giordano (Feb 13 2024 at 22:48):

make -C stdlib VERBOSE=1

view this post on Zulip mnemnion (Feb 13 2024 at 22:59):

Identical output

view this post on Zulip Fredrik Ekre (Feb 13 2024 at 23:01):

Pkg-.tar.gz looks suspicious. There should be a hash after the -. The URL for this would be something with that hash trailing I think, I guess if that is missing curl gives that error

view this post on Zulip mnemnion (Feb 13 2024 at 23:02):

Yes, I wasn't sure what was supposed to be after Pkg- but that looked obviously wrong.

view this post on Zulip Fredrik Ekre (Feb 13 2024 at 23:05):

It should be read from https://github.com/JuliaLang/julia/blob/ba34d899edcbed48bddfdcf59c09057f3c97b663/stdlib/Pkg.version#L2

view this post on Zulip mnemnion (Feb 13 2024 at 23:14):

my version of that file is identical to the link. I'd be rather surprised if that wasn't true, but wanted to confirm it here

view this post on Zulip mnemnion (Feb 13 2024 at 23:15):

the srccache directory is completely empty, which is expected since filling it is the operation that failed (?)

view this post on Zulip mnemnion (Feb 14 2024 at 22:37):

As of the latest commit to master (3e6ff3d37161932f52) the build fails in the same way.

view this post on Zulip mnemnion (Feb 15 2024 at 19:45):

So I've been doing the following daily: trash the julia repo, clone it, and run make VERBOSE=1. It fails the same way each time, I think this is going on a week

view this post on Zulip mnemnion (Feb 15 2024 at 19:46):

Is this the point where I should file an issue? The problem must be idiosyncratic in some sense, there's no way that builds have simply been broken for everyone on macOS for this long. But I ran out of ideas about what could be wrong a long time ago.

view this post on Zulip Mosè Giordano (Feb 15 2024 at 22:44):

I have macOS 12.6 as well and I don't have any issues

view this post on Zulip Mosè Giordano (Feb 15 2024 at 22:45):

Find out where that URL is coming from? Debugging makefiles isn't terrible, you can put all the echo commands you want to trace things

view this post on Zulip mnemnion (Feb 15 2024 at 22:46):

Yeah it beggars belief that this is a universal problem. Which is why I've been sticking to helpdesk, the issue tracker isn't for idiosyncratic stuff

view this post on Zulip mnemnion (Feb 15 2024 at 22:53):

I've both written and debugged makefiles in the past, and would, in fact, describe debugging them as "terrible". but yeah at this point I don't have much choice. VERBOSE=0 is still pretty terse

view this post on Zulip Fredrik Ekre (Feb 15 2024 at 23:09):

$ make -C stdlib print-PKG_SHA1 prints what?

view this post on Zulip mnemnion (Feb 15 2024 at 23:09):

PKG_SHA1=6dd0e7c9e99d578aa5477e2c78c91a161ce4c357

view this post on Zulip Fredrik Ekre (Feb 15 2024 at 23:10):

Strange, it finds that, but omits in from the URL

view this post on Zulip mnemnion (Feb 15 2024 at 23:12):

sourced from Pkg.version

PKG_BRANCH = master
PKG_SHA1 = 6dd0e7c9e99d578aa5477e2c78c91a161ce4c357
PKG_GIT_URL := https://github.com/JuliaLang/Pkg.jl.git
PKG_TAR_URL = https://api.github.com/repos/JuliaLang/Pkg.jl/tarball/$1

I've haven't managed to suss out where jldownload is trying to read this stuff from

view this post on Zulip Fredrik Ekre (Feb 15 2024 at 23:12):

This is in deps/tools/git-external.mk.

view this post on Zulip mnemnion (Feb 15 2024 at 23:15):

Screen-Shot-2024-02-15-at-6.14.59-PM.png
Imma keep it real w. y'all, this is line noise to me

view this post on Zulip Fredrik Ekre (Feb 15 2024 at 23:15):

make -C stdlib print-PKG_SRC_FILE?

view this post on Zulip mnemnion (Feb 15 2024 at 23:16):

oh ho: PKG_SRC_FILE=

view this post on Zulip mnemnion (Feb 15 2024 at 23:18):

no results found in VSCode search for PKG_SRC_FILE either, and it found PKG_SHA1

view this post on Zulip Fredrik Ekre (Feb 15 2024 at 23:18):

It is defined here: https://github.com/JuliaLang/julia/blob/e460d357d48d121f55e8007d35f6848e9983cfac/deps/tools/git-external.mk#L61

view this post on Zulip Fredrik Ekre (Feb 15 2024 at 23:19):

Is PKG_SRC_DIR also empty then?

view this post on Zulip mnemnion (Feb 15 2024 at 23:20):

PKG_SRC_DIR is also empty, yep

view this post on Zulip Fredrik Ekre (Feb 15 2024 at 23:22):

Strange, that should just interpolate PKG_SHA1 which you said was defined: https://github.com/JuliaLang/julia/blob/e460d357d48d121f55e8007d35f6848e9983cfac/deps/tools/git-external.mk#L60

view this post on Zulip Fredrik Ekre (Feb 15 2024 at 23:23):

Whats your make --version?

view this post on Zulip mnemnion (Feb 15 2024 at 23:23):

3.81

view this post on Zulip mnemnion (Feb 15 2024 at 23:24):

I'll try it with gmake and see if that gets things going

view this post on Zulip mnemnion (Feb 15 2024 at 23:25):

I had assumed that would be mentioned in the macos-specific build instructions, it didn't occur to me to try it

view this post on Zulip mnemnion (Feb 15 2024 at 23:26):

weirdly that doesn't even start:

  gmake VERBOSE=0                                                                                                                           (base)
Makefile:53: *** empty variable name.  Stop.
gmake: *** [Makefile:79: julia-stdlib] Error 2

view this post on Zulip mnemnion (Feb 15 2024 at 23:36):

here's where gmake quits from a fresh clone:

echo 'libwhich-81e9723c0273d78493dc8c8ed570f68d9ce7e89e staged-uninstaller' > /Users/atman/orb/julia/usr/manifest/libwhich
gmake[1]: Leaving directory '/Users/atman/orb/julia/deps'
gmake[1]: Entering directory '/Users/atman/orb/julia/stdlib'
Makefile:53: *** empty variable name.  Stop.
gmake[1]: Leaving directory '/Users/atman/orb/julia/stdlib'
gmake: *** [Makefile:79: julia-stdlib] Error 2

view this post on Zulip Fredrik Ekre (Feb 15 2024 at 23:56):

Whats the version of gmake then? I am using

$ make --version
GNU Make 4.3

view this post on Zulip mnemnion (Feb 15 2024 at 23:56):

4.4.1

view this post on Zulip Fredrik Ekre (Feb 15 2024 at 23:56):

:shrugging:

view this post on Zulip Mark Kittisopikul (Feb 28 2024 at 19:52):

Commenting so that I can find the conversation later.

view this post on Zulip Mark Kittisopikul (Feb 28 2024 at 19:56):

Have you tried make distcleanall ?

view this post on Zulip Mark Kittisopikul (Feb 29 2024 at 00:33):

I updated the Github issue with a few other things to try:
https://github.com/JuliaLang/julia/issues/53444#issuecomment-1969363676

view this post on Zulip mnemnion (Mar 05 2024 at 02:30):

Just gave make distcleanall a shot, same error. which at this point doesn't surprise me, something isn't populating PKG_SRC_DIR and PKG_SRC_FILE and on any given day that I try a build, it's from a fresh clone of the repo. Thanks for helping me troubleshoot this


Last updated: Nov 06 2024 at 04:40 UTC