Stream: helpdesk (published)

Topic: ✔ BinaryBuilder not finding file on Windows


view this post on Zulip Leandro Martínez (Jul 11 2023 at 18:51):

I'm trying to build an artifact, that the compilation works on all platforms on my machine. However, in the pull request to Yggdrasil, the binaries for windows are failing, because the binary created is not found:

https://buildkite.com/julialang/yggdrasil/builds/4037#01894634-7a9c-454c-bfaa-b681110e586f/644-750

Anyone has a hint on how to fix this? I don't know how to debug that. The path seems correct and it works when I run the build locally.

view this post on Zulip Mosè Giordano (Jul 11 2023 at 20:55):

Leandro Martínez said:

The path seems correct

That's clearly not the case since install can't find the file

Leandro Martínez said:

and it works when I run the build locally.

What do you mean exactly by this?

Leandro Martínez said:

I don't know how to debug that.

https://docs.binarybuilder.org/stable/#Manually-create-or-edit-build_tarballs.jl

The --debug option will drop you into the BinaryBuilder interactive shell if an error occurs.

view this post on Zulip Mosè Giordano (Jul 11 2023 at 20:57):

My slightly educated guess is that you're missing the exe extension, but I don't think gcc 4.8 forces it, I believe that was introduced in gcc 8

view this post on Zulip Leandro Martínez (Jul 11 2023 at 22:36):

What do you mean exactly by this?

I mean that I can run the BinaryBuilder.run_wizard() locally, build the Windows binaries and I don´t get any error. The file mdlovofit is there. I tried now a different approach to managing the extension (as suggested by another user). Let us see.

The problem for debuging is that I cannot reproduce the error here in my machine.

view this post on Zulip Mosè Giordano (Jul 11 2023 at 23:29):

That's not possible

view this post on Zulip Mosè Giordano (Jul 11 2023 at 23:32):

Ah, you're expanding libgfortran version and it's failing only with libgfortran v5, which is gcc 8, so my educated guess is indeed correct

view this post on Zulip Mosè Giordano (Jul 11 2023 at 23:34):

I don't know how you were trying to reproduce it because you'd have reproduced the error by running the script with --debug

view this post on Zulip Mosè Giordano (Jul 11 2023 at 23:34):

The Makefile could make it with some improvements

view this post on Zulip Mosè Giordano (Jul 11 2023 at 23:37):

https://github.com/JuliaPackaging/Yggdrasil/pull/7033#issuecomment-1631643386

view this post on Zulip Mosè Giordano (Jul 11 2023 at 23:48):

Mosè Giordano said:

Leandro Martínez said:

I don't know how to debug that.

https://docs.binarybuilder.org/stable/#Manually-create-or-edit-build_tarballs.jl

The --debug option will drop you into the BinaryBuilder interactive shell if an error occurs.

The same section of the manual explains that you can pass as argument to the script a comma-separated list of platform triplets, so the easiest way for you to reproduce the error would have been

julia build_tarballs.jl --debug --verbose x86_64-w64-mingw32-libgfortran5

(I hope I got it all right typing on the phone). You can try it yourself without believing me on my word

view this post on Zulip Leandro Martínez (Jul 12 2023 at 01:20):

I changed now the Makefile by one generated by cmake, which seem to be more complete.

Locally, I run the command you suggested above, with the the current script and I don´t seem to get any error.

There is some red stuff being printed, but at the end, what I get seems to indicate that everything ran fine here.

The output of

julia build_tarballs.jl -debug --verbose  x86_64-w64-mingw32-libgfortran5

is, at the end, here:

[22:04:50] make[2]: Leaving directory '/workspace/srcdir/MDLovoFit'
[22:04:50] [100%] Built target mdlovofit
[22:04:50] make[1]: Leaving directory '/workspace/srcdir/MDLovoFit'
[22:04:50] /usr/bin/cmake -E cmake_progress_start /workspace/srcdir/MDLovoFit/CMakeFiles 0
[22:04:50]  ---> install -Dvm 755 "./bin/mdlovofit${exeext}" "${bindir}/mdlovofit${exeext}"
[22:04:50] './bin/mdlovofit.exe' -> '/workspace/destdir/bin/mdlovofit.exe'
[22:04:50]  ---> source "$_script"
[22:04:50]  ---> source "$_script"
[22:04:50] Child Process exited, exit code 0
[ Info: Beginning audit of /home/leandro/Downloads/lixo/build/x86_64-w64-mingw32-libgfortran5/IJF1fngi/x86_64-w64-mingw32-libgfortran5-cxx11/destdir
[ Info: /home/leandro/Downloads/lixo/build/x86_64-w64-mingw32-libgfortran5/IJF1fngi/x86_64-w64-mingw32-libgfortran5-cxx11/destdir/bin/mdlovofit.exe locks us to libgfortran v5.0.0
[ Info: Checking bin/mdlovofit.exe with RPath list String[]
[ Info: Ignored system libraries msvcrt.dll, libgfortran-5.dll, KERNEL32.dll
[ Info: Checking license file
[ Info: Found license file(s): LICENSE
[ Info: /home/leandro/Downloads/lixo/build/x86_64-w64-mingw32-libgfortran5/IJF1fngi/x86_64-w64-mingw32-libgfortran5-cxx11/destdir/bin/mdlovofit.exe matches our search criteria of ["mdlovofit"]
[ Info: Compressing files in /home/leandro/Downloads/lixo/build/x86_64-w64-mingw32-libgfortran5/IJF1fngi/x86_64-w64-mingw32-libgfortran5-cxx11/destdir/logs/MDLovoFit
[ Info: Tree hash of contents of MDLovoFit.v20.0.4.x86_64-w64-mingw32-libgfortran5.tar.gz: 332374e51ae33478c41eea9273c52dd6c208a54d
[ Info: SHA256 of MDLovoFit.v20.0.4.x86_64-w64-mingw32-libgfortran5.tar.gz: 0c3ae256c4cfaaa71aa3ee5c07cfd6bcf1c0859cd46f958ad551dcb38b02e17f
[ Info: Tree hash of contents of MDLovoFit-logs.v20.0.4.x86_64-w64-mingw32-libgfortran5.tar.gz: 23484ecad463a072bdb0275c4ca6e64bc4b0a0d9
[ Info: SHA256 of MDLovoFit-logs.v20.0.4.x86_64-w64-mingw32-libgfortran5.tar.gz: b912978b27b276b846ab223e9faec19e0932cb00940054a13f39a807a9e5cd0d
[ Info: Timings: setup: 10.55s, build: 0.71s, audit: 4.49s, packaging: 0.67s

Seems that I don't have any error?

The script, on the server, still fails for the same reasons. I guess the install line is still wrong (tried now to change it, I don't know if the source file must have ${exeext} or not, so I'm trying all combinations I can guess (and still without being able to reproduce the same error here).

I am really sorry for being so incapable here. For me this is like trying random stuff, I really don't have any clear understanding on how these things work.

view this post on Zulip Leandro Martínez (Jul 12 2023 at 01:27):

In any case the problem seems to be again in this line:

install -Dvm 755 "./bin/mdlovofit${exeext}" "${bindir}/mdlovofit${exeext}"

Now that the Makefile creates a default binary, should that be changed to something else?

The first path, that is ./bin/mdlovofit${exeext} seems to be platform dependent, even within windows builds. In the last script I removed the first ${exeext}, to read:

install -Dvm 755 "./bin/mdlovofit" "${bindir}/mdlovofit${exeext}"

And one of the builds failed (the one that passes here, apparently). Should I still think about braching that code for that specific platform? Or the information of the Makefile now provides a more robust approach for that?

view this post on Zulip Leandro Martínez (Jul 12 2023 at 02:10):

After trying all possible combinations of install with and without ${exeext} on source and target files, I was only able to make all tests pass by moving the file "manually" to a common name, with:

if [ -e ./bin/mdlovofit.exe ]; then
    mv ./bin/mdlovofit.exe ./bin/mdlovofit
fi
install -Dvm 755 "./bin/mdlovofit" "${bindir}/mdlovofit${exeext}"

I don´t know if this is a resonable solution. But it is the only thing I found made it work for all platforms.

view this post on Zulip Leandro Martínez (Jul 12 2023 at 09:33):

So, finally, with all the help from Mosè, this is what made all work, correctly:

  1. Replace the Makefile by a CMakeLists.txt and use cmake to build the software.
  2. Use the flags for cmake as indicated in the documentation
  3. Use make install instead of install -Dvm ... .

The final file resulted to be:

# Note that this script can accept some limited command-line arguments, run
# `julia build_tarballs.jl --help` to see a usage message.
using BinaryBuilder, Pkg

name = "MDLovoFit"
version = v"20.0.4"

# Collection of sources required to complete build
sources = [
    GitSource("https://github.com/m3g/MDLovoFit.git", "e320c2a503c9bd88fe2dc102f2abfd0b2f5d4cce")
]

# Bash recipe for building across all platforms
script = raw"""
cd ${WORKSPACE}/srcdir/MDLovoFit
install_license LICENSE
mkdir build && cd build
cmake .. \
    -DCMAKE_INSTALL_PREFIX=${prefix} \
    -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} \
    -DCMAKE_BUILD_TYPE=Release
make -j${nproc}
make install
"""

# These are the platforms we will build for by default, unless further
# platforms are passed in on the command line
platforms = supported_platforms()
platforms = expand_gfortran_versions(platforms)

# The products that we will ensure are always built
products = [
    ExecutableProduct("mdlovofit", :mdlovofit)
]

# Dependencies that must be installed before this package can be built
dependencies = Dependency[
    Dependency(PackageSpec(name="CompilerSupportLibraries_jll", uuid="e66e0078-7015-5450-92f7-15fbd957f2ae")),
]

# Build the tarballs, and possibly a `build.jl` as well.
build_tarballs(ARGS, name, version, sources, script, platforms, products, dependencies; julia_compat="1.6")

view this post on Zulip Notification Bot (Jul 12 2023 at 09:33):

Leandro Martínez has marked this topic as resolved.

view this post on Zulip Notification Bot (Jul 13 2023 at 10:38):

Francois Van Zyl has marked this topic as unresolved.

view this post on Zulip Notification Bot (Jul 13 2023 at 10:39):

Francois Van Zyl has marked this topic as resolved.


Last updated: Nov 22 2024 at 04:41 UTC