Undefined references to `utf8proc_*` when building julia from source on mingw32

[ajinkya-k]

Building julia from source fails on mingw32 possibly because utf8proc library is missing. Note that this error does not occur for mingw64, but happens only on 32bit

See errors below:

    LINK src/flisp/libflisp.a
    LINK src/flisp/flisp.exe
C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/14.2.0/../../../../i686-w64-mingw32/bin/ld.exe: C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.o: in function `normalize':
C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.c:305:(.text+0x27d): undefined reference to `utf8proc_decompose_custom'
C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/14.2.0/../../../../i686-w64-mingw32/bin/ld.exe: C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.c:314:(.text+0x2cd): undefined reference to `utf8proc_decompose_custom'
C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/14.2.0/../../../../i686-w64-mingw32/bin/ld.exe: C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.c:317:(.text+0x2ed): undefined reference to `utf8proc_reencode'
C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/14.2.0/../../../../i686-w64-mingw32/bin/ld.exe: C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.c:321:(.text+0x345): undefined reference to `utf8proc_errmsg'
C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/14.2.0/../../../../i686-w64-mingw32/bin/ld.exe: C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.o: in function `jl_id_start_char':
C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.c:133:(.text+0x8ed): undefined reference to `utf8proc_category'
C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/14.2.0/../../../../i686-w64-mingw32/bin/ld.exe: C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.o: in function `never_id_char':
C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.c:180:(.text+0x9a7): undefined reference to `utf8proc_category'
C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/14.2.0/../../../../i686-w64-mingw32/bin/ld.exe: C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.o: in function `jl_id_char':
C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.c:143:(.text+0xd2e): undefined reference to `utf8proc_category'
C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/14.2.0/../../../../i686-w64-mingw32/bin/ld.exe: C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.c:143:(.text+0xf8d): undefined reference to `utf8proc_category'
C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/14.2.0/../../../../i686-w64-mingw32/bin/ld.exe: C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.o: in function `jl_id_start_char':
C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.c:133:(.text+0x1294): undefined reference to `utf8proc_category'
C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/14.2.0/../../../../i686-w64-mingw32/bin/ld.exe: C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.o:C:/Users/kajin/Documents/julia/src/flisp/julia_extensions.c:143: more undefined references to `utf8proc_category' follow
C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/14.2.0/../../../../i686-w64-mingw32/bin/ld.exe: C:/Users/kajin/Documents/julia/src/flisp/../support/libsupport.a(utf8.o): in function `u8_strwidth':
C:/Users/kajin/Documents/julia/src/support/utf8.c:262:(.text+0x8bc): undefined reference to `utf8proc_charwidth'
C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/14.2.0/../../../../i686-w64-mingw32/bin/ld.exe: C:/Users/kajin/Documents/julia/src/flisp/../support/libsupport.a(ios.o): in function `ios_getutf8':
C:/Users/kajin/Documents/julia/src/support/ios.c:1163:(.text+0x48d8): undefined reference to `utf8proc_charwidth'
C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/14.2.0/../../../../i686-w64-mingw32/bin/ld.exe: C:/Users/kajin/Documents/julia/src/support/ios.c:1177:(.text+0x4949): undefined reference to `utf8proc_charwidth'
collect2.exe: error: ld returned 1 exit status
make[2]: *** [Makefile:109: /c/Users/kajin/Documents/julia/src/flisp/flisp.exe] Error 1
make[1]: *** [Makefile:392: flisp/libflisp.a] Error 2
make: *** [Makefile:62: julia_flisp.boot.inc.phony] Error 2

can someone help with this?

[giordano]

Build with make ... VERBOSE=1 to show the compiler invocation, otherwise it's impossible to say anything.

[ajinkya-k]

this might have happened because the julia directory was not clean. I will try to build from a clean directory again.

[inkydragon]

Mingw32

$ make -C src/flisp/ VERBOSE=1
make: 进入目录“/d/jl/julia/src/flisp”
 gcc -march=pentium4 -m32 -O3 -ggdb2 -falign-functions  -momit-leaf-frame-pointer -I/d/jl/julia/src/flisp/../support -std=gnu11 -pipe  -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -Wformat -Wformat-security -fno-gnu-unique -mfpmath=sse -mincoming-stack-boundary=2  -I/d/jl/julia/usr/include -I/d/jl/julia/usr/include -I/d/jl/julia/usr/include  -DJL_LIBRARY_EXPORTS_INTERNAL -DUTF8PROC_EXPORTS -DUSE_COMPUTED_GOTO -Wall -Wno-strict-aliasing -fvisibility=hidden -Wpointer-arith -Wundef -Wold-style-definition -Wstrict-prototypes -Wc++-compat -Wl,--stack,8388608 --disable-auto-import --disable-runtime-pseudo-reloc -Wl,--large-address-aware ./flisp.o ./builtins.o ./string.o ./equalhash.o ./table.o ./iostream.o ./julia_extensions.o ./flmain.o -o flisp.exe ./libflisp.a ./../support/libsupport.a /d/jl/julia/usr/lib/libuv.a /d/jl/julia/usr/lib/libutf8proc.a  -Wl,--export-all-symbols -Wl,--version-script=/d/jl/julia/src/julia.expmap -Wl,--no-whole-archive -lpsapi -lkernel32 -lws2_32 -liphlpapi -lwinmm -ldbghelp -luserenv -lsecur32 -latomic -lole32 -lssp
D:/env/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/14.2.0/../../../../i686-w64-mingw32/bin/ld.exe: ./julia_extensions.o: in function `normalize':

$ file /d/jl/julia/usr/lib/libutf8proc.a
/d/jl/julia/usr/lib/libutf8proc.a: current ar archive

$ nm -g /d/jl/julia/usr/lib/libutf8proc.a
D:\env\msys64\mingw32\bin\nm.exe: utf8proc.o: file format not recognized

$ ar t /d/jl/julia/usr/lib/libutf8proc.a
utf8proc.o

$ objdump -f /d/jl/julia/usr/lib/libutf8proc.a
In archive D:/jl/julia/usr/lib/libutf8proc.a:
D:\env\msys64\mingw32\bin\objdump.exe: utf8proc.o: file format not recognized


$ ar x  /d/jl/julia/usr/lib/libutf8proc.a

$ ls utf8proc.o
utf8proc.o

$ file utf8proc.o
utf8proc.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped

$ objdump -f utf8proc.o
D:\env\msys64\mingw32\bin\objdump.exe: utf8proc.o: file format not recognized

$ file src/flisp/flisp.o
src/flisp/flisp.o: Intel i386 COFF object file, no line number info, not stripped, 14 sections, symbol offset=0x9f2be, 294 symbols, 1st section name ".text", 2nd section name ".data"

There is something wrong with libutf8proc.a, seems a version mismatch.

[vtjnash]

It looks like it is using the wrong compiler (host gcc). Only cross-compiler (XC_HOST-gcc) is supported on Windows.

[ajinkya-k]

@vtjnash do you mean only cross compilation using cygwin is supported?

[vtjnash]

msys2 is also mostly supported, though it presents itself as an odd sort of native cross-compiler, which can get you into some trouble when trying to build things if you don't force it to operate as a normal cross-compiler

[ajinkya-k]

Given that using msys2/mignw is brittle, maybe the msys2/mingw section should be removed from the build doc?

[ajinkya-k]

For what its worth, in a clean directory the build worked, using this PR #57451