Compiling problems with Elixir 1.8, OTP21 and OSX 10.14.2

[andrelip]

Can't compile the dependency

System:

Erlang/OTP 21 [erts-10.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] [dtrace]

Elixir 1.8.0 (compiled with Erlang/OTP 21)

Output:

===> Compiling prometheus_process_collector
cc prometheus_process_collector_nif.o prometheus_process_info_macos.o -L/usr/local/opt/openssl/lib -shared -L /usr/local/Cellar/erlang/21.2.2/lib/erlang/lib/erl_interface-3.10.4/lib -lerl_interface -lei -lstdc++ -o /Users/andresouza/arvore/deps/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so
Undefined symbols for architecture x86_64:
  "_enif_make_atom", referenced from:
      on_load(enif_environment_t*, void**, unsigned long) in prometheus_process_collector_nif.o
  "_enif_make_double", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_int", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_list_from_array", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_long", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_tuple", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_ulong", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [/Users/andresouza/arvore/deps/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so] Error 1
===> Hook for compile failed!

[mberrueta]

same thing here.

#define ATOM(Id, Value)                         \
    {                                                              \
        Id = enif_make_atom(env, Value);        \
    }

there is any way to include it?
the code is https://github.com/erlang/otp/blob/master/erts/emulator/beam/erl_nif.c but no idea how to include / reference

[mberrueta]

I tried to help here, but my no knowledge in c make it impossible

here there is more details if helps in any way

matiasberrueta@mac:~/c/p/w|update_libs⚡*?
➤ env DEBUG=1  mix deps.compile prometheus_process_collector
===> Expanded command sequence to be run: []
===> Provider: {default,do}
===> Expanded command sequence to be run: [{default,app_discovery},
                                           {bare,compile}]
===> Provider: {default,app_discovery}
===> Provider: {bare,compile}
===> Compiling prometheus_process_collector
===> sh info:
	cwd: "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector"
	cmd: make -C c_src

===> 	opts: [use_stdout,
               {cd,"/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector"},
               {env,[{"REBAR_DEPS_DIR",
                      "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/lib"},
                     {"REBAR_BUILD_DIR",
                      "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default"},
                     {"REBAR_ROOT_DIR",
                      "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/."},
                     {"REBAR_CHECKOUTS_DIR",
                      "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_checkouts"},
                     {"REBAR_PLUGINS_DIR",
                      "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/plugins"},
                     {"REBAR_GLOBAL_CONFIG_DIR",
                      "/Users/matiasberrueta/.config/rebar3"},
                     {"REBAR_GLOBAL_CACHE_DIR",
                      "/Users/matiasberrueta/.cache/rebar3"},
                     {"REBAR_TEMPLATE_DIR",
                      "/Users/matiasberrueta/.config/rebar3/templates"},
                     {"REBAR_APP_DIRS",
                      "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/apps/*:/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/lib/*:/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/."},
                     {"REBAR_SRC_DIRS",[]},
                     {"ERLANG_ERTS_VER","10.3.4"},
                     {"ERLANG_ROOT_DIR",
                      "/usr/local/Cellar/erlang/21.3.7/lib/erlang"},
                     {"ERL",
                      "/usr/local/Cellar/erlang/21.3.7/lib/erlang/bin/erl"},
                     {"ERLC",
                      "/usr/local/Cellar/erlang/21.3.7/lib/erlang/bin/erlc"},
                     {"ERLANG_ARCH","64"},
                     {"ERLANG_TARGET","21.3.7-x86_64-apple-darwin18.5.0-64"},
                     {"ERLANG_LIB_DIR_erl_interface",
                      "/usr/local/Cellar/erlang/21.3.7/lib/erlang/lib/erl_interface-3.11.2"},
                     {"ERLANG_LIB_VER_erl_interface","3.11.2"}]},
               {abort_on_error,"Hook for compile failed!\n"}]

===> Port Cmd: make -C c_src
Port Opts: [{cd,"/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector"},
            {env,[{"REBAR_DEPS_DIR",
                   "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/lib"},
                  {"REBAR_BUILD_DIR",
                   "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default"},
                  {"REBAR_ROOT_DIR",
                   "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/."},
                  {"REBAR_CHECKOUTS_DIR",
                   "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_checkouts"},
                  {"REBAR_PLUGINS_DIR",
                   "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/plugins"},
                  {"REBAR_GLOBAL_CONFIG_DIR",
                   "/Users/matiasberrueta/.config/rebar3"},
                  {"REBAR_GLOBAL_CACHE_DIR",
                   "/Users/matiasberrueta/.cache/rebar3"},
                  {"REBAR_TEMPLATE_DIR",
                   "/Users/matiasberrueta/.config/rebar3/templates"},
                  {"REBAR_APP_DIRS",
                   "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/apps/*:/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/lib/*:/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/."},
                  {"REBAR_SRC_DIRS",[]},
                  {"ERLANG_ERTS_VER","10.3.4"},
                  {"ERLANG_ROOT_DIR",
                   "/usr/local/Cellar/erlang/21.3.7/lib/erlang"},
                  {"ERL","/usr/local/Cellar/erlang/21.3.7/lib/erlang/bin/erl"},
                  {"ERLC",
                   "/usr/local/Cellar/erlang/21.3.7/lib/erlang/bin/erlc"},
                  {"ERLANG_ARCH","64"},
                  {"ERLANG_TARGET","21.3.7-x86_64-apple-darwin18.5.0-64"},
                  {"ERLANG_LIB_DIR_erl_interface",
                   "/usr/local/Cellar/erlang/21.3.7/lib/erlang/lib/erl_interface-3.11.2"},
                  {"ERLANG_LIB_VER_erl_interface","3.11.2"}]},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

cc prometheus_process_collector_nif.o prometheus_process_info_macos.o -L/usr/local/opt/openssl/lib -shared -L /usr/local/Cellar/erlang/21.3.7/lib/erlang/lib/erl_interface-3.11.2/lib -lerl_interface -lei -lstdc++ -o /Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so
Undefined symbols for architecture x86_64:
  "_enif_make_atom", referenced from:
      on_load(enif_environment_t*, void**, unsigned long) in prometheus_process_collector_nif.o
  "_enif_make_double", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_int", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_list_from_array", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_long", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_tuple", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_ulong", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so] Error 1
===> Hook for compile failed!

** (Mix) Could not compile dependency :prometheus_process_collector, "/Users/matiasberrueta/.mix/rebar3 bare compile --paths "/Users/matiasberrueta/code/p/w/_build/dev/lib/*/ebin"" command failed. You can recompile this dependency with "mix deps.compile prometheus_process_collector", update it with "mix deps.update prometheus_process_collector" or clean it with "mix deps.clean prometheus_process_collector"
matiasberrueta@mac:~/c/p/w|update_libs⚡*?

I tried compiling myself https://github.com/erlang/otp that has the code but I don't know what else is missing

also now is not working with the previous version neither, so not sure what else was changed :(

[mberrueta]

https://bugs.erlang.org/browse/ERL-935

with
Erlang/OTP 21 [erts-10.3.4] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] [dtrace] Eshell V10.3.4 (abort with ^G) fails

but with

Erlang/OTP 21 [erts-10.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1]

Eshell V10.2  (abort with ^G)

works

[deadtrickster]

Just compiled here:

Erlang/OTP 21 [erts-10.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Eshell V10.3  (abort with ^G)

no idea what's up.

[josevalim]

@andrelip @mberrueta what is your mix-installed rebar3 version? You can run this:

~/.mix/rebar3 version

Please try running mix local.rebar and trying again.

[krisalyssa]

@josevalim:

$ ~/.asdf/installs/elixir/1.8.1/.mix/rebar3 version
rebar 3.6.1 on Erlang/OTP 21 Erts 10.3.4

[mberrueta]

now it's working for me, but I don't know what solve it. I update and restart everything

[josevalim]

@mberrueta what is your rebar3 version? I think this is actually a build tool issue, so perhaps it is worth closing the bug report in bugs.erlang.org until we can trim it down?

[mberrueta]

~/.mix/rebar3 --version
rebar 3.6.1 on Erlang/OTP 21 Erts 10.3.5

I opened a rebar issue before, but they said that wasn't related
erlang/rebar3#2064

[krisalyssa]

@andrelip @mberrueta I think your problem may be this:

-L/usr/local/opt/openssl/lib

I had CPPFLAGS and LDFLAGS set in my environment for compiling in support for MySQL, and their being set was preventing the prometheus_process_collector Makefile from setting them correctly itself. I unset both env vars and now this compiles fine.

[krisalyssa]

My Makefile skills are rusty, but I think the line at https://github.com/deadtrickster/prometheus_process_collector/blob/master/c_src/Makefile#L25 says "set LDFLAGS if it's not already set". It probably should be appending to LDFLAGS instead.

[deadtrickster]

that's true!

[krisalyssa]

@deadtrickster I'm not trying to duck the responsibility of creating a PR for my suggestion -- I'm just elbows deep in something else at the moment. 😄

[deadtrickster]

oh, no expectations at all. just agreeing.

[clifton-mcintosh]

This problem is occurring for me as well. While troubleshooting, one of my colleagues and I tried running make on the c_src and it fails. My colleague, whose experience with C is more extensive than mine, says it seems to him like the problem is with ld. I have a newer version of ld. Here is the output from running make on the c_src.

https://gist.github.com/clifton-mcintosh/9e5c9d546f8f45bd1d12377c1d709ed9

This is my version of ld, which fails when running make on the c_src:

ld -v
@(#)PROGRAM:ld  PROJECT:ld64-450.3
BUILD 18:45:16 Apr  4 2019
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
LTO support using: LLVM version 10.0.1, (clang-1001.0.46.4) (static support for 22, runtime is 22)
TAPI support using: Apple TAPI version 10.0.1 (tapi-1001.0.4.1)

My colleague's earlier version of ld, which works when running make on the c_src.

ld -v
@(#)PROGRAM:ld  PROJECT:ld64-409.12
BUILD 02:04:28 Aug 14 2018
configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
LTO support using: LLVM version 10.0.0, (clang-1000.10.44.2) (static support for 21, runtime is 21)
TAPI support using: Apple TAPI version 10.0.0 (tapi-1000.10.8)