ejabberd 16.12

[jcgraybill]

Updating ejabberd to 16.12. Version 16 is the first version I was able to build on Leopard/PPC, and 16.12 is the highest minor version of ejabberd 16.

Also switch to using openssl3.

Build successfully tested on Leopard/PPC, and sure enough I can connect Adium clients to the server. I would love help testing on Snow Leopard if anyone is able to, as the pam_appl.h workaround I needed to add is apparently not needed on SL, but I don't currently have a way to test that.

[sevan]

When attempting to link against openssl or openssl3, it fails for me on Tiger.

Compiling /private/tmp/ejabberd20250425-20607-m5p9hm/ejabberd-16.12/deps/fast_tls/c_src/fast_tls_drv.c
/private/tmp/ejabberd20250425-20607-m5p9hm/ejabberd-16.12/deps/fast_tls/c_src/fast_tls_drv.c: In function ‘setup_ecdh’:
/private/tmp/ejabberd20250425-20607-m5p9hm/ejabberd-16.12/deps/fast_tls/c_src/fast_tls_drv.c:386: error: ‘EC_KEY’ undeclared (first use in this function)
/private/tmp/ejabberd20250425-20607-m5p9hm/ejabberd-16.12/deps/fast_tls/c_src/fast_tls_drv.c:386: error: (Each undeclared identifier is reported only once
/private/tmp/ejabberd20250425-20607-m5p9hm/ejabberd-16.12/deps/fast_tls/c_src/fast_tls_drv.c:386: error: for each function it appears in.)
/private/tmp/ejabberd20250425-20607-m5p9hm/ejabberd-16.12/deps/fast_tls/c_src/fast_tls_drv.c:386: error: ‘ecdh’ undeclared (first use in this function)
/private/tmp/ejabberd20250425-20607-m5p9hm/ejabberd-16.12/deps/fast_tls/c_src/fast_tls_drv.c:393: error: ‘SSL_OP_SINGLE_ECDH_USE’ undeclared (first use in this function)
ERROR: compile failed while processing /private/tmp/ejabberd20250425-20607-m5p9hm/ejabberd-16.12/deps/fast_tls: rebar_abort

You can skip cloning the git repos manually, but patching the rebar.config to switch the urls from git:// protocol to https://. Need to add rebar as a build dep.

e.g.

--- rebar.config.orig   2025-04-25 13:51:37.000000000 +0100
+++ rebar.config        2025-04-25 13:51:50.000000000 +0100
@@ -27,7 +27,7 @@
 {port_specs, [{"priv/lib/fast_tls_drv.so", ["c_src/fast_tls_drv.c"]},
               {"priv/lib/p1_sha.so", ["c_src/p1_sha.c"]}]}.
 
-{deps, [{p1_utils, ".*", {git, "git://github.com/processone/p1_utils", {tag, "1.0.6"}}}]}.
+{deps, [{p1_utils, ".*", {git, "https://github.com/processone/p1_utils", {tag, "1.0.6"}}}]}.
 
 {clean_files, ["c_src/fast_tls_drv.gcda", "c_src/fast_tls_drv.gcno",
                "c_src/fast_sha.gcda", "c_src/fast_sha.gcno"]}.

One for me as a reminder:
Since the original autoconf'd source archives have disappeared and it requires to run autogen now, best to add autoconf & automake as build deps since the versions in Tiger are too old.

[sevan]

Ok, regarding cloning repos manually, the problem is that there's a bunch of deps which are listed in the rebar.config included in the root of the source for ejabberd. These deps use HTTPS urls, unfortunately the deps themselves include rebar.config files which list their own dependencies and there is where the git protocol URLs are showing up. For the goldrushdep, it is introduce via the Basho fork of lager, upping the version fixed the issue as they switched to HTTPS for a fork of goldrush made by Basho.

--- rebar.config.orig   2025-04-25 14:32:33.000000000 +0100
+++ rebar.config        2025-04-25 14:33:42.000000000 +0100
@@ -7,7 +7,7 @@
 %%% Created :  1 May 2013 by Evgeniy Khramtsov <[email protected]>
 %%%-------------------------------------------------------------------

-{deps, [{lager, ".*", {git, "https://github.com/basho/lager", {tag, "3.2.1"}}},
+{deps, [{lager, ".*", {git, "https://github.com/basho/lager", {tag, "3.2.4"}}},
         {p1_utils, ".*", {git, "https://github.com/processone/p1_utils", {tag, "1.0.6"}}},
         {cache_tab, ".*", {git, "https://github.com/processone/cache_tab", {tag, "1.0.5"}}},
         {fast_tls, ".*", {git, "https://github.com/processone/fast_tls", {tag, "1.0.9"}}},

Haven't make any progress on the OpenSSL build issue.

[jcgraybill]

When attempting to link against openssl or openssl3, it fails for me on Tiger.

Oh, interesting! Between version 15.07 and 16.12, ejabberd switched from using their tls library to a new one called fast_tls. I couldn't get tls to build on Leopard, but I wonder if fast_tls isn't backward compatible with Tiger.

It would be interesting to see whether it would be possible to hack tls back into ejabberd 16.12 and build it for Tiger. I don't actually know whether these two libraries are API-compatible. If that doesn't work, this might be a tough situation where there isn't a single version of ejabberd that builds on all three of Tiger, Leopard, and Snow Leopard.

In a case like that, what approach do you all like to take in this project? My mind is going to, is there some way to provide different software versions based on OS version? I haven't seen a formula that does this, but I've only read a very small percentage of the tigerbrew formulas. Other ideas could be to try cross-compiling and releasing a bottle for Tiger users, or to somehow mark the package as incompatible with Tiger. Or of course, to keep hacking at it until I get something to work across all three :)

I think the moral is, I really need to get Tiger and Snow Leopard onto some machines here so that I can test all of the target OS versions myself. There are a couple Mac Minis in the mail right now that should take care of that shortly.

You can skip cloning the git repos manually, but patching the rebar.config to switch the urls from git:// protocol to https://.

Oh, thanks for that tip. I've been trying to use inreplace whenever possible, too. The challenge I found with ejabberd is that goldrush is a transitive dependency, via lager, so the URL that needs to be fixed is in lager's own rebar.config. That file doesn't exist in the build directory until rebar compile has already been called - at which point rebar downloads lager, and then tries to use lager's rebar.config to fetch goldbar and fails. I don't know if there's a way to return control to tigerbrew in the midst of this process so that I can apply the needed patches. However, if rebar finds goldrush already downloaded, it skips trying to download it again. You'll see that I had to do this with three erlang libraries in disco.rb. It was also the only way I could figure out to patch the include paths in epam.

Need to add rebar as a build dep.

One for me as a reminder: Since the original autoconf'd source archives have disappeared and it requires to run autogen now, best to add autoconf & automake as build deps since the versions in Tiger are too old.

I'll go ahead and add all three of these to the branch in this PR and do a test build tonight or this weekend. (erlang is compiling right now, and wow does it take a long time.) True confession, I also realize I haven't tried a build that includes imagemagick and that might be a whole can of worms too.

Thanks as always for all the help, @sevan .

[jcgraybill]

Ok, regarding cloning repos manually, the problem is that there's a bunch of deps which are listed in the rebar.config included in the root of the source for ejabberd. These deps use HTTPS urls, unfortunately the deps themselves include rebar.config files which list their own dependencies and there is where the git protocol URLs are showing up. For the goldrushdep, it is introduce via the Basho fork of lager, upping the version fixed the issue as they switched to HTTPS for a fork of goldrush made by Basho.

Haha, looks like we were typing at the same time. This is a great find - it's not a very big version change for lager, so it looks like a more robust way to solve this.

[sevan]

When attempting to link against openssl or openssl3, it fails for me on Tiger.

Oh, interesting! Between version 15.07 and 16.12, ejabberd switched from using their tls library to a new one called fast_tls. I couldn't get tls to build on Leopard, but I wonder if fast_tls isn't backward compatible with Tiger.

It would be interesting to see whether it would be possible to hack tls back into ejabberd 16.12 and build it for Tiger. I don't actually know whether these two libraries are API-compatible. If that doesn't work, this might be a tough situation where there isn't a single version of ejabberd that builds on all three of Tiger, Leopard, and Snow Leopard.

In this case, I suspect the issues is not with Tiger itself, but it is being highlighted there since regardless of OS, we're trying to link to the openssl3 formula (also tried openssl) without success. I don't see anything about flags for pointing to a TLS library in the ./configure --help output

In a case like that, what approach do you all like to take in this project? My mind is going to, is there some way to provide different software versions based on OS version? I haven't seen a formula that does this, but I've only read a very small percentage of the tigerbrew formulas. Other ideas could be to try cross-compiling and releasing a bottle for Tiger users, or to somehow mark the package as incompatible with Tiger. Or of course, to keep hacking at it until I get something to work across all three :)

I did this for Freepascal which is not ready yet.
I think in this scenario, I suspect there's a bug in the build infrastructure because I have a current version of OpenSSL installed (1.1.1 and 3.50).

I think the moral is, I really need to get Tiger and Snow Leopard onto some machines here so that I can test all of the target OS versions myself. There are a couple Mac Minis in the mail right now that should take care of that shortly.

More machines running Tiger are always good :)
I can try testing on Snow Leopard over the weekend.

Oh, thanks for that tip. I've been trying to use inreplace whenever possible, too. The challenge I found with ejabberd is that goldrush is a transitive dependency, via lager, so the URL that needs to be fixed is in lager's own rebar.config. That file doesn't exist in the build directory until rebar compile has already been called - at which point rebar downloads lager, and then tries to use lager's rebar.config to fetch goldbar and fails. I don't know if there's a way to return control to tigerbrew in the midst of this process so that I can apply the needed patches. However, if rebar finds goldrush already downloaded, it skips trying to download it again. You'll see that I had to do this with three erlang libraries in disco.rb. It was also the only way I could figure out to patch the include paths in epam.

Yeah, I was sitting in the build directory, having spawned a shell when the build failed and generated the patches then, only to realise that the deps directory is not there at the start and the patching fails.

I'll go ahead and add all three of these to the branch in this PR and do a test build tonight or this weekend. (erlang is compiling right now, and wow does it take a long time.) True confession, I also realize I haven't tried a build that includes imagemagick and that might be a whole can of worms too.

Oh, and all the security issues with imagemagick :D

Thanks as always for all the help, @sevan .

Thank you for all your contributions so far @jcgraybill. It's cool to have a version of erlang to play with.
LFE looks very interesting. :)

[sevan]

As it currently stands, what does otool -L opt/ejabberd/lib/fast_tls-1.0.9/priv/lib/fast_tls_drv.so and otool -L opt/ejabberd/lib/fast_tls-1.0.9/priv/lib/p1_sha.so say on your system?
They should have linked to the openssl3 formula's libraries.

[jcgraybill]

It looks like they did link openssl3:

kabooties-emac:local kabootie$ otool -L opt/ejabberd/lib/fast_tls-1.0.9/priv/lib/fast_tls_drv.so
opt/ejabberd/lib/fast_tls-1.0.9/priv/lib/fast_tls_drv.so:
	/usr/local/opt/openssl3/lib/libssl.3.dylib (compatibility version 3.0.0, current version 3.0.0)
	/usr/local/opt/openssl3/lib/libcrypto.3.dylib (compatibility version 3.0.0, current version 3.0.0)
	/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.7)

kabooties-emac:local kabootie$ otool -L opt/ejabberd/lib/fast_tls-1.0.9/priv/lib/p1_sha.so
opt/ejabberd/lib/fast_tls-1.0.9/priv/lib/p1_sha.so:
	/usr/local/opt/openssl3/lib/libssl.3.dylib (compatibility version 3.0.0, current version 3.0.0)
	/usr/local/opt/openssl3/lib/libcrypto.3.dylib (compatibility version 3.0.0, current version 3.0.0)
	/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.7)

[sevan]

It looks like they did link openssl3:

How weird, not for me.
I'll take a look at testing on other OS X versions in the morning and report back if it has issues on them but I'll land this now in the meantime and fight the build on Tiger later. :)

Thanks again for patches.

[jcgraybill]

That is weird. I'll definitely be standing by to look into anything you find.

Also, coda: the build succeeded on Leopard/PPC with the --with-imagemagick flag, and I'm currently test-building the 32-bit build.

Thanks!

[sevan]

Build tested on OS X 10.5 i386 & 10.6 x86_64.
erlang needed a change to build on 10.6, but otherwise everything build without issue.
On both version OS, fast_tls library linked to openssl3 without issue.
I have a hunch what it may be on Tiger but I'll take a look another day.

I was wondering if maybe instead of fetching the deps during build, we could switch to using resources in the formula, that way brew will fetch everything, making it easy to integrate patches, and cache those downloads.
Not sure how you control where brew puts the resources though as the build expects to find them in deps/

[jcgraybill]

Using a resource is an interesting idea. It looks like this is the code that handles resources? https://github.com/mistydemeo/tigerbrew/blob/master/Library/Homebrew/resource.rb

I'm not familiar enough with the brew codebase (or experienced enough at reading Ruby) to immediately see a way to achieve "just unzip the zip file to the specified directory" with that, though, TBH.

[sevan]

I'm not familiar enough with the brew codebase (or experienced enough at reading Ruby)

Likewise but from the link you provided, doesn't target sort of do that?

[jcgraybill]

Indeed! I got a little tripped up expecting unpack to behave similarly to stage, but I just have to download & verify the zipfile directly in the formula, and it works great.

Here's a PR with this change: #1355

Let me also go revise the disco PR (https://github.com/mistydemeo/tigerbrew/pull/1350/files) to use the same approach.

Much nicer!