Unable to install on non-Intel Macbook

[giblet22]

Describe the bug
It appears that when installing on an M1 Mac, when installing postgres, the install is failing as it tries to download the aarch64 version as opposed to the x86_64 version

To Reproduce
Steps to reproduce the behavior:

1. On M1 or newer Mac
2. make -f Makefile.docker up
3. Fails at postgres install step 6

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots

 > [postgres stage-0 6/8] RUN /tmp/postgres-install.sh 13 && rm -f /tmp/*.sh:                                                                                                             
8.165 Delta RPMs disabled because /usr/bin/applydeltarpm not installed.                                                                                                                   
20.84 warning: /var/cache/yum/aarch64/7/pgdg13/packages/postgresql13-contrib-13.16-2PGDG.rhel7.aarch64.rpm: Header V4 RSA/SHA1 Signature, key ID 55b374b8: NOKEY                          
20.84 Public key for postgresql13-contrib-13.16-2PGDG.rhel7.aarch64.rpm is not installed                                                                                                  
21.39 Importing GPG key 0x73E3B907:                                                                                                                                                       
21.39  Userid     : "PostgreSQL RPM Repository <pgsql-pkg-yum@lists.postgresql.org>"                                                                                                      
21.39  Fingerprint: f245 f0bf 96ac 1827 44ca ff2e 64fa ce11 73e3 b907                                                                                                                     
21.39  From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-PGDG-13                                                                                                                                  
21.40                                                                                                                                                                                     
21.40                                                                                                                                                                                     
21.40 Public key for postgresql13-libs-13.16-2PGDG.rhel7.aarch64.rpm is not installed                                                                                                     
21.40 
21.40 
21.40  Failing package is: postgresql13-libs-13.16-2PGDG.rhel7.aarch64
21.40  GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-PGDG-13
21.40 

Desktop (please complete the following information):

• OS: Mac Sonoma 14.6.1

Additional context
Add any other context about the problem here.

[hbruch]

Besides this issue, I encountered some more on Mac:

 > [core-services stage-0  7/20] RUN yum install -y         armadillo-10.8.2         devtoolset-8         devtoolset-8-libasan-devel         geos-3.11.3         geos-devel-3.11.3         glpk-4.64         glpk-devel-4.64         gdal-3.8.0         gdal-devel-3.8.0         gdal-python-tools-3.8.0         google-chrome-stable-$google_chrome_version         libgeotiff-1.6.0         libgeotiff-devel-1.6.0         liboauthcpp-0.1.0         liboauthcpp-devel-0.1.0         libphonenumber-8.12.39         libphonenumber-devel-8.12.39         libpostal-1.1         libpostal-data-1.1         libpostal-devel-1.1         npm-8.19.4         nodejs-16.20.2         nodejs-devel-16.20.2         nodejs-docs-16.20.2         nodejs-libs-16.20.2         postgresql13         postgresql13-contrib         postgresql13-server         proj-7.2.1         proj-devel-7.2.1         stxxl-1.3.1         stxxl-devel-1.3.1         v8-devel-9.4.146.26:
0.265 Loaded plugins: fastestmirror, ovl
0.357 Loading mirror speeds from cached hostfile
6.370  * epel: d2lzkl7pfhq30w.cloudfront.net
7.667 http://vault.centos.org/centos/7/sclo/aarch64/rh/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found
...
7.672 failure: repodata/repomd.xml from centos-sclo-rh: [Errno 256] No more mirrors to try.
7.672 http://vault.centos.org/centos/7/sclo/aarch64/rh/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found

which I worked around changing the repo url http://vault.centos.org/centos to http://vault.centos.org/altarch

diff --git a/docker/Dockerfile.core-services b/docker/Dockerfile.core-services
index faa8b9cd7..cbe4fbd4b 100644
--- a/docker/Dockerfile.core-services
+++ b/docker/Dockerfile.core-services
@@ -44,7 +44,8 @@ RUN sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo && \
 
 RUN yum -y install epel-release centos-release-scl
 
-RUN sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo && \
+RUN sed -i s/mirror.centos.org\\/centos\\/7\\/sclo/vault.centos.org\\/altarch\\/7\\/sclo/g /etc/yum.repos.d/CentOS-SCLo*.repo && \
+    sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo && \
     sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/*.repo && \
     sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/*.repo 

And additionally this for frontend, related to nodesource/distributions#1617:

=> ERROR [frontend stage-0  6/10] RUN yum -y install         bzip2         git         git-core         nodejs-devel yarn         google-chrome-stable-91.0.4472.114 &&     yum -q -y clean all && rm -f /tmp*.sh                        33.0s
...
32.89 Package nodejs-devel-16.20.2-1nodesource.aarch64.rpm is not signed

which can be worked around via

diff --git a/docker/Dockerfile.frontend b/docker/Dockerfile.frontend
index 7ddf9e04d..35daff976 100644
--- a/docker/Dockerfile.frontend
+++ b/docker/Dockerfile.frontend
@@ -27,6 +27,8 @@ RUN sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo && \
 
 RUN yum -q -y install epel-release
 
+RUN curl -sL https://rpm.nodesource.com/setup_16.x | bash -
+
 RUN yum -y install \
         bzip2 \
         git \

[hbruch]

For aarch64, the key in /scripts/yum/pgdg-repo.sh needs to be replaced by https://ftp.postgresql.org/pub/repos/yum/keys/PGDG-RPM-GPG-KEY-AARCH64-RHEL7. See also https://yum.postgresql.org/news/pgdg-rpm-repo-gpg-key-update/

[olearytd]

I don't think we want to update everything to Mac silicone / aarch64 since the source containers aren't aarch64 and rpms are built for el7.

We're working on upgrading everything to EL9 but in the meantime I am testing a change to utilize docker buildx --platform flag to specify that the build platform is DOCKER_DEFAULT_PLATFORM=linux/amd64

this will work on both linux and Mac silicone. I'll put in a PR once I have the build test finished. The changes will be part of the makefile so you won't need to update how you're building on your end if this works

[olearytd]

#5797

[olearytd]

build succeeded on our end and we've merged the PR

[hbruch]

Thanks for looking into this.

Using db4020e core-services does not start correctly:

$ docker logs hootenanny-core-services-1
...
/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8/ld.gold: error: tmp/release/ChangesetReplacementCreatorTest.o: file is empty
/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8/ld.gold: error: tmp/release/EdgeDistanceExtractorTest.o: file is empty
/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8/ld.gold: error: tmp/release/HausdorffDistanceExtractorTest.o: file is empty
/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8/ld.gold: error: tmp/release/HistogramTest.o: file is empty
/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8/ld.gold: error: tmp/release/IntersectionOverUnionExtractorTest.o: file is empty
/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8/ld.gold: error: tmp/release/NameExtractorTest.o: file is empty
/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8/ld.gold: error: tmp/release/RelationMemberSimilarityExtractorTest.o: file is empty
/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8/ld.gold: error: tmp/release/SampledAngleHistogramExtractorTest.o: file is empty
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:1216: ../lib/libHootCoreTest.so.1.0.0] Error 1
make[3]: Leaving directory '/var/lib/hootenanny/hoot-core-test'
make[2]: *** [Makefile.qmake:181: sub-hoot-core-test-make_first-ordered] Error 2
make[2]: Leaving directory '/var/lib/hootenanny'
make[1]: *** [Makefile.hoot:357: qt-make] Error 1
make[1]: Leaving directory '/var/lib/hootenanny'
make: *** [Makefile:884: core] Error 2

My config: Mac Sonoma 14.5 (23F79), Docker version 28.2.2, build e6534b4.

I guess I'll wait for EL9. If I can assist, e.g. testing, please let me know.

[olearytd]

I also see a different issue when I run mine I had just tested the build error. We can continue looking into this and let you know of updates

[hbruch]

@olearytd do you have a roadmap/timeframe for porting to EL9? Is there already some work done and a public branch available to test/contribute to?