Skip to content

Commit 64a1f75

Browse files
IvoDDIvoDD
authored
Upgrade gcc to 11.2 and add sparrow dependency (#2355)
#### Reference Issues/PRs Monday ref: 8873104825 #### What does this implement or fix? This PR upgrades the manylinux image to use gcc-11.2.1 and adds the vcpkg sparrow dependency. This also required updating boost from 1.84 to 1.87. We were using g++-10 for the manylinux2014 image. Discussions with QS lead to deciding that we should upgrade compiler to gcc 11.2.1 to be able to compile sparrow 0.8.0. Adding the sparrow dependency required updating the vcpkg baseline to pick up the new package. The new baseline requires extra package version pinning to make sure everything is compatible with eachother. I've made sure to not upgrade any major sdks which already had pins. This PR only updates the version of boost from 1.84 to 1.87 to resolve a compilation issue with boost-regex. The boost upgrade is in a separate commit. #### Any other comments? Successful CI run from this branch's `Build and Test` workflow [here](https://github.com/man-group/ArcticDB/actions/runs/15063528758). It uses the updated arcticdb-dev docker image for the c++ tests instead of the clang one. Successful CI run of `Build with analisys tools` [here](https://github.com/man-group/ArcticDB/actions/runs/15068953136/job/42360192162). #### Checklist <details> <summary> Checklist for code changes... </summary> - [ ] Have you updated the relevant docstrings, documentation and copyright notice? - [ ] Is this contribution tested against [all ArcticDB's features](../docs/mkdocs/docs/technical/contributing.md)? - [ ] Do all exceptions introduced raise appropriate [error messages](https://docs.arcticdb.io/error_messages/)? - [ ] Are API changes highlighted in the PR description? - [ ] Is the PR labelled as enhancement or bug so it appears in autogenerated release notes? </details> <!-- Thanks for contributing a Pull Request to ArcticDB! Please ensure you have taken a look at: - ArcticDB's Code of Conduct: https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md - ArcticDB's Contribution Licensing: https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing -->
1 parent bc554c9 commit 64a1f75

File tree

12 files changed

+56
-256
lines changed

12 files changed

+56
-256
lines changed

‎.github/workflows/analysis_workflow.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,7 @@ jobs:
125125
name: Executes asv tests checks
126126
timeout-minutes: 120
127127
runs-on: ubuntu-latest
128-
container: ghcr.io/man-group/arcticdb-dev:latest
128+
container: ghcr.io/man-group/arcticdb-dev:${{ inputs.dev_image_tag }}
129129
env:
130130
SCCACHE_GHA_VERSION: ${{vars.SCCACHE_GHA_VERSION || 1}} # Setting this env var enables the caching
131131
VCPKG_NUGET_USER: ${{secrets.VCPKG_NUGET_USER || github.repository_owner}}

‎.github/workflows/static_analysis.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@
2828
- name: Setup build dependencies
2929
run: |
3030
sudo apt-get update
31-
sudo apt-get install -y gcc-10 g++-10 make mono-complete libkrb5-dev libsasl2-dev
31+
sudo apt-get install -y gcc-11 g++-11 make mono-complete libkrb5-dev libsasl2-dev
3232
3333
- name: Setup VCPKG cache
3434
run: |
@@ -44,12 +44,12 @@
4444
- name: CMake configure
4545
uses: lukka/run-cmake@v10
4646
env:
47-
CC: "gcc-10"
48-
CXX: "g++-10"
47+
CC: "gcc-11"
48+
CXX: "g++-11"
4949
with:
5050
cmakeListsTxtPath: ${{github.workspace}}/cpp/CMakeLists.txt
5151
configurePreset: linux-release
52-
configurePresetAdditionalArgs: "['-DVCPKG_INSTALL_OPTIONS=--clean-after-build', '-DCMAKE_C_COMPILER=gcc-10', '-DCMAKE_CXX_COMPILER=g++-10']"
52+
configurePresetAdditionalArgs: "['-DVCPKG_INSTALL_OPTIONS=--clean-after-build', '-DCMAKE_C_COMPILER=gcc-11', '-DCMAKE_CXX_COMPILER=g++-11']"
5353

5454
- name: Copy Coverity config
5555
run: cp ${{github.workspace}}/coverity.yaml ${{github.workspace}}/cpp/out/linux-release-build

‎build_tooling/build_many_linux_image.sh

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,10 +44,17 @@ RUN rpmkeys --import 'https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x3F
4444
s/#?exclude.*/exclude=.edu/' /etc/yum/pluginconf.d/fastestmirror.conf
4545
ADD sccache /usr/local/bin/
4646
RUN yum update -y && \
47-
yum install -y zip jq less devtoolset-11-gdb perl-IPC-Cmd \
48-
openssl-devel cyrus-sasl-devel devtoolset-10-libatomic-devel libcurl-devel python3-devel flex && \
47+
yum remove -y devtoolset-* && \
48+
yum install -y zip jq less devtoolset-11 devtoolset-11-gdb perl-IPC-Cmd \
49+
openssl-devel cyrus-sasl-devel devtoolset-11-libatomic-devel libcurl-devel python3-devel flex && \
4950
rpm -Uvh --nodeps \$(repoquery --location mono-{core,web,devel,data,wcf,winfx}) && \
5051
yum clean all && touch /etc/arcticdb_deps_installed
52+
ENV CC=/opt/rh/devtoolset-11/root/bin/gcc
53+
ENV CMAKE_C_COMPILER=/opt/rh/devtoolset-11/root/bin/gcc
54+
ENV CXX=/opt/rh/devtoolset-11/root/bin/g++
55+
ENV CMAKE_CXX_COMPILER=/opt/rh/devtoolset-11/root/bin/g++
56+
ENV LD_LIBRARY_PATH=/opt/rh/devtoolset-11/root/usr/lib64:/opt/rh/devtoolset-11/root/usr/lib:/opt/rh/devtoolset-11/root/usr/lib64/dyninst
57+
ENV PATH=\"/opt/rh/devtoolset-11/root/usr/bin/:$PATH\"
5158
LABEL io.arcticdb.cibw_ver=\"${cibuildwheel_ver}\" io.arcticdb.base=\"${manylinux_image}\"
5259
" > Dockerfile
5360

‎cpp/third_party/vcpkg_overlays/boost-filesystem/5864f397ccad30f6e73221b90bdac57a303b9752.patch

Lines changed: 0 additions & 13 deletions
This file was deleted.

‎cpp/third_party/vcpkg_overlays/boost-filesystem/portfile.cmake

Lines changed: 0 additions & 16 deletions
This file was deleted.

‎cpp/third_party/vcpkg_overlays/boost-filesystem/vcpkg.json

Lines changed: 0 additions & 86 deletions
This file was deleted.

‎cpp/vcpkg

Submodule vcpkg updated 3858 files

‎cpp/vcpkg.json

Lines changed: 29 additions & 122 deletions
Original file line numberDiff line numberDiff line change
@@ -37,50 +37,17 @@
3737
"default-features": false,
3838
"features": [ "s3", "identity-management" ]
3939
},
40-
{
41-
"name": "aws-crt-cpp",
42-
"version>=": "0.29.7"
43-
},
44-
{
45-
"name": "aws-c-mqtt",
46-
"version>=": "0.11.0"
47-
},
48-
{
49-
"name": "aws-c-s3",
50-
"version>=": "0.6.6"
51-
},
52-
{
53-
"name": "aws-c-io",
54-
"version>=": "0.14.18"
55-
},
56-
{
57-
"name": "aws-c-common",
58-
"version>=": "0.9.28"
59-
},
60-
{
61-
"name": "aws-c-auth",
62-
"version>=": "0.7.31"
63-
},
64-
{
65-
"name": "aws-c-cal",
66-
"version>=": "0.7.4"
67-
},
68-
{
69-
"name": "aws-c-http",
70-
"version>=": "0.8.10"
71-
},
72-
{
73-
"name": "aws-c-sdkutils",
74-
"version>=": "0.1.19"
75-
},
76-
{
77-
"name": "aws-c-event-stream",
78-
"version>=": "0.4.3"
79-
},
80-
{
81-
"name": "aws-checksums",
82-
"version>=": "0.1.20"
83-
},
40+
"aws-crt-cpp",
41+
"aws-c-mqtt",
42+
"aws-c-s3",
43+
"aws-c-io",
44+
"aws-c-common",
45+
"aws-c-auth",
46+
"aws-c-cal",
47+
"aws-c-http",
48+
"aws-c-sdkutils",
49+
"aws-c-event-stream",
50+
"aws-checksums",
8451
"boost-dynamic-bitset",
8552
"boost-interprocess",
8653
"boost-callable-traits",
@@ -104,91 +71,31 @@
10471
"azure-core-cpp",
10572
"azure-identity-cpp",
10673
"azure-storage-blobs-cpp",
107-
"benchmark"
74+
"benchmark",
75+
{
76+
"name": "arcticdb-sparrow",
77+
"version>=": "0.8.0"
78+
}
10879
],
10980
"overrides": [
11081
{ "name": "openssl", "version-string": "3.3.0" },
11182
{ "name": "arrow", "version": "18.0.0" },
11283
{ "name": "aws-sdk-cpp", "version": "1.11.474", "$note": "Update overlay json to upgrade; Upgrade to >=1.11.486 blocked by default integrity change" },
84+
{ "name": "aws-crt-cpp", "version": "0.29.7" },
85+
{ "name": "aws-c-mqtt", "version": "0.11.0" },
86+
{ "name": "aws-c-s3", "version": "0.6.6" },
87+
{ "name": "aws-c-io", "version": "0.14.18" },
88+
{ "name": "aws-c-common", "version": "0.9.28" },
89+
{ "name": "aws-c-auth", "version": "0.7.31" },
90+
{ "name": "aws-c-cal", "version": "0.7.4" },
91+
{ "name": "aws-c-http", "version": "0.8.10" },
92+
{ "name": "aws-c-sdkutils", "version": "0.1.19" },
93+
{ "name": "aws-c-event-stream", "version": "0.4.3" },
94+
{ "name": "aws-checksums", "version": "0.1.20" },
11395
{ "name": "azure-core-cpp", "version": "1.12.0" },
96+
{ "name": "azure-identity-cpp", "version": "1.6.0" },
11497
{ "name": "benchmark", "version": "1.9.0" },
11598
{ "name": "bitmagic", "version": "7.12.3" },
116-
{ "name": "boost-algorithm", "version": "1.84.0" },
117-
{ "name": "boost-align", "version": "1.84.0" },
118-
{ "name": "boost-any", "version": "1.84.0" },
119-
{ "name": "boost-array", "version": "1.84.0" },
120-
{ "name": "boost-assert", "version": "1.84.0" },
121-
{ "name": "boost-atomic", "version": "1.84.0" },
122-
{ "name": "boost-bind", "version": "1.84.0" },
123-
{ "name": "boost-build", "version": "1.84.0" },
124-
{ "name": "boost-callable-traits", "version": "1.84.0" },
125-
{ "name": "boost-chrono", "version": "1.84.0" },
126-
{ "name": "boost-circular-buffer", "version": "1.84.0" },
127-
{ "name": "boost-concept-check", "version": "1.84.0" },
128-
{ "name": "boost-config", "version": "1.84.0" },
129-
{ "name": "boost-container", "version": "1.84.0" },
130-
{ "name": "boost-container-hash", "version": "1.84.0" },
131-
{ "name": "boost-context", "version": "1.84.0" },
132-
{ "name": "boost-conversion", "version": "1.84.0" },
133-
{ "name": "boost-core", "version": "1.84.0" },
134-
{ "name": "boost-crc", "version": "1.84.0" },
135-
{ "name": "boost-date-time", "version": "1.84.0" },
136-
{ "name": "boost-detail", "version": "1.84.0" },
137-
{ "name": "boost-dynamic-bitset", "version": "1.84.0" },
138-
{ "name": "boost-endian", "version": "1.84.0" },
139-
{ "name": "boost-exception", "version": "1.84.0" },
140-
{ "name": "boost-filesystem", "version": "1.84.0" },
141-
{ "name": "boost-foreach", "version": "1.84.0" },
142-
{ "name": "boost-function", "version": "1.84.0" },
143-
{ "name": "boost-function-types", "version": "1.84.0" },
144-
{ "name": "boost-fusion", "version": "1.84.0" },
145-
{ "name": "boost-integer", "version": "1.84.0" },
146-
{ "name": "boost-interprocess", "version": "1.84.0" },
147-
{ "name": "boost-intrusive", "version": "1.84.0" },
148-
{ "name": "boost-io", "version": "1.84.0" },
149-
{ "name": "boost-iterator", "version": "1.84.0" },
150-
{ "name": "boost-lexical-cast", "version": "1.84.0" },
151-
{ "name": "boost-locale", "version": "1.84.0" },
152-
{ "name": "boost-math", "version": "1.84.0" },
153-
{ "name": "boost-modular-build-helper", "version": "1.84.0#3" },
154-
{ "name": "boost-move", "version": "1.84.0" },
155-
{ "name": "boost-mp11", "version": "1.84.0" },
156-
{ "name": "boost-mpl", "version": "1.84.0" },
157-
{ "name": "boost-multiprecision", "version": "1.84.0" },
158-
{ "name": "boost-multi-index", "version": "1.84.0" },
159-
{ "name": "boost-numeric-conversion", "version": "1.84.0" },
160-
{ "name": "boost-optional", "version": "1.84.0" },
161-
{ "name": "boost-phoenix", "version": "1.84.0" },
162-
{ "name": "boost-pool", "version": "1.84.0" },
163-
{ "name": "boost-predef", "version": "1.84.0" },
164-
{ "name": "boost-preprocessor", "version": "1.84.0" },
165-
{ "name": "boost-program-options", "version": "1.84.0" },
166-
{ "name": "boost-proto", "version": "1.84.0" },
167-
{ "name": "boost-random", "version": "1.84.0" },
168-
{ "name": "boost-range", "version": "1.84.0" },
169-
{ "name": "boost-ratio", "version": "1.84.0" },
170-
{ "name": "boost-rational", "version": "1.84.0" },
171-
{ "name": "boost-regex", "version": "1.84.0" },
172-
{ "name": "boost-scope-exit", "version": "1.84.0" },
173-
{ "name": "boost-serialization", "version": "1.84.0" },
174-
{ "name": "boost-smart-ptr", "version": "1.84.0" },
175-
{ "name": "boost-spirit", "version": "1.84.0" },
176-
{ "name": "boost-static-assert", "version": "1.84.0" },
177-
{ "name": "boost-system", "version": "1.84.0" },
178-
{ "name": "boost-thread", "version": "1.84.0" },
179-
{ "name": "boost-throw-exception", "version": "1.84.0" },
180-
{ "name": "boost-tokenizer", "version": "1.84.0" },
181-
{ "name": "boost-tuple", "version": "1.84.0" },
182-
{ "name": "boost-type-index", "version": "1.84.0" },
183-
{ "name": "boost-type-traits", "version": "1.84.0" },
184-
{ "name": "boost-typeof", "version": "1.84.0" },
185-
{ "name": "boost-uninstall", "version": "1.84.0" },
186-
{ "name": "boost-unordered", "version": "1.84.0" },
187-
{ "name": "boost-utility", "version": "1.84.0" },
188-
{ "name": "boost-variant", "version": "1.84.0" },
189-
{ "name": "boost-variant2", "version": "1.84.0" },
190-
{ "name": "boost-vcpkg-helpers", "version": "1.84.0" },
191-
{ "name": "boost-winapi", "version": "1.84.0" },
19299
{ "name": "curl", "version": "8.4.0" },
193100
{ "name": "double-conversion", "version": "3.2.1" },
194101
{ "name": "fmt", "version": "10.2.1" },
@@ -216,5 +123,5 @@
216123
"overlay-ports": ["third_party/vcpkg_overlays"]
217124
},
218125
"$note on builtin-baseline": "Remember to regenerate third_party/vcpkg_overlays",
219-
"builtin-baseline": "0c20b2a97c390e106150837042d921b0939e7ecb"
126+
"builtin-baseline": "533a5fda5c0646d1771345fb572e759283444d5f"
220127
}

‎docker/Dockerfile

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ FROM quay.io/pypa/manylinux_2_28_x86_64
22

33
RUN dnf update -y
44
RUN dnf remove -y 'gcc-toolset-*'
5-
RUN dnf install -y zip flex bison gcc-toolset-10 gcc-toolset-10-gdb gcc-toolset-10-libatomic-devel krb5-devel cyrus-sasl-devel openssl-devel \
5+
RUN dnf install -y zip flex bison gcc-toolset-11 gcc-toolset-11-gdb gcc-toolset-11-libatomic-devel krb5-devel cyrus-sasl-devel openssl-devel \
66
unzip tar epel-release jq wget libcurl-devel git-lfs \
77
python3.11-devel python3.11-pip perl-IPC-Cmd
88

@@ -18,9 +18,9 @@ RUN chmod 555 sccache
1818

1919
RUN cp sccache /usr/local/bin/
2020

21-
ENV CC=/opt/rh/gcc-toolset-10/root/bin/gcc
22-
ENV CMAKE_C_COMPILER=/opt/rh/gcc-toolset-10/root/bin/gcc
23-
ENV CXX=/opt/rh/gcc-toolset-10/root/bin/g++
24-
ENV CMAKE_CXX_COMPILER=/opt/rh/gcc-toolset-10/root/bin/g++
25-
ENV LD_LIBRARY_PATH=/opt/rh/gcc-toolset-10/root/usr/lib64:/opt/rh/gcc-toolset-10/root/usr/lib:/opt/rh/gcc-toolset-10/root/usr/lib64/dyninst
26-
ENV PATH=/opt/rh/devtoolset-10/root/usr/bin:/opt/python/cp311-cp311/bin:${PATH}
21+
ENV CC=/opt/rh/gcc-toolset-11/root/bin/gcc
22+
ENV CMAKE_C_COMPILER=/opt/rh/gcc-toolset-11/root/bin/gcc
23+
ENV CXX=/opt/rh/gcc-toolset-11/root/bin/g++
24+
ENV CMAKE_CXX_COMPILER=/opt/rh/gcc-toolset-11/root/bin/g++
25+
ENV LD_LIBRARY_PATH=/opt/rh/gcc-toolset-11/root/usr/lib64:/opt/rh/gcc-toolset-11/root/usr/lib:/opt/rh/gcc-toolset-11/root/usr/lib64/dyninst
26+
ENV PATH=/opt/rh/gcc-toolset-11/root/usr/bin:/opt/python/cp311-cp311/bin:${PATH}

‎docs/mkdocs/docs/technical/contributing.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -202,7 +202,7 @@ GitHub output [here](https://github.com/man-group/ArcticDB/pkgs/container/cibuil
202202
Setting up Linux
203203
----------------
204204
The codebase and build system can work with any reasonably recent Linux distribution with at least
205-
GCC 8 (10+ recommended) and CMake 3.12 (these instructions assume 3.21+).
205+
GCC 11 and CMake 3.12 (these instructions assume 3.21+).
206206

207207
A development install of Python 3.6+ (with `libpython.a` or `.so` and full headers) is also necessary.
208208
See [pybind11 configuration](#pybind11-configuration).
@@ -216,8 +216,8 @@ already have `mongod` available.
216216
### Dependencies by distro
217217
| Distro | Versions reported to work | Packages |
218218
|---|---|---|
219-
| Ubuntu | 20.04, 22.04 | build-essential g++-10 libpcre3-dev libsasl2-dev libsodium-dev libkrb5-dev libcurl4-openssl-dev python3-dev |
220-
| Centos | 7 | devtoolset-10-gcc-c++ openssl-devel cyrus-sasl-devel devtoolset-10-libatomic-devel libcurl-devel python3-devel |
219+
| Ubuntu | 20.04, 22.04 | build-essential g++-11 libpcre3-dev libsasl2-dev libsodium-dev libkrb5-dev libcurl4-openssl-dev python3-dev |
220+
| Centos | 7 | devtoolset-11-gcc-c++ openssl-devel cyrus-sasl-devel devtoolset-11-libatomic-devel libcurl-devel python3-devel |
221221

222222
Setting up Windows
223223
------------------

0 commit comments

Comments
 (0)