Build instructions are out of date

[JanEbbing]

I'm following the build instructions for Ubuntu on the website:

docker run -it --name ubuntu ubuntu:24.10 bash
## in docker image shell:

# apt-get update
# apt-get install \
    g++ \
    cmake \
    make \
    ninja-build \
    bison flex \
    git curl \
    rsync m4 \
    libzstd-dev \
    libboost-all-dev \
    libevent-dev \
    libdouble-conversion-dev \
    libgoogle-glog-dev \
    libgflags-dev \
    libiberty-dev \
    liblz4-dev \
    liblzma-dev \
    libsnappy-dev \
    zlib1g-dev \
    binutils-dev \
    libjemalloc-dev \
    libssl-dev \
    pkg-config \
    libunwind-dev \
    libsodium-dev \
    libpcre3-dev \
    libfftw3-dev \
    libxxhash-dev \
    libgtest-dev \
    libfmt-dev \
    clang-12 \
    llvm-12 \
    libclang-12-dev

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Package clang-12 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'clang-12' has no installation candidate

The closest version available is clang-14, which I am trying now. See ubuntu packages list

[JanEbbing]

EDIT: Adding libgmp-dev as a required install

I also needed to # apt-get install libfast-float-dev libgmp-dev (this is a dependency of folly, so maybe the issue should go there?), before that I got

CMake Error at /usr/share/cmake-3.30/Modules/FindPackageHandleStandardArgs.cmake:233 (message):
  Could NOT find FastFloat (missing: FASTFLOAT_INCLUDE_DIR)
Call Stack (most recent call first):
  /usr/share/cmake-3.30/Modules/FindPackageHandleStandardArgs.cmake:603 (_FPHSA_FAILURE_MESSAGE)
  CMake/FindFastFloat.cmake:25 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  CMake/folly-deps.cmake:55 (find_package)
  CMakeLists.txt:135 (include)

when running ./install_deps.sh

[JanEbbing]

EDIT: This seems to mean my ghc version is too new. Trying with stack now....
EDIT 2: The following helped:

## Install ghc 9.2.8, last version with base < 4.17 like the project requires
# ghcup install ghc 9.2.8
# ghcup set ghc 9.2.8
## Continue on from build guide at step "make"
# make

I am currently getting

# make
Error: [Cabal-7107]
Could not resolve dependencies:
[__0] trying: fb-stubs-0.1.0.0 (user goal)
[__1] next goal: base (dependency of fb-stubs)
[__1] rejecting: base-4.17.2.1/installed-4.17.2.1 (conflict: fb-stubs => base>=4.11.1.0 && <4.17)
[__1] skipping: base; 4.21.0.0, 4.20.0.1, 4.20.0.0, 4.19.2.0, 4.19.1.0, 4.19.0.0, 4.18.2.1, 4.18.2.0, 4.18.1.0, 4.18.0.0, 4.17.2.1, 4.17.2.0, 4.17.1.0, 4.17.0.0 (has the same characteristics that caused the previous version to fail: excluded by constraint '>=4.11.1.0 && <4.17' from 'fb-stubs')
[__1] rejecting: base; 4.16.4.0, 4.16.3.0, 4.16.2.0, 4.16.1.0, 4.16.0.0, 4.15.1.0, 4.15.0.0, 4.14.3.0, 4.14.2.0, 4.14.1.0, 4.14.0.0, 4.13.0.0, 4.12.0.0, 4.11.1.0, 4.11.0.0, 4.10.1.0, 4.10.0.0, 4.9.1.0, 4.9.0.0, 4.8.2.0, 4.8.1.0, 4.8.0.0, 4.7.0.2, 4.7.0.1, 4.7.0.0, 4.6.0.1, 4.6.0.0, 4.5.1.0, 4.5.0.0, 4.4.1.0, 4.4.0.0, 4.3.1.0, 4.3.0.0, 4.2.0.2, 4.2.0.1, 4.2.0.0, 4.1.0.0, 4.0.0.0, 3.0.3.2, 3.0.3.1 (constraint from non-reinstallable package requires installed instance)
[__1] fail (backjumping, conflict set: base, fb-stubs)
After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: base, fb-stubs

(cd hsthrift && make CABAL="cabal --jobs --ghc-options='' -vnormal+nowrap --project-file=/Glean/cabal.project  " compiler)
make[1]: Entering directory '/Glean/hsthrift'
cabal --jobs --ghc-options='' -vnormal+nowrap --project-file=/Glean/cabal.project   build thrift-compiler
Warning: Specifying an absolute path to the project file is deprecated. Use --project-dir to set the project's directory.
Resolving dependencies...
Error: [Cabal-7107]
Could not resolve dependencies:
[__0] trying: fb-stubs-0.1.0.0 (user goal)
[__1] next goal: base (dependency of fb-stubs)
[__1] rejecting: base-4.17.2.1/installed-4.17.2.1 (conflict: fb-stubs => base>=4.11.1.0 && <4.17)
[__1] skipping: base; 4.21.0.0, 4.20.0.1, 4.20.0.0, 4.19.2.0, 4.19.1.0, 4.19.0.0, 4.18.2.1, 4.18.2.0, 4.18.1.0, 4.18.0.0, 4.17.2.1, 4.17.2.0, 4.17.1.0, 4.17.0.0 (has the same characteristics that caused the previous version to fail: excluded by constraint '>=4.11.1.0 && <4.17' from 'fb-stubs')
[__1] rejecting: base; 4.16.4.0, 4.16.3.0, 4.16.2.0, 4.16.1.0, 4.16.0.0, 4.15.1.0, 4.15.0.0, 4.14.3.0, 4.14.2.0, 4.14.1.0, 4.14.0.0, 4.13.0.0, 4.12.0.0, 4.11.1.0, 4.11.0.0, 4.10.1.0, 4.10.0.0, 4.9.1.0, 4.9.0.0, 4.8.2.0, 4.8.1.0, 4.8.0.0, 4.7.0.2, 4.7.0.1, 4.7.0.0, 4.6.0.1, 4.6.0.0, 4.5.1.0, 4.5.0.0, 4.4.1.0, 4.4.0.0, 4.3.1.0, 4.3.0.0, 4.2.0.2, 4.2.0.1, 4.2.0.0, 4.1.0.0, 4.0.0.0, 3.0.3.2, 3.0.3.1 (constraint from non-reinstallable package requires installed instance)
[__1] fail (backjumping, conflict set: base, fb-stubs)
After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: base, fb-stubs

make[1]: *** [Makefile:16: compiler] Error 1
make[1]: Leaving directory '/Glean/hsthrift'
make: *** [Makefile:233: thrift-compiler] Error 2

Trying to find a way to fix this

[JanEbbing]

Edit: Also need to # apt-get install clang-15 llvm-15 libclang-15-dev

make works eventually, make test fails. Internet search recommended cabal build all afterwards, which is running now...

[pepeiborra]

Hey @JanEbbing thanks for reporting this!

• fast-float is a new Folly dependency. Our CI system was updated to include it in Add libfast-float-dev to RPM deps #467 but we forgot to update the build instructions. I'll will update them
• The cabal error you are seeing is fixed by running cabal update. I'll add a note to the build instructions
• In general, I recommend using the CI build script as a reference:

https://github.com/facebookincubator/Glean/blob/main/.github/workflows/ci.yml#L2

[mohamedmansour]

@pepeiborra @JanEbbing how did you get fast-float working on Ubuntu?

I installed sudo apt install libfast-float-dev

I am getting errors after install dependencies. To solve this problem, I had to download fast-float from source and install the header to /usr/lib in Ubuntu 24.04

fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
[12/353] Building CXX object CMakeFiles/folly_base.dir/folly/Conv.cpp.o
FAILED: CMakeFiles/folly_base.dir/folly/Conv.cpp.o
/usr/bin/ccache /usr/bin/c++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFMT_SHARED -DFOLLY_XLOG_STRIP_PREFIXES=\"/tmp/fbcode_builder_getdeps-ZhomeZmmansourZcodeZGleanZhsthriftZbuildZfbcode_builder/repos/github.com-facebook-folly.git:/tmp/fbcode_builder_getdeps-ZhomeZmmansourZcodeZGleanZhsthriftZbuildZfbcode_builder/build/folly\" -DGFLAGS_IS_A_DLL=0 -D_GNU_SOURCE -D_REENTRANT -I/tmp/fbcode_builder_getdeps-ZhomeZmmansourZcodeZGleanZhsthriftZbuildZfbcode_builder/repos/github.com-facebook-folly.git -I/tmp/fbcode_builder_getdeps-ZhomeZmmansourZcodeZGleanZhsthriftZbuildZfbcode_builder/build/folly -I/usr/include/libiberty -I/home/mmansour/.hsthrift/include -march=haswell -O2 -g -DNDEBUG -std=gnu++17 -fPIC -fcoroutines -g -finput-charset=UTF-8 -fsigned-char -Wall -Wno-deprecated -Wno-deprecated-declarations -Wno-sign-compare -Wno-unused -Wuninitialized -Wunused-label -Wunused-result -Wshadow-compatible-local -Wno-noexcept-type -faligned-new -fopenmp -MD -MT CMakeFiles/folly_base.dir/folly/Conv.cpp.o -MF CMakeFiles/folly_base.dir/folly/Conv.cpp.o.d -o CMakeFiles/folly_base.dir/folly/Conv.cpp.o -c /tmp/fbcode_builder_getdeps-ZhomeZmmansourZcodeZGleanZhsthriftZbuildZfbcode_builder/repos/github.com-facebook-folly.git/folly/Conv.cpp
/tmp/fbcode_builder_getdeps-ZhomeZmmansourZcodeZGleanZhsthriftZbuildZfbcode_builder/repos/github.com-facebook-folly.git/folly/Conv.cpp: In function ‘folly::Expected<Tgt, folly::ConversionCode> folly::detail::str_to_floating_fast_float_from_chars(folly::StringPiece*)’:
/tmp/fbcode_builder_getdeps-ZhomeZmmansourZcodeZGleanZhsthriftZbuildZfbcode_builder/repos/github.com-facebook-folly.git/folly/Conv.cpp:365:33: error: ‘allow_leading_plus’ is not a member of ‘fast_float::chars_format’
  365 |       fast_float::chars_format::allow_leading_plus};
      |                                 ^~~~~~~~~~~~~~~~~~
/tmp/fbcode_builder_getdeps-ZhomeZmmansourZcodeZGleanZhsthriftZbuildZfbcode_builder/repos/github.com-facebook-folly.git/folly/Conv.cpp: In instantiation of ‘folly::Expected<Tgt, folly::ConversionCode> folly::detail::str_to_floating_fast_float_from_chars(folly::StringPiece*) [with T = float; folly::StringPiece = folly::Range<const char*>]’:
/tmp/fbcode_builder_getdeps-ZhomeZmmansourZcodeZGleanZhsthriftZbuildZfbcode_builder/repos/github.com-facebook-folly.git/folly/Conv.cpp:379:64:   required from here
/tmp/fbcode_builder_getdeps-ZhomeZmmansourZcodeZGleanZhsthriftZbuildZfbcode_builder/repos/github.com-facebook-folly.git/folly/Conv.cpp:363:29: error: no matching function for call to ‘fast_float::parse_options::parse_options(<brace-enclosed initializer list>)’
  363 |   fast_float::parse_options options{
      |                             ^~~~~~~
In file included from /tmp/fbcode_builder_getdeps-ZhomeZmmansourZcodeZGleanZhsthriftZbuildZfbcode_builder/repos/github.com-facebook-folly.git/folly/Conv.cpp:24:
/usr/include/fast_float/fast_float.h:21:22: note: candidate: ‘constexpr fast_float::parse_options::parse_options(fast_float::chars_format, char)’
   21 |   constexpr explicit parse_options(chars_format fmt = chars_format::general,
      |                      ^~~~~~~~~~~~~
/usr/include/fast_float/fast_float.h:21:22: note:   conversion of argument 1 would be ill-formed:
/usr/include/fast_float/fast_float.h:20:8: note: candidate: ‘constexpr fast_float::parse_options::parse_options(const fast_float::parse_options&)’
   20 | struct parse_options {
      |        ^~~~~~~~~~~~~
/usr/include/fast_float/fast_float.h:20:8: note:   conversion of argument 1 would be ill-formed:
/usr/include/fast_float/fast_float.h:20:8: note: candidate: ‘constexpr fast_float::parse_options::parse_options(fast_float::parse_options&&)’
/usr/include/fast_float/fast_float.h:20:8: note:   conversion of argument 1 would be ill-formed:
/tmp/fbcode_builder_getdeps-ZhomeZmmansourZcodeZGleanZhsthriftZbuildZfbcode_builder/repos/github.com-facebook-folly.git/folly/Conv.cpp: In instantiation of ‘folly::Expected<Tgt, folly::ConversionCode> folly::detail::str_to_floating_fast_float_from_chars(folly::StringPiece*) [with T = double; folly::StringPiece = folly::Range<const char*>]’:
/tmp/fbcode_builder_getdeps-ZhomeZmmansourZcodeZGleanZhsthriftZbuildZfbcode_builder/repos/github.com-facebook-folly.git/folly/Conv.cpp:381:65:   required from here
/tmp/fbcode_builder_getdeps-ZhomeZmmansourZcodeZGleanZhsthriftZbuildZfbcode_builder/repos/github.com-facebook-folly.git/folly/Conv.cpp:363:29: error: no matching function for call to ‘fast_float::parse_options::parse_options(<brace-enclosed initializer list>)’
  363 |   fast_float::parse_options options{
      |                             ^~~~~~~
/usr/include/fast_float/fast_float.h:21:22: note: candidate: ‘constexpr fast_float::parse_options::parse_options(fast_float::chars_format, char)’
   21 |   constexpr explicit parse_options(chars_format fmt = chars_format::general,
      |                      ^~~~~~~~~~~~~
/usr/include/fast_float/fast_float.h:21:22: note:   conversion of argument 1 would be ill-formed:
/usr/include/fast_float/fast_float.h:20:8: note: candidate: ‘constexpr fast_float::parse_options::parse_options(const fast_float::parse_options&)’
   20 | struct parse_options {
      |        ^~~~~~~~~~~~~
/usr/include/fast_float/fast_float.h:20:8: note:   conversion of argument 1 would be ill-formed:
/usr/include/fast_float/fast_float.h:20:8: note: candidate: ‘constexpr fast_float::parse_options::parse_options(fast_float::parse_options&&)’
/usr/include/fast_float/fast_float.h:20:8: note:   conversion of argument 1 would be ill-formed:
[31/353] Building CXX object CMakeFiles/folly_base.dir/folly/Subprocess.cpp.o
ninja: build stopped: subcommand failed.
Command '['/usr/bin/cmake', '--build', '/tmp/fbcode_builder_getdeps-ZhomeZmmansourZcodeZGleanZhsthriftZbuildZfbcode_builder/build/folly', '--target', 'install', '--config', 'RelWithDebInfo', '-j', '20']' returned non-zero exit status 1.
!! Failed

[pepeiborra]

Upgrade to Ubuntu 25

[simonmar]

I suspect we need to tell the build system to install fast-float from source instead of using the RPM. There was already a fix landed for this in getdeps: facebookincubator/hsthrift@b55e63e

We probably need a tweak in our build setup somewhere. I'll try to take a look. Requiring Ubuntu 25 is too restrictive, IMO.

[simonmar]

See facebookincubator/hsthrift#145

[mohamedmansour]

@pepeiborra that should not be the answer Ubuntu 25 isn't even in Beta, The final release is April 17th, shouldn't we just support the latest LTS instead of Canary OS?

[pepeiborra]

I didn't even realise that Ubuntu 25 was a canary release. But I shall note that we (Meta) have not made any commitments to stay on Ubuntu LTS releases. PRs to improve suboptimal choices are welcome!

[simonmar]

#498 goes back to Ubuntu 24