KMC compilation error when compiling from source

[Sawwave]

When compiling cuttlefish from source, the included kmc_core seems to have compilation errors due to an incorrect #include.

Specifically, the following errors are generated.

In file included from kmc_core/mem_disk_file.cpp:12:
kmc_core/critical_error_handler.h: In member function ‘void CCriticalErrorHandler::HandleCriticalError(const std::string&)’:
kmc_core/critical_error_handler.h:77:28: error: ‘runtime_error’ is not a member of ‘std’
77 |                 throw std::runtime_error(msg);
|                            ^~~~~~~~~~~~~
kmc_core/critical_error_handler.h:7:1: note: ‘std::runtime_error’ is defined in header ‘<stdexcept>’; this is probably fixable by adding ‘#include <stdexcept>’
6 | #include <condition_variable>
+++ |+#include <stdexcept>
7 |
In file included from kmc_core/queues.h:25,
from kmc_core/params.h:16,
from kmc_core/bkb_writer.h:14,
from kmc_core/bkb_writer.cpp:11:
kmc_core/critical_error_handler.h: In member function ‘void CCriticalErrorHandler::HandleCriticalError(const std::string&)’:
kmc_core/critical_error_handler.h:77:28: error: ‘runtime_error’ is not a member of ‘std’
77 |                 throw std::runtime_error(msg);
|                            ^~~~~~~~~~~~~
kmc_core/critical_error_handler.h:7:1: note: ‘std::runtime_error’ is defined in header ‘<stdexcept>’; this is probably fixable by adding ‘#include <stdexcept>’
6 | #include <condition_variable>
+++ |+#include <stdexcept>
7 |
make[3]: *** [Makefile:118: kmc_core/mem_disk_file.o] Error 1
make[3]: *** Waiting for unfinished jobs....
In file included from kmc_core/queues.h:25,
from kmc_core/params.h:16,
from kmc_core/bkb_reader.h:15,
from kmc_core/bkb_reader.cpp:11:
kmc_core/critical_error_handler.h: In member function ‘void CCriticalErrorHandler::HandleCriticalError(const std::string&)’:
kmc_core/critical_error_handler.h:77:28: error: ‘runtime_error’ is not a member of ‘std’
77 |                 throw std::runtime_error(msg);
|                            ^~~~~~~~~~~~~
kmc_core/critical_error_handler.h:7:1: note: ‘std::runtime_error’ is defined in header ‘<stdexcept>’; this is probably fixable by adding ‘#include <stdexcept>’
6 | #include <condition_variable>
+++ |+#include <stdexcept>
7 |
make[3]: *** [Makefile:118: kmc_core/bkb_writer.o] Error 1
make[3]: *** [Makefile:118: kmc_core/bkb_reader.o] Error 1
make[2]: *** [CMakeFiles/prj_kmc.dir/build.make:86: prj_kmc-prefix/src/prj_kmc-stamp/prj_kmc-build] Error 2
make[1]: *** [CMakeFiles/Makefile2:104: CMakeFiles/prj_kmc.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

Is the wrong KMC version being downloaded with curl in the cmake build? Currently, the cmake file specifies the following for KMC:

# Prepare the `kmc` library — required by the Cuttlefish algorithm implementation.
# NOTE: do something more intelligent below than the -j4
message("Build system will fetch and install KMC3")
ExternalProject_Add(prj_kmc
    DOWNLOAD_DIR        ${CMAKE_SOURCE_DIR}/external
    DOWNLOAD_COMMAND    curl -k -L  /KMC/archive/refs/tags/v3.2.1.tar.gz -o KMC-3.2.1.tar.gz &&
                        tar -xzf KMC-3.2.1.tar.gz &&
                        rm KMC-3.2.1.tar.gz
    PATCH_COMMAND       patch --strip 1 < ${CMAKE_SOURCE_DIR}/patches/kmc_patch.diff
    SOURCE_DIR          ${CMAKE_SOURCE_DIR}/external/KMC-3.2.1
    BUILD_IN_SOURCE     TRUE
    INSTALL_DIR         ${CMAKE_SOURCE_DIR}/external/
    CONFIGURE_COMMAND   ""
    BUILD_COMMAND       make -j4 CC=${CMAKE_CXX_COMPILER} simde && make -j4 CC=${CMAKE_CXX_COMPILER} kmc
    INSTALL_COMMAND     cp bin/libkmc_core.a ${EXT_LIB} &&
                        cp include/kmc_runner.h ${EXT_INCLUDE}
)

[rob-p]

Can you say what version of GCC or clang you are using? Sometimes compilers update and break in an incompatible way (the newer compilers are stricter). We generally test on GCC 11, so there may be compilation problems on GCC 13 we aren't catching.

[Sawwave]

gcc --version
gcc (GCC) 14.2.1 20240910
Copyright (C) 2024 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[Sawwave]

hi @rob-p , any idea if this issue will be addressed?

[rob-p]

Can you please see if it's addressed on the develop branch, @Sawwave? If so, we can merge into master, but we don't have a copy of a new enough GCC on our development machine to test locally.

[Sawwave]

Unfortunately, it seems to still have build errors. I've included the first few pages of the errors when make is run

patching file kmc_core/queues.h
patching file Makefile
[  8%] No configure step for 'prj_kmc'
[ 10%] Performing build step for 'prj_kmc'
In file included from kmc_core/kff_writer.cpp:1:
kmc_core/kff_writer.h:7:6: error: variable or field ‘StoreBigEndian’ declared void
7 | void StoreBigEndian(uint8_t* buff, const T& data)
|      ^~~~~~~~~~~~~~
kmc_core/kff_writer.h:7:21: error: ‘uint8_t’ was not declared in this scope
7 | void StoreBigEndian(uint8_t* buff, const T& data)
|                     ^~~~~~~
kmc_core/kff_writer.h:5:1: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
4 | #include <vector>
+++ |+#include <cstdint>
5 |
kmc_core/kff_writer.h:7:30: error: ‘buff’ was not declared in this scope
7 | void StoreBigEndian(uint8_t* buff, const T& data)
|                              ^~~~
kmc_core/kff_writer.h:7:36: error: expected primary-expression before ‘const’
7 | void StoreBigEndian(uint8_t* buff, const T& data)
|                                    ^~~~~
kmc_core/kff_writer.h:18:26: error: ‘uint8_t’ does not name a type
18 |         static constexpr uint8_t VER_MAJOR = 1;
|                          ^~~~~~~
kmc_core/kff_writer.h:18:26: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:19:26: error: ‘uint8_t’ does not name a type
19 |         static constexpr uint8_t VER_MINOR = 0;
|                          ^~~~~~~
kmc_core/kff_writer.h:19:26: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:22:9: error: ‘uint64_t’ does not name a type
22 |         uint64_t k;
|         ^~~~~~~~
kmc_core/kff_writer.h:22:9: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:24:9: error: ‘uint64_t’ does not name a type
24 |         uint64_t counter_size;
|         ^~~~~~~~
kmc_core/kff_writer.h:24:9: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:25:9: error: ‘uint64_t’ does not name a type
25 |         uint64_t min_count;
|         ^~~~~~~~
kmc_core/kff_writer.h:25:9: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:26:9: error: ‘uint64_t’ does not name a type
26 |         uint64_t max_count;
|         ^~~~~~~~
kmc_core/kff_writer.h:26:9: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:29:9: error: ‘uint64_t’ does not name a type
29 |         uint64_t cur_file_size = 0;
|         ^~~~~~~~
kmc_core/kff_writer.h:29:9: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:31:72: error: ‘uint8_t’ was not declared in this scope
31 |         void storeIndexPair(const char* type, int64_t val, std::vector<uint8_t>& tmp);
|                                                                        ^~~~~~~
kmc_core/kff_writer.h:31:72: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:31:79: error: template argument 1 is invalid
31 |         void storeIndexPair(const char* type, int64_t val, std::vector<uint8_t>& tmp);
|                                                                               ^
kmc_core/kff_writer.h:31:79: error: template argument 2 is invalid
kmc_core/kff_writer.h:37:17: error: ‘uint64_t’ does not name a type
37 |                 uint64_t where_to_store_nb_recs;
|                 ^~~~~~~~
kmc_core/kff_writer.h:37:17: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:38:17: error: ‘uint64_t’ does not name a type
38 |                 uint64_t nb_recs;
|                 ^~~~~~~~
kmc_core/kff_writer.h:38:17: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:43:54: error: ‘uint8_t’ has not been declared
43 |         explicit CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding = 0b00011011);
|                                                      ^~~~~~~
kmc_core/kff_writer.h:43:73: error: ‘uint64_t’ has not been declared
43 |         explicit CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding = 0b00011011);
|                                                                         ^~~~~~~~
kmc_core/kff_writer.h:43:85: error: ‘uint64_t’ has not been declared
43 |         explicit CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding = 0b00011011);
|                                                                                     ^~~~~~~~
kmc_core/kff_writer.h:43:108: error: ‘uint64_t’ has not been declared
43 | CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding = 0b00011011);
|                                                                                           ^~~~~~~~

kmc_core/kff_writer.h:43:128: error: ‘uint64_t’ has not been declared
43 | ::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding = 0b00011011);
|                                                                                           ^~~~~~~~

kmc_core/kff_writer.h:43:148: error: ‘uint8_t’ has not been declared
43 | 8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding = 0b00011011);
|                                                                                           ^~~~~~~

kmc_core/kff_writer.h:45:32: error: ‘uint8_t’ has not been declared
45 |         void StoreWholeSection(uint8_t* data, uint64_t n_kmers);
|                                ^~~~~~~
kmc_core/kff_writer.h:45:47: error: ‘uint64_t’ has not been declared
45 |         void StoreWholeSection(uint8_t* data, uint64_t n_kmers);
|                                               ^~~~~~~~
kmc_core/kff_writer.h:48:31: error: ‘uint8_t’ has not been declared
48 |         void StoreSectionPart(uint8_t* data, uint64_t n_recs);
|                               ^~~~~~~
kmc_core/kff_writer.h:48:46: error: ‘uint64_t’ has not been declared
48 |         void StoreSectionPart(uint8_t* data, uint64_t n_recs);
|                                              ^~~~~~~~
kmc_core/kff_writer.cpp:5:11: error: ‘uint8_t’ does not name a type
5 | constexpr uint8_t CKFFWriter::VER_MAJOR;
|           ^~~~~~~
kmc_core/kff_writer.cpp:4:1: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
3 | #include <cstring>
+++ |+#include <cstdint>
4 |
kmc_core/kff_writer.cpp:6:11: error: ‘uint8_t’ does not name a type
6 | constexpr uint8_t CKFFWriter::VER_MINOR;
|           ^~~~~~~
kmc_core/kff_writer.cpp:6:11: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.cpp:7:49: error: ‘uint8_t’ has not been declared
7 | CKFFWriter::CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding) :
|                                                 ^~~~~~~
kmc_core/kff_writer.cpp:7:68: error: ‘uint64_t’ has not been declared
7 | CKFFWriter::CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding) :
|                                                                    ^~~~~~~~
kmc_core/kff_writer.cpp:7:80: error: ‘uint64_t’ has not been declared
7 | CKFFWriter::CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding) :
|                                                                                ^~~~~~~~
kmc_core/kff_writer.cpp:7:103: error: ‘uint64_t’ has not been declared
7 | CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding) :
|                                                                                           ^~~~~~~~

kmc_core/kff_writer.cpp:7:123: error: ‘uint64_t’ has not been declared
7 | ::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding) :
|                                                                                           ^~~~~~~~

kmc_core/kff_writer.cpp:7:143: error: ‘uint8_t’ has not been declared
7 | 8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding) :
|                                                                                           ^~~~~~~

kmc_core/kff_writer.cpp: In constructor ‘CKFFWriter::CKFFWriter(const std::string&, int, int, int, int, int, int)’:
kmc_core/kff_writer.cpp:8:9: error: class ‘CKFFWriter’ does not have any field named ‘k’
8 |         k(k), counter_size(counter_size),
|         ^
kmc_core/kff_writer.cpp:8:15: error: class ‘CKFFWriter’ does not have any field named ‘counter_size’
8 |         k(k), counter_size(counter_size),
|               ^~~~~~~~~~~~
kmc_core/kff_writer.cpp:9:9: error: class ‘CKFFWriter’ does not have any field named ‘min_count’
9 |         min_count(min_count),
|         ^~~~~~~~~
kmc_core/kff_writer.cpp:10:9: error: class ‘CKFFWriter’ does not have any field named ‘max_count’
10 |         max_count(max_count)
|         ^~~~~~~~~
kmc_core/kff_writer.cpp:16:9: error: ‘cur_file_size’ was not declared in this scope
16 |         cur_file_size += 3;
|         ^~~~~~~~~~~~~
kmc_core/kff_writer.cpp:18:17: error: ‘VER_MAJOR’ was not declared in this scope
18 |         fwrite(&VER_MAJOR, 1, 1, file);
|                 ^~~~~~~~~
kmc_core/kff_writer.cpp:21:17: error: ‘VER_MINOR’ was not declared in this scope
21 |         fwrite(&VER_MINOR, 1, 1, file);
|                 ^~~~~~~~~
kmc_core/kff_writer.cpp:27:9: error: ‘uint8_t’ was not declared in this scope
27 |         uint8_t unique_kmers = 1;
|         ^~~~~~~
kmc_core/kff_writer.cpp:27:9: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.cpp:28:17: error: ‘unique_kmers’ was not declared in this scope
28 |         fwrite(&unique_kmers, 1, 1, file);
|                 ^~~~~~~~~~~~
kmc_core/kff_writer.cpp:34:9: error: ‘uint32_t’ was not declared in this scope
34 |         uint32_t free_size = 0;
|         ^~~~~~~~
kmc_core/kff_writer.cpp:34:9: note: ‘uint32_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.cpp:35:28: error: template argument 2 is invalid
35 |         std::vector<uint8_t> tmp(sizeof(uint64_t));
|                            ^
kmc_core/kff_writer.cpp:35:41: error: ‘uint64_t’ was not declared in this scope
35 |         std::vector<uint8_t> tmp(sizeof(uint64_t));
|                                         ^~~~~~~~
kmc_core/kff_writer.cpp:35:41: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.cpp:36:28: error: request for member ‘data’ in ‘tmp’, which is of non-class type ‘int’
36 |         StoreBigEndian(tmp.data(), free_size);
|                            ^~~~

We can use the conda package, so this isn't a blocker on our end, but it's likely something you will want to address at some point. Thank you for looking into this.

[rob-p]

Thanks for the update. The KMC reliance on undeclared types under the newer compiler seems to be killing us. I’ll try and set up a newer GCC somewhere to test this and see what can be easily patched.

[rob-p]

Hi @Sawwave,

Can you try on develop once more? I made a docker image with ubuntu 24.04 and gcc-14 and fixed the patch so that KMC (and cuttlefish) now seem to properly compile.

Thanks!
Rob

[Sawwave]

Still get a few errors, but it's a lot less this time! Here's all the error I received.

simde-amalgamated-0.7.2/x86/xop.h
simde-amalgamated-0.7.2/arm/
simde-amalgamated-0.7.2/arm/neon.h
In file included from kmc_core/mem_disk_file.cpp:12:
kmc_core/critical_error_handler.h: In member function ‘void CCriticalErrorHandler::HandleCriticalError(const std::string&)’:
kmc_core/critical_error_handler.h:77:28: error: ‘runtime_error’ is not a member of ‘std’
77 |                 throw std::runtime_error(msg);
|                            ^~~~~~~~~~~~~
kmc_core/critical_error_handler.h:7:1: note: ‘std::runtime_error’ is defined in header ‘<stdexcept>’; this is probably fixable by adding ‘#include <stdexcept>’
6 | #include <condition_variable>
+++ |+#include <stdexcept>
7 |
In file included from kmc_core/queues.h:25,
from kmc_core/params.h:16,
from kmc_core/bkb_writer.h:14,
from kmc_core/bkb_writer.cpp:11:
kmc_core/critical_error_handler.h: In member function ‘void CCriticalErrorHandler::HandleCriticalError(const std::string&)’:
kmc_core/critical_error_handler.h:77:28: error: ‘runtime_error’ is not a member of ‘std’
77 |                 throw std::runtime_error(msg);
|                            ^~~~~~~~~~~~~
kmc_core/critical_error_handler.h:7:1: note: ‘std::runtime_error’ is defined in header ‘<stdexcept>’; this is probably fixable by adding ‘#include <stdexcept>’
6 | #include <condition_variable>
+++ |+#include <stdexcept>
7 |
make[3]: *** [Makefile:118: kmc_core/mem_disk_file.o] Error 1
make[3]: *** Waiting for unfinished jobs....
In file included from kmc_core/queues.h:25,
from kmc_core/params.h:16,
from kmc_core/bkb_reader.h:15,
from kmc_core/bkb_reader.cpp:11:
kmc_core/critical_error_handler.h: In member function ‘void CCriticalErrorHandler::HandleCriticalError(const std::string&)’:
kmc_core/critical_error_handler.h:77:28: error: ‘runtime_error’ is not a member of ‘std’
77 |                 throw std::runtime_error(msg);
|                            ^~~~~~~~~~~~~
kmc_core/critical_error_handler.h:7:1: note: ‘std::runtime_error’ is defined in header ‘<stdexcept>’; this is probably fixable by adding ‘#include <stdexcept>’
6 | #include <condition_variable>
+++ |+#include <stdexcept>
7 |
make[3]: *** [Makefile:118: kmc_core/bkb_writer.o] Error 1
make[3]: *** [Makefile:118: kmc_core/bkb_reader.o] Error 1
make[2]: *** [CMakeFiles/prj_kmc.dir/build.make:86: prj_kmc-prefix/src/prj_kmc-stamp/prj_kmc-build] Error 2
make[1]: *** [CMakeFiles/Makefile2:104: CMakeFiles/prj_kmc.dir/all] Error 2
make: *** [Makefile:136: all] Error 2