feat(silo): read in phylogenetic tree files and create tree structure

[anna-parker]

resolves #807

Summary

1. Adds new files src/silo/preprocessing/phylo_tree_file this contains:

• functions to read a phylogenetic tree from a simple newick or auspice.json file (also handles edge cases if the file format is incorrect), adds tests for correct and incorrect case
• a PhyloTreeFile class, this class contains a map from treeNodeIds to TreeNode pointers std::unordered_map<TreeNodeId, std::shared_ptr<TreeNode>> nodes,
• TreeNode class with

class TreeNode {
  public:
   TreeNodeId node_id;
   std::vector<std::shared_ptr<TreeNode>> children;
   std::optional<std::shared_ptr<TreeNode>> parent;
   int depth;
};

2. Adds src/silo/common/tree_node_id.h with the simple TreeNodeId class to define a treeNodeId "type" (just a string of the node name)
3. Allow the phylogenetic tree file path to be passed in the preprocessing_config.yaml and allow the metadata column corresponding to the nodeId to be annotated with generatePhyloTreeIndex in the database_config.yaml - this column must be a type STRING but must not have a bitmap index generated (searches will be performed using the tree structure instead), also adds tests for this
4. Update the initializeDatabase code to generate a phyloTree from the file specified in the config and then create a schema using the phyloTree. For metadata columns of type StringColumnPartition allow them to be linked to a phyloTree if generatePhyloTreeIndex is set to true.

PR Checklist

• All necessary documentation has been adapted or there is an issue to do so.
• The implemented feature is covered by an appropriate test.

[github-actions]

This is a preview of the changelog of the next release. If this branch is not up-to-date with the current main branch, the changelog may not be accurate. Rebase your branch on the main branch to get the most accurate changelog.

Note that this might contain changes that are on main, but not yet released.

Changelog:

0.7.0 (2025-06-20)

⚠ BREAKING CHANGES

• the field sequenceName in the Fasta and FastaAligned is now named sequenceNames and required to be an array
• streaming implementation of Details action. select-k also for Fasta and FastaAligned
• streaming implementation of the Fasta action with the arrow framework

Features

• arrow framework for query engine and specialized implementation for FastaAligned action (b719e15), closes #654
• lapis: add newick parser and tests (19aa118)
• silo: add better error type (4ca730f)
• silo: add docs and improve variable names (4a060fd)
• silo: add link between metadata field and tree in initializer (6a9ddf6)
• silo: add newick file to e2e tests (5ba8a32)
• silo: add recommended vscode extensions to .vscode folder for developers using vscode (d2c74db)
• silo: add tests for edge cases and error handling (cd898b6)
• silo: add to e2e tests (ad4bf5f)
• silo: improve function structure (6b2e010)
• silo: improve the docs and update build_with_conan.py (#779) (f176fd6)
• silo: move phylo_tree class to common (816e5a3)
• silo: parse auspice.json (92216fc)
• silo: refactor reading from file code (467be3a)
• silo: switch to using pointers (893c634)
• silo: try to create TreeNodeId type (00eeb7d)
• silo: update names (1680e6d)
• streaming implementation of the Fasta action with the arrow framework (0392207)
• update onetbb, nlohmann_json, gtest, roaring, yaml-cpp and zstd to most recent versions (3abd2d9)

Bug Fixes

• bug in table_scan, that confused row_id in the database with row_id in the current batch (9b7dce4)
• correctly error when the input data contains illegal insertion characters (4867c32)
• make better use of the arrow::acero API, no longer rely on passing the stream inside an extra operator (dabb186)
• properly catch exceptions during arrow execution to avoid program termination (6caee0e)
• remove erroneous warning printed on every preprocessing run (82b65c8)

Code Refactoring

• make sequenceName in Fasta and FastaAligned actions an array (6c345fb)
• streaming implementation of Details action. select-k also for Fasta and FastaAligned (bfbb3ee)

[taepper]

Only some comments, otherwise looks good to go!

[taepper]

Some error handling could still be stream-lined (the invalid_argument exception is never caught and could therefore also be the user facing initialize_exception already), but there were many other such cases in the code base. I think this will be cleaned up at a later point, where error reporting, specifically during preprocessing should be improved, as per user reports.

Therefore I would say this is good to go!