Skip to content

build: Migrate to task-based dependency management: #115

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 47 commits into from
May 20, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
d73c717
Add new taskfiles.
davidlion Apr 28, 2025
956b643
Merge branch 'main' into task-build
davidlion Apr 28, 2025
d99f2e1
Merge branch 'main' into task-build
davidlion May 12, 2025
1508d47
Add set+shopt header to lint.yaml.
davidlion May 12, 2025
2a5b10a
Add .task to .gitignore.
davidlion May 12, 2025
c986ac8
Bump dev-utils.
davidlion May 12, 2025
bab25bb
Finish swap to task.
davidlion May 12, 2025
28150e3
Remove catch2 script.
davidlion May 12, 2025
96f2a3e
Add install task.
davidlion May 12, 2025
b655acf
Add tasks for installation and examples.
davidlion May 12, 2025
0452ae7
Recursively pull for dev-utils submodule.
davidlion May 12, 2025
f96f383
Drop condition on find_package for GSL.
davidlion May 12, 2025
b61c3a6
Try bumping catch2 to 3.8.1.
davidlion May 12, 2025
8449d48
Drop ubuntu 24.04.
davidlion May 12, 2025
5bd0e1e
Try catch2 v3.6.0.
davidlion May 12, 2025
c24a462
Update workflow paths.
davidlion May 12, 2025
d3f8ae4
Update examples cmake minimum required version.
davidlion May 12, 2025
eb2dc5c
Back to catch2 3.8.1.
davidlion May 12, 2025
5f9cc7c
Remove dead variable.
davidlion May 12, 2025
4cea8cc
Remove needless check.
davidlion May 12, 2025
86713b2
Fix ls install prefix for examples.
davidlion May 12, 2025
6cfb73c
Fix macos to 15 rather than latest.
davidlion May 12, 2025
681010b
Respect build_type in github workflow.
davidlion May 12, 2025
975ad13
Add ubuntu 24.04.
davidlion May 12, 2025
cdd0a84
Revert "Add ubuntu 24.04."
davidlion May 12, 2025
4dc291a
Refactor tests CMakeLists.txt.
davidlion May 12, 2025
df4ae21
test examples on macos
davidlion May 13, 2025
c2c83af
Revert "test examples on macos"
davidlion May 13, 2025
773c2c6
Explicitly require catch2 3.8.1.
davidlion May 13, 2025
82af8a8
Build catch2 without testing and set c++20.
davidlion May 13, 2025
f5e9a6e
Update catch version in readme.
davidlion May 13, 2025
c6f4490
Remove ctest inclusion from examples/CMakeLists.txt.
davidlion May 13, 2025
3180b2b
Address coderabbit review.
davidlion May 14, 2025
e062f0f
Mirror deps naming in clp.
davidlion May 14, 2025
287dbc3
Apply suggestions from code review.
davidlion May 15, 2025
b23c265
Address review comments.
davidlion May 15, 2025
1a216ca
Add back deps: for utils:cmake:install-deps-and-generate-settings.
davidlion May 15, 2025
8774438
Obey IWYU.
davidlion May 15, 2025
81b8d21
Fix include paths.
davidlion May 15, 2025
ce355d3
Address review comments.
davidlion May 16, 2025
8fcadb0
Forgot to flatten catch2 build.
davidlion May 16, 2025
a218aa0
Use build_type in examples path.
davidlion May 16, 2025
adb4729
Reorder internal tasks to meet guideline.
davidlion May 16, 2025
fa985bf
Run all examples in gh workflow to ensure they don't crash.
davidlion May 16, 2025
f70d13b
Apply suggestions from code review.
davidlion May 16, 2025
a51a02e
Bump to latest dev-utils.
davidlion May 20, 2025
fc77661
Update .github/workflows/build.yaml.
davidlion May 20, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 26 additions & 23 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: "Build"
name: "build"

on:
pull_request:
Expand All @@ -7,16 +7,18 @@ on:
- "CMakeLists.txt"
- "examples/**/*"
- "src/**/*"
- "taskfile.yaml"
- "taskfiles/**/*"
- "tests/**/*"
- "tools/deps-install/ubuntu/**/*"
push:
paths:
- ".github/workflows/build.yaml"
- "CMakeLists.txt"
- "examples/**/*"
- "src/**/*"
- "taskfile.yaml"
- "taskfiles/**/*"
- "tests/**/*"
- "tools/deps-install/ubuntu/**/*"
workflow_call:

concurrency:
Expand All @@ -28,31 +30,32 @@ jobs:
build:
strategy:
matrix:
os: ["macos-latest", "ubuntu-22.04"]
build_type: ["Debug", "Release"]
os:
- "macos-15"
- "ubuntu-22.04"
build_type:
- "debug"
- "release"
runs-on: "${{matrix.os}}"
Comment on lines +33 to +38
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Invalid GitHub runner name: "macos-15"
As of now, GitHub Actions supports macos-13 (Ventura) or the macos-latest alias. There is no macos-15 runner, so macOS jobs will fail.

Apply this diff to correct the runner:

-        os:
-          - "macos-15"
+        os:
+          - "macos-13"
           - "ubuntu-22.04"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
os:
- "macos-15"
- "ubuntu-22.04"
build_type:
- "debug"
- "release"
os:
- "macos-13"
- "ubuntu-22.04"
build_type:
- "debug"
- "release"
🤖 Prompt for AI Agents
In .github/workflows/build.yaml at lines 33 to 38, the runner name "macos-15" is
invalid and will cause the macOS jobs to fail. Replace "macos-15" with a valid
runner name such as "macos-13" or "macos-latest" to ensure the workflow runs
correctly on macOS.

steps:
- uses: "actions/checkout@v4"
with:
submodules: "recursive"

- name: "Install Catch2 on macOS"
if: "matrix.os == 'macos-latest'"
run: "brew install catch2"
- name: "Install task"
shell: "bash"
run: "npm install -g @go-task/cli"

- name: "Install Catch2 on Ubuntu"
if: "matrix.os == 'ubuntu-22.04'"
run: "./tools/deps-install/ubuntu/install-catch2.sh 3.6.0"

- name: "Build Executables"
run: |-
cmake -B ./build -DBUILD_TESTING=ON -DCMAKE_BUILD_TYPE=${{matrix.build_type}}
cmake --build ./build --config ${{matrix.build_type}}
cmake --install ./build --prefix ./install
cmake -S examples -B ./examples/build -DCMAKE_BUILD_TYPE=${{matrix.build_type}}
cmake --build ./examples/build --config ${{matrix.build_type}}

- name: "Run Unit Tests"
run: "ctest --test-dir ./build"
- name: "Build and run unit tests"
run: "task test:${{matrix.build_type}}"

- name: "Print test log on failure"
if: "failure()"
run: "cat ./build/Testing/Temporary/LastTest.log"
run: "cat ./build/log-surgeon/Testing/Temporary/LastTest.log"

- name: "Build and run examples"
run: |-
task build:examples-${{matrix.build_type}}
./build/examples/${{matrix.build_type}}/buffer-parser ./examples/schema.txt README.md
./build/examples/${{matrix.build_type}}/reader-parser ./examples/schema.txt README.md
./build/examples/${{matrix.build_type}}/intersect-test
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
.task/
build/
82 changes: 17 additions & 65 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@ endif()

include(CMakePackageConfigHelpers)
include(GNUInstallDirs)
include(FetchContent)

option(BUILD_SHARED_LIBS "Build using shared libraries" OFF)
option(log_surgeon_BUILD_TESTING "Build the testing tree." ON)

set(CMAKE_EXPORT_COMPILE_COMMANDS
ON
CACHE BOOL
Expand All @@ -31,6 +31,11 @@ if (NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)
endif()

if(log_surgeon_IS_TOP_LEVEL)
# Include dependency settings if the project isn't being included as a subproject.
# NOTE: We mark the file optional because if the user happens to have the dependencies
# installed, this file is not necessary.
include("${CMAKE_SOURCE_DIR}/build/deps/cmake-settings/all.cmake" OPTIONAL)

# If previously undefined, `BUILD_TESTING` will be set to ON.
include(CTest)
endif()
Expand All @@ -39,46 +44,19 @@ if(BUILD_TESTING AND log_surgeon_BUILD_TESTING)
set(log_surgeon_ENABLE_TESTS ON)
endif()

list(APPEND DEPS_TO_FETCH "")

# Use <fmt/*.h> as <format> is unsupported in gcc-10.
find_package(fmt 8.0.1 QUIET)
if(fmt_FOUND)
message(STATUS "Found fmt ${fmt_VERSION}.")
else()
FetchContent_Declare(fmt
GIT_REPOSITORY https://github.com/fmtlib/fmt
GIT_TAG "8.0.1"
GIT_SHALLOW ON
)
# Force fmt to generate install rules
set(FMT_INSTALL ON CACHE BOOL "Enable installation for fmt." FORCE)
list(APPEND DEPS_TO_FETCH fmt)
endif()
find_package(fmt 8.0.1 REQUIRED)
message(STATUS "Found fmt ${fmt_VERSION}.")

find_package(Microsoft.GSL QUIET)
if(Microsoft.GSL_FOUND)
message(STATUS "Found Microsoft.GSL ${Microsoft.GSL_VERSION}.")
else()
FetchContent_Declare(GSL
GIT_REPOSITORY "https://github.com/microsoft/GSL"
GIT_TAG "v4.0.0"
GIT_SHALLOW ON
)
list(APPEND DEPS_TO_FETCH GSL)
endif()
find_package(Microsoft.GSL 4.0.0 REQUIRED)
message(STATUS "Found Microsoft.GSL ${Microsoft.GSL_VERSION}.")

if(log_surgeon_ENABLE_TESTS)
find_package(Catch2 3 REQUIRED)
if(Catch2_FOUND)
message(STATUS "Found Catch2 ${Catch2_VERSION}.")
endif()
find_package(Catch2 3.8.1 REQUIRED)
message(STATUS "Found Catch2 ${Catch2_VERSION}.")
include(Catch)
endif()

# Declare the details of all fetched content before making them available.
FetchContent_MakeAvailable("${DEPS_TO_FETCH}")

set(SOURCE_FILES
src/log_surgeon/Buffer.hpp
src/log_surgeon/BufferParser.cpp
Expand Down Expand Up @@ -138,31 +116,15 @@ set(SOURCE_FILES

set(LOG_SURGEON_INSTALL_CONFIG_DIR ${CMAKE_INSTALL_LIBDIR}/cmake/log_surgeon)
set(LOG_SURGEON_INSTALL_INCLUDE_DIR ${CMAKE_INSTALL_INCLUDEDIR})
# Directory for installing third-party includes that the user doesn't have installed.
set(LOG_SURGEON_THIRD_PARTY_INCLUDE_DIR "${LOG_SURGEON_INSTALL_INCLUDE_DIR}/log_surgeon/third_party_include")

add_library(log_surgeon ${SOURCE_FILES})
add_library(log_surgeon::log_surgeon ALIAS log_surgeon)

if (Microsoft.GSL_FOUND)
target_link_libraries(log_surgeon
PUBLIC
Microsoft.GSL::GSL
)
else()
# Since the user doesn't have GSL installed, use the GSL headers directly.
# NOTE:
# - We can't link against the `Microsoft.GSL::GSL` target since that would require adding `GSL`
# to the `install` command and force the user to have GSL installed when using log-surgeon.
# - At install time, we'll copy GSL into log-surgeon's third-party includes directory.
target_include_directories(log_surgeon
PUBLIC
$<BUILD_INTERFACE:${GSL_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:${LOG_SURGEON_THIRD_PARTY_INCLUDE_DIR}>
)
endif()

target_link_libraries(log_surgeon PUBLIC fmt::fmt)
target_link_libraries(log_surgeon
PUBLIC
fmt::fmt
Microsoft.GSL::GSL
)

target_include_directories(log_surgeon
PUBLIC
Expand Down Expand Up @@ -213,16 +175,6 @@ install(
PATTERN "*.hpp"
PATTERN "*.tpp"
)
if (NOT Microsoft.GSL_FOUND)
install(
DIRECTORY
# NOTE: We don't include a trailing slash so that the gsl directory is copied rather than
# its contents.
"${GSL_SOURCE_DIR}/include/gsl"
DESTINATION
"${LOG_SURGEON_THIRD_PARTY_INCLUDE_DIR}"
)
endif ()

configure_package_config_file(
${CMAKE_CURRENT_LIST_DIR}/cmake/log_surgeon-config.cmake.in
Expand Down
56 changes: 29 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ constraints](docs/parsing-constraints.md) on how log events can be parsed.
## Motivating example

Let's say we want to parse and inspect multi-line log events like this:

```
2023-02-23T18:10:14-0500 DEBUG task_123 crashed. Dumping stacktrace:
#0 0x000000000040110e in bar () at example.cpp:6
Expand All @@ -29,13 +30,15 @@ Let's say we want to parse and inspect multi-line log events like this:
```

Using the [example schema file](examples/schema.txt) which includes these rules:

```
timestamp:\d{4}\-\d{2}\-\d{2}T\d{2}:\d{2}:\d{2}\-\d{4}
...
loglevel:INFO|DEBUG|WARN|ERROR
```

We can parse and inspect the events as follows:

```cpp
// Define a reader to read from your data source
Reader reader{/* <Omitted> */};
Expand Down Expand Up @@ -81,35 +84,30 @@ For advanced uses, `log-surgeon` also has a

Requirements:

* CMake
* CMake >= 3.22.1
* GCC >= 10 or Clang >= 7
* [Catch2] >= 3
* On Ubuntu <= 20.04, you can install it using:
```shell
sudo tools/deps-install/ubuntu/install-catch2.sh 3.6.0
```
* On Ubuntu >= 22.04, you can install it using:
```shell
sudo apt-get update
sudo apt-get install catch2
```
* On macOS, you can install it using:
```shell
brew install catch2
```

From the repo's root, run:
* [Catch2] >= 3.8.1
* [fmt] >= 8.0.1
* [GSL] >= 4.0.0
* [Task] >= 3.38

To build and install the project to `~/.local`:

```shell
task install:release INSTALL_PREFIX="~/.local"
```

Or to only build the project:

```shell
# Generate the CMake project
cmake -S . -B build -DBUILD_TESTING=OFF
# Build the project
cmake --build ./build -j
# Install the project to ~/.local
cmake --install ./build --prefix ~/.local
task build:release
```

To build the debug version and tests replace the first command with:
`cmake -S . -B ./build -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTING=ON`
To build the debug version:

```shell
task build:debug
```

## Documentation and examples

Expand All @@ -121,9 +119,10 @@ To build the debug version and tests replace the first command with:

## Testing

To run unit tests, run:
To build and run all unit tests:

```shell
cmake --build ./build --target test
task test:debug
```

When generating targets, the CMake variable `BUILD_TESTING` is followed (unless overruled by setting
Expand Down Expand Up @@ -158,6 +157,7 @@ To run the linting tools, besides commonly installed tools like `tar`, you'll ne
### Running the linters

Currently, `clang-tidy` has to be run manually:

```shell
find src tests \
-type f \
Expand Down Expand Up @@ -200,5 +200,7 @@ The following are issues we're aware of and working on:
[Catch2]: https://github.com/catchorg/Catch2/tree/devel
[clang-tidy]: https://clang.llvm.org/extra/clang-tidy/
[feature-req]: https://github.com/y-scope/log-surgeon/issues/new?assignees=&labels=enhancement&template=feature-request.yml
[fmt]: https://github.com/fmtlib/fmt
[GSL]: https://github.com/microsoft/GSL
[lint]: https://github.com/y-scope/log-surgeon/blob/main/.github/workflows/lint.yml
[Task]: https://taskfile.dev/
15 changes: 9 additions & 6 deletions examples/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
cmake_minimum_required(VERSION 3.16)
cmake_minimum_required(VERSION 3.22.1)
project(log-surgeon-examples)

if (NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)
Expand All @@ -7,13 +7,16 @@ if (NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)
set(CMAKE_BUILD_TYPE "${default_build_type}" CACHE STRING "Choose the type of build." FORCE)
endif()

find_package(log_surgeon QUIET)
if(log_surgeon_FOUND)
message(STATUS "Found log_surgeon ${log_surgeon_VERSION}.")
else()
add_subdirectory(.. log-surgeon-build EXCLUDE_FROM_ALL)
if(log-surgeon-examples_IS_TOP_LEVEL)
# Include dependency settings if the project isn't being included as a subproject.
# NOTE: We mark the file optional because if the user happens to have the dependencies
# installed, this file is not necessary.
include("${CMAKE_SOURCE_DIR}/../build/deps/cmake-settings/all.cmake" OPTIONAL)
endif()

find_package(log_surgeon REQUIRED)
message(STATUS "Found log_surgeon ${log_surgeon_VERSION}.")

function(add_to_target target libraries)
target_link_libraries(${target} ${libraries})
target_compile_features(${target} PRIVATE cxx_std_20)
Expand Down
13 changes: 5 additions & 8 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,23 +11,20 @@ the intersection between a schema DFA and a search query DFA.
## Building

First, ensure you've built and installed the library by following
[these steps][2]. Then run the following commands (from the repo's root):
[these steps][2]. Then run the following command:

```shell
# Generate the CMake project
cmake -S examples -B examples/build
# Build the project
cmake --build examples/build -j
task build:examples-debug
```

## Running

The example programs can be run as follows:

```shell
./examples/build/buffer-parser ./examples/schema.txt log.txt
./examples/build/reader-parser ./examples/schema.txt log.txt
./examples/build/intersect-test
./build/examples/debug/buffer-parser ./examples/schema.txt log.txt
./build/examples/debug/reader-parser ./examples/schema.txt log.txt
./build/examples/debug/intersect-test
```

where:
Expand Down
9 changes: 8 additions & 1 deletion taskfile.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,18 @@
version: "3"

set: ["u", "pipefail"]
shopt: ["globstar"]

includes:
build: "taskfiles/build.yaml"
deps: "taskfiles/deps.yaml"
lint: "taskfiles/lint.yaml"
utils: "tools/yscope-dev-utils/exports/taskfiles/utils/utils.yaml"
test: "taskfiles/test.yaml"

vars:
G_BUILD_DIR: "{{.ROOT_DIR}}/build"
G_EXAMPLES_BUILD_DIR: "{{.G_BUILD_DIR}}/examples"
G_LOG_SURGEON_BUILD_DIR: "{{.G_BUILD_DIR}}/log-surgeon"

tasks:
clean: "rm -rf '{{.G_BUILD_DIR}}'"
Expand Down
Loading
Loading