v0.10.0

A release that includes multi-dataset query support, log-ingestor fault tolerance, external third-party service configuration, and several bug fixes and improvements.

Note

This release includes some changes that are incompatible with previous releases. If this affects you, reach out and we may be able to help with version migration. These changes are marked with "Breaking".

This release includes two tars:

• clp-json for compressing and searching JSON logs
• clp-text for compressing and searching unstructured text logs

Docs

The docs for this release are available here.

New features

• clp-json: Add support for querying multiple datasets in a single query. (#1992, #2060, #2062)
  • Breaking:
    • The --dataset CLI flag can now be specified multiple times to search multiple datasets.
    • In API server requests the dataset field has been renamed to datasets and changed from a string to an array of strings.
• log-ingestor: Add fault tolerance support. See the docs for the tolerance guarantees. (#1988, #2001, #2014, #2017, #2035, #2038, #2045, #2053, #2057)
  • Breaking:
    • In the API, job IDs are now numeric (database-generated) instead of UUIDs.
    • In the API, DELETE /jobs/{id} has been replaced with POST /jobs/{id}/terminate.
• clp-json: Replace the DateString column type with the new Timestamp column type that supports nanosecond precision and is more compressible. (#1385, #1564, #1599, #1626, #1646, #1673, #1708, #1723, #1757, #1788, #1847, #1873, #2013, #2028)
  • Breaking:
    • Archives created with this version (v0.10.0) will not be readable by older CLP versions.
    • Datasets containing non-ISO 8601 timezone formats (e.g., the PostgreSQL sample dataset) are temporarily unsupported.
• helm/docker-compose: Support configuration of external third-party services (database, queue, Redis, results cache) via the bundled list in clp-config.yaml. (#1648, #1681, #1694, #2056, #2066)
• api-server: Add DELETE /query/{search_job_id} endpoint to cancel in-progress query jobs. (#1964)
• presto-clp: Add support for custom S3 endpoint URLs and buckets, enabling use with S3-compatible storage providers (e.g., MinIO). (#1917)

Bug fixes & improvements

• helm: Remove all host-path volume mounts; use dynamically provisioned PVCs and stdout/stderr logging. (#2023, #2075, #2077)
  • Breaking: The storage section and data_directory/logs_directory/tmp_directory values in values.yaml have been removed. Existing custom Helm values referencing these fields must be updated.
• log-ingestor: Increase buffer_flush_threshold default from 256 MiB to 4 GiB. (#1965)
  • Breaking: Since the default value has changed, any deployments relying on the default (rather than an explicit value) will experience different buffering behavior.
• log-ingestor: Allow SQS listener jobs to spawn multiple concurrent tasks for higher throughput, with configurable concurrency and max wait time. (#1989)
• webui: Cap compression metadata query results to 1000 to prevent OOM crash (fixes #2040). (#2041)
• clp-json: Fix use-after-free in the SchemaMatch pass that could cause crashes during queries (fixes #1986). (#1990)
• helm: Remove Job TTL to prevent init container CrashLoopBackOff on pod restarts (fixes #2043). (#2044)
• clp-text: Generate all possible wildcard subqueries for non-capture schema-based search, preventing unnecessary archive decompression and improving search accuracy. (#1313, #1959, #1972)
• clp-json/clp-text: Remove unused staged-streams volume mount from the api-server service (fixes #1810). (#1947)
• clp-json/clp-text: Update results cache health check command to use mongosh --eval (resolves #1742). (#1832)
• build: Add --pull flag to all Docker build commands to ensure the latest base images are used (fixes #1051). (#1943)
• webui: Fix scroll.x console warning in VirtualTable (fixes #1892). (#1949)
• helm: Add --clp-package-image CLI arg to setup scripts for local image testing. (#2020)

View the full changelog for more details.

Thanks to @20001020ycx, @Bill-hbrhbr, @davidlion, @gibber9809, @hoophalab, @jonathan-imanu, @junhaoliao, @LinZhihao-723, @Nathan903, @quinntaylormitchell, and @SharafMohamed for their contributions.