Skip to content

TIKA-4846 -- add jsonl reporter - #3082

Merged
tballison merged 5 commits into
mainfrom
TIKA-4846
Aug 27, 2026
Merged

TIKA-4846 -- add jsonl reporter#3082
tballison merged 5 commits into
mainfrom
TIKA-4846

Conversation

@tballison

Copy link
Copy Markdown
Contributor

Thanks for your contribution to Apache Tika! Your help is appreciated!

Before opening the pull request, please verify that

  • there is an open issue on the Tika issue tracker which describes the problem or the improvement. We cannot accept pull requests without an issue because the change wouldn't be listed in the release notes.
  • the issue ID (TIKA-XXXX)
    • is referenced in the title of the pull request
    • and placed in front of your commit messages surrounded by square brackets ([TIKA-XXXX] Issue or pull request title)
  • commits are squashed into a single one (or few commits for larger changes)
  • Tika is successfully built and unit tests pass by running ./mvnw clean test
  • there should be no conflicts when merging the pull request branch into the recent main branch. If there are conflicts, please try to rebase the pull request branch on top of a freshly pulled main branch
  • if you add new module that downstream users will depend upon add it to relevant group in tika-bom/pom.xml.

We will be able to faster integrate your pull request if these conditions are met. If you have any questions how to fix your problem or about using Tika in general, please sign up for the Tika mailing list. Thanks!

@tballison
tballison marked this pull request as draft August 27, 2026 12:41
@tballison
tballison marked this pull request as ready for review August 27, 2026 17:03
@THausherr
THausherr requested a lite review from Copilot August 27, 2026 17:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new JSONL (one-JSON-object-per-line) pipes reporter for the file-system pipes plugin to provide an append-only per-document audit log, and updates reporter/async-processing behavior and documentation to support it.

Changes:

  • Introduces file-system-jsonl-reporter (config record, reporter implementation, PF4J factory) plus extensive unit tests and config examples.
  • Updates CompositePipesReporter to invoke all configured reporters even if one throws, then rethrow the first exception with the rest suppressed.
  • Updates async CLI/config writing and docs to apply --on-exists mapping to the JSONL reporter (skipAPPEND) and documents the new reporter.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/test/resources/config-examples/file-system-jsonl-reporter.json Adds a file-system JSONL reporter config example for validation tests.
tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/test/java/org/apache/tika/pipes/reporter/fs/FileSystemJsonlReporterTest.java Adds unit tests for JSONL line format, filtering, truncation, concurrency, and failure modes.
tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/test/java/org/apache/tika/pipes/fs/ConfigExamplesTest.java Validates the new JSONL reporter config example and factory registration.
tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/main/java/org/apache/tika/pipes/reporter/fs/FileSystemJsonlReporterFactory.java Registers the new reporter via PF4J and exposes its component name.
tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/main/java/org/apache/tika/pipes/reporter/fs/FileSystemJsonlReporterConfig.java Adds strongly-typed config (path, includes/excludes, onExists, maxMessageLength).
tika-pipes/tika-pipes-plugins/tika-pipes-file-system/src/main/java/org/apache/tika/pipes/reporter/fs/FileSystemJsonlReporter.java Implements the JSONL reporter with per-line flush, on-exists behavior, and message truncation.
tika-pipes/tika-pipes-plugins/tika-pipes-file-system/pom.xml Adds dependency on tika-pipes-reporter-commons for PipesReporterBase.
tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/reporter/CompositePipesReporterTest.java Adds coverage to ensure throwing reporters don’t starve sibling reporters.
tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/reporter/CompositePipesReporter.java Ensures all reporters see calls and aggregates exceptions via suppression.
tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/async/AsyncProcessor.java Latches first worker/emitter failure; adjusts error/report behavior around reporter failures.
tika-pipes/tika-async-cli/src/test/java/org/apache/tika/async/cli/TikaConfigAsyncWriterTest.java Tests --on-exists mapping through to the JSONL reporter configuration.
tika-pipes/tika-async-cli/src/main/java/org/apache/tika/async/cli/PluginsWriter.java Applies onExists mapping to file-system-jsonl-reporter (SKIPAPPEND).
docs/modules/ROOT/pages/using-tika/cli/index.adoc Documents that --on-exists also applies to the JSONL reporter.
docs/modules/ROOT/pages/pipes/reporters.adoc Documents composite reporter behavior (all reporters called; first exception rethrown).
docs/modules/ROOT/pages/pipes/plugins/filesystem.adoc Adds full documentation for the new JSONL reporter (format, guarantees, configuration).
CHANGES.txt Adds a release-note entry for the new reporter (TIKA-4846).

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@tballison
tballison merged commit ac31d0c into main Aug 27, 2026
4 checks passed
@tballison
tballison deleted the TIKA-4846 branch August 27, 2026 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants