TIKA-4856 presets (#3118)

Author: tballison

CHANGES.txt

@@ -1,5 +1,7 @@
 Release 4.1.0 - unreleased
 
+   * tika-server: named configuration presets (TIKA-4856).
+
    * Temp files follow -Djava.io.tmpdir on the parent JVM (Tika, its
      libraries, and forks all honor it); TikaLoader fails at config load
      if it is unusable. pipes.tempDirectory is deprecated for removal in

docs/modules/ROOT/pages/pipes/configuration.adoc

@@ -136,7 +136,7 @@ See also xref:pipes/timeouts.adoc[Timeouts] for the full timeout model.
 
 |`maxTotalTaskTimeoutMillis`
 |`3600000`
-|Ceiling for *request-supplied* timeout limits: a per-request `timeout-limits` override may lower its timeouts freely but can never raise `totalTaskTimeoutMillis` or `progressTimeoutMillis` above this value (values over the cap are clamped with a warning). Limits set in the server's own `parse-context` are trusted and not subject to this cap.
+|Ceiling for *request-supplied* timeout limits: a per-request `timeout-limits` override may lower its timeouts freely but can never raise `totalTaskTimeoutMillis` or `progressTimeoutMillis` above this value (values over the cap are clamped with a warning). Limits set in the server's own `parse-context` — or in a preset a request selects — are trusted and not subject to this cap.
 
 |`maxWaitForClientMillis`
 |`60000`

docs/modules/ROOT/pages/pipes/plugins/json.adoc

@@ -57,6 +57,6 @@ Unlike the other iterators, this one takes no `fetcherId` / `emitterId`: each li
 == Notes
 
 * The file format is JSON-lines (also called NDJSON) — one valid JSON object per line, no surrounding array brackets.
-* Each line must be a serialized `FetchEmitTuple`: `id`, `fetcher`, `fetchKey`, `emitter`, `emitKey`, and optionally `fetchRangeStart`, `fetchRangeEnd`, `metadata`, `parse-context`, `onParseException`. Any other
+* Each line must be a serialized `FetchEmitTuple`: `id`, `fetcher`, `fetchKey`, `emitter`, `emitKey`, and optionally `fetchRangeStart`, `fetchRangeEnd`, `metadata`, `parse-context`, `onParseException`, `preset`. Any other
   field is rejected — the deserializer does not tolerate unknown keys.
 * For columnar work items in a CSV, use the xref:pipes/plugins/csv.adoc[CSV iterator] instead.

docs/modules/ROOT/pages/using-tika/server/index.adoc

@@ -105,6 +105,62 @@ WARNING: Enable this only behind network controls (firewalls, private subnets),
 or xref:using-tika/server/tls.adoc[2-way TLS authentication]. Per-request configuration lets
 callers change how documents are parsed, widening what anyone who can reach the server can do.
 
+=== Presets — named configuration without `/config`
+
+A *preset* is a named, vetted parse-context fragment: parser and component configurations keyed
+by friendly name, defined once in the server config (or shipped with Tika) and selected whole by
+name. Callers apply one by inserting `preset/{name}` directly after the resource root:
+`/tika/preset/{name}[/text|/html|/xml|/md|/json[/{handlerType}]]`,
+`/rmeta/preset/{name}[/{handlerType}]`, `/unpack/preset/{name}[/all]`.
+
+[source,bash]
+----
+curl -T document.pdf http://localhost:9998/tika/preset/my-preset/text
+curl -T document.pdf http://localhost:9998/rmeta/preset/my-preset/text
+curl -T document.pdf http://localhost:9998/unpack/preset/my-preset
+----
+
+Nothing is active unless the top-level `presets` key names it. `true` activates a preset from
+the classpath *catalog* (definitions shipped with Tika, so their content tracks the Tika
+version); an object defines a preset in place, replacing any same-named catalog definition
+wholesale; `false`/`null` is an explicit no-op. Catalog jars can never activate themselves —
+every active preset is a visible line in the operator's config, and `true` naming nothing in
+the catalog fails startup:
+
+[source,json]
+----
+{
+  "presets": {
+    "some-catalog-preset": true,
+    "no-ocr": { "pdf-parser": { "ocr": { "strategy": "NO_OCR" } } }
+  }
+}
+----
+
+Presets are deliberately narrow: a request selects exactly one, the preset routes take no
+`config` part, and a preset never combines with request-supplied configuration — a variant is
+another preset. Because preset content is operator- or Tika-vetted, the preset routes do *not*
+require `allowPerRequestConfig`, and the `preset/{name}` path segment gives network controls an
+addressable surface — a reverse proxy can allow `/rmeta/preset/` while blocking `/rmeta/config`
+entirely. An unknown preset name answers `404`. Names are letters, digits, `.`, `_`, `-`
+(max 100 chars) and may not start with `config` (that path fragment gates the `/config`
+endpoints).
+
+Only the preset's *name* travels with a request: the forked parse worker resolves the content
+from its own copy of the server config, with the same trust as the config's own
+`parse-context` block. A preset can therefore bind components per-request `/config` input may
+not (detectors, embedded-document extraction, exception reporting, ...) and raise timeout
+limits above the per-request clamp. Every active preset resolves at startup, so a bad preset
+fails the server, not its first request.
+
+Output format on the preset routes: an explicit format segment (`/tika/preset/{name}/text`)
+always wins; without one, a `ContentHandlerFactory` the preset binds decides, then the
+config's, then the endpoint default (Markdown).
+
+On the batch surfaces, a `FetchEmitTuple` submitted to `/pipes` or `/async` may carry a
+top-level `"preset": "name"` field, resolved the same way beneath the tuple's own
+`parseContext`; an unknown name there answers the `PRESET_NOT_FOUND` status (HTTP `400`).
+
 === `allowPipes` — the `/pipes` and `/async` endpoints
 
 `/pipes` and `/async` drive process-isolated batch parsing through your configured fetchers and

tika-pipes/tika-pipes-api/src/main/java/org/apache/tika/pipes/api/FetchEmitTuple.java

@@ -38,6 +38,8 @@ public enum ON_PARSE_EXCEPTION {
     private final Metadata metadata;
     private final ParseContext parseContext;
     private final ON_PARSE_EXCEPTION onParseException;
+    // Preset selector only: the server resolves the content from its own config
+    private final String presetName;
 
     public FetchEmitTuple(String id, FetchKey fetchKey, EmitKey emitKey) {
         this(id, fetchKey, emitKey, new Metadata());
@@ -52,12 +54,19 @@ public FetchEmitTuple(String id, FetchKey fetchKey, EmitKey emitKey, Metadata me
 
     public FetchEmitTuple(String id, FetchKey fetchKey, EmitKey emitKey, Metadata metadata, ParseContext parseContext,
                           ON_PARSE_EXCEPTION onParseException) {
+        this(id, fetchKey, emitKey, metadata, parseContext, onParseException, null);
+    }
+
+    /** @since Apache Tika 4.1.0 */
+    public FetchEmitTuple(String id, FetchKey fetchKey, EmitKey emitKey, Metadata metadata, ParseContext parseContext,
+                          ON_PARSE_EXCEPTION onParseException, String presetName) {
         this.id = id;
         this.fetchKey = fetchKey;
         this.emitKey = emitKey;
         this.metadata = metadata;
         this.parseContext = parseContext;
         this.onParseException = onParseException;
+        this.presetName = presetName;
     }
 
     public String getId() {
@@ -86,6 +95,15 @@ public ON_PARSE_EXCEPTION getOnParseException() {
         return onParseException;
     }
 
+    /**
+     * The selected preset's name, or null for none.
+     *
+     * @since Apache Tika 4.1.0
+     */
+    public String getPresetName() {
+        return presetName;
+    }
+
     @Override
     public boolean equals(Object o) {
         if (this == o) {
@@ -98,7 +116,8 @@ public boolean equals(Object o) {
         FetchEmitTuple that = (FetchEmitTuple) o;
         return Objects.equals(id, that.id) && Objects.equals(fetchKey, that.fetchKey) && Objects.equals(emitKey, that.emitKey)
                 && Objects.equals(metadata, that.metadata) &&
-                Objects.equals(parseContext, that.parseContext) && onParseException == that.onParseException;
+                Objects.equals(parseContext, that.parseContext) && onParseException == that.onParseException &&
+                Objects.equals(presetName, that.presetName);
     }
 
     @Override
@@ -109,13 +128,14 @@ public int hashCode() {
         result = 31 * result + Objects.hashCode(metadata);
         result = 31 * result + Objects.hashCode(parseContext);
         result = 31 * result + Objects.hashCode(onParseException);
+        result = 31 * result + Objects.hashCode(presetName);
         return result;
     }
 
     @Override
     public String toString() {
         return "FetchEmitTuple{" + "id='" + id + '\'' + ", fetchKey=" + fetchKey + ", emitKey=" + emitKey +
                 ", metadata=" + metadata + ", parseContext=" + parseContext +
-                ", onParseException=" + onParseException + '}';
+                ", onParseException=" + onParseException + ", presetName='" + presetName + "'}";
     }
 }

tika-pipes/tika-pipes-api/src/main/java/org/apache/tika/pipes/api/PipesResult.java

@@ -66,6 +66,7 @@ public enum RESULT_STATUS {
         EMIT_EXCEPTION(CATEGORY.TASK_EXCEPTION),
         FETCHER_NOT_FOUND(CATEGORY.TASK_EXCEPTION),
         EMITTER_NOT_FOUND(CATEGORY.TASK_EXCEPTION),
+        PRESET_NOT_FOUND(CATEGORY.TASK_EXCEPTION),
         PAYLOAD_LIMIT_EXCEEDED(CATEGORY.TASK_EXCEPTION),
 
         // Process crashes - forked process died, auto-restart

tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/serialization/FetchEmitTupleDeserializer.java

@@ -26,6 +26,7 @@
 import static org.apache.tika.pipes.core.serialization.FetchEmitTupleSerializer.ID;
 import static org.apache.tika.pipes.core.serialization.FetchEmitTupleSerializer.METADATA_KEY;
 import static org.apache.tika.pipes.core.serialization.FetchEmitTupleSerializer.ON_PARSE_EXCEPTION;
+import static org.apache.tika.pipes.core.serialization.FetchEmitTupleSerializer.PRESET;
 import static org.apache.tika.serialization.serdes.ParseContextSerializer.PARSE_CONTEXT;
 
 import java.io.IOException;
@@ -40,6 +41,7 @@
 import com.fasterxml.jackson.databind.JsonDeserializer;
 import com.fasterxml.jackson.databind.JsonNode;
 
+import org.apache.tika.config.loader.PresetRegistry;
 import org.apache.tika.metadata.Metadata;
 import org.apache.tika.parser.ParseContext;
 import org.apache.tika.pipes.api.ComponentIds;
@@ -55,7 +57,7 @@ public class FetchEmitTupleDeserializer extends JsonDeserializer<FetchEmitTuple>
 
     private static final Set<String> KNOWN_KEYS = Set.of(
             ID, FETCHER, FETCH_KEY, EMITTER, EMIT_KEY, FETCH_RANGE_START, FETCH_RANGE_END,
-            METADATA_KEY, PARSE_CONTEXT, ON_PARSE_EXCEPTION);
+            METADATA_KEY, PARSE_CONTEXT, ON_PARSE_EXCEPTION, PRESET);
 
     private final boolean restricted;
 
@@ -114,10 +116,11 @@ public FetchEmitTuple deserialize(JsonParser jsonParser, DeserializationContext
         ParseContext parseContext = parseContextNode == null ? new ParseContext()
                 : ParseContextDeserializer.readParseContext(parseContextNode, true);
         FetchEmitTuple.ON_PARSE_EXCEPTION onParseException = readOnParseException(root);
+        String presetName = readPresetName(root);
 
         return new FetchEmitTuple(id, new FetchKey(fetcherId, fetchKey, fetchRangeStart, fetchRangeEnd),
                 new EmitKey(emitterName, emitKey), metadata, parseContext,
-                onParseException);
+                onParseException, presetName);
     }
 
     /**
@@ -147,6 +150,16 @@ private static void rejectUnknownKeys(JsonNode root) throws IOException {
         }
     }
 
+    // A preset name is only a selector; the shared syntax rule also bounds its length.
+    private static String readPresetName(JsonNode root) throws IOException {
+        String presetName = readVal(PRESET, root, null, false);
+        if (presetName != null && !PresetRegistry.isValidName(presetName)) {
+            throw new IOException("invalid preset name (letters, digits, '.', '_', '-'; " +
+                    "max 100 chars; may not start with 'config')");
+        }
+        return presetName;
+    }
+
     private static FetchEmitTuple.ON_PARSE_EXCEPTION readOnParseException(JsonNode root) throws IOException {
         JsonNode onParseExNode = root.get(ON_PARSE_EXCEPTION);
         if (onParseExNode == null) {

tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/serialization/FetchEmitTupleSerializer.java

@@ -40,6 +40,7 @@ public class FetchEmitTupleSerializer extends JsonSerializer<FetchEmitTuple> {
     public static final String EMIT_KEY = "emitKey";
     public static final String METADATA_KEY = "metadata";
     public static final String ON_PARSE_EXCEPTION = "onParseException";
+    public static final String PRESET = "preset";
 
     public void serialize(FetchEmitTuple t, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException {
 
@@ -59,6 +60,9 @@ public void serialize(FetchEmitTuple t, JsonGenerator jsonGenerator, SerializerP
             jsonGenerator.writeObjectField(METADATA_KEY, t.getMetadata());
         }
         jsonGenerator.writeStringField(ON_PARSE_EXCEPTION, t.getOnParseException().name().toLowerCase(Locale.US));
+        if (t.getPresetName() != null) {
+            jsonGenerator.writeStringField(PRESET, t.getPresetName());
+        }
         ParseContext parseContext = t.getParseContext();
         // Tailored: ParseContextSerializer's generic refusal suggests registering the
         // component -- for InlineBytes, exactly the forbidden fix.

tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/serialization/PipesRequest.java

@@ -60,7 +60,7 @@ public static PipesRequest of(FetchEmitTuple t) {
         copy.copyFrom(ctx);
         copy.set(InlineBytes.class, null);
         FetchEmitTuple stripped = new FetchEmitTuple(t.getId(), t.getFetchKey(), t.getEmitKey(),
-                t.getMetadata(), copy, t.getOnParseException());
+                t.getMetadata(), copy, t.getOnParseException(), t.getPresetName());
         return new PipesRequest(stripped, inline.getBytes());
     }
 

tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/server/ConnectionHandler.java

@@ -216,7 +216,8 @@ private void mainLoop() {
                         ParseContext mergedContext = null;
                         try {
                             long ctxStart = System.nanoTime();
-                            mergedContext = resources.createMergedParseContext(fetchEmitTuple.getParseContext());
+                            mergedContext = resources.createMergedParseContext(
+                                    fetchEmitTuple.getParseContext(), fetchEmitTuple.getPresetName());
                             ParseContextUtils.resolveAll(mergedContext, getClass().getClassLoader());
                             ServerProtocolIO.validateParseContext(mergedContext);
                             ServerProtocolIO.clampRequestTimeoutLimits(
@@ -239,6 +240,11 @@ private void mainLoop() {
 
                             loopUntilDone(fetchEmitTuple, mergedContext, intermediateResult, countDownLatch, parseTimeout);
                             logTiming(fetchEmitTuple.getId());
+                        } catch (PresetNotFoundException e) {
+                            // caller error, not a server fault: answer it and keep serving
+                            LOG.warn("handlerId={}: id={}: {}", handlerId, fetchEmitTuple.getId(), e.getMessage());
+                            protocolIO.writeFinished(new PipesResult(
+                                    PipesResult.RESULT_STATUS.PRESET_NOT_FOUND, e.getMessage()));
                         } catch (TikaConfigException e) {
                             LOG.error("handlerId={}: config error processing request", handlerId, e);
                             handleCrash(PipesMessageType.UNSPECIFIED_CRASH, fetchEmitTuple.getId(), e);