TIKA-4872 - improve ocr wiring (#3119)

Author: tballison

CHANGES.txt

@@ -1,5 +1,38 @@
 Release 4.1.0 - unreleased
 
+   * New "content-enrichers" config list (TIKA-4872): select the OCR engine
+     ("tesseract-ocr-parser", "tess4j-parser", "openai-vlm-parser", ...) by
+     name instead of by classpath registration of the image/ocr-* pseudo
+     media types. Enrichers advertise real media types (legacy engines that
+     still advertise image/ocr-* are mapped to the real type, so all are
+     nameable) and are invoked by the image and PDF parsers rather than
+     dispatched to by the composite, so an enricher no longer displaces the
+     parser registered for the same type. Enricher selection uses the
+     detected media type, captured before a parser can refine Content-Type.
+     Every enricher matching a media type runs, in config order (e.g. an
+     OCR engine then a VLM tagger for the same image), best-effort: one
+     enricher's failure does not stop the others and is still reported;
+     timeouts abort the chain. The list is authoritative: a media type no
+     configured enricher matches gets no enrichment -- never a classpath
+     engine that was not named -- and a named engine that reports no media
+     types at load (missing binary, unreachable inference server) fails
+     config load instead of going silently inert. With no
+     "content-enrichers" configured, the legacy ocr-* dispatch applies
+     unchanged; a WARN at config load now names colliding OCR engines and
+     the winner. TesseractOCRParser's
+     component name is pinned as "tesseract-ocr-parser".
+
+   * Inference/OCR hardening (TIKA-4871): OpenAIVLMParser no longer
+     auto-registers via SPI, matching its Claude/Gemini siblings; select
+     it by name ("openai-vlm-parser") in config. Per-request parse-context
+     config for the embedding filters now works and is validated:
+     {"openai-embedding-filter": {"skipEmbedding": true}} (likewise
+     "jina-embedding-filter") merges over the server config, and
+     baseUrl/apiKey/model may not be changed at runtime. The embedding
+     filters release their HTTP client resources on close(). Inline PDF
+     page OCR now accumulates tk:chunks from every OCR'd page onto the
+     parent document instead of keeping only the first page's.
+
    * Placeholder streams -- the empty stand-ins parsers hand parseEmbedded
      for content that is never extracted -- report an unknown length rather
      than their own zero, and the macro-failure entry is registered without

docs/modules/ROOT/pages/configuration/index.adoc

@@ -38,6 +38,7 @@ optional; anything you omit uses its defaults.
   "encoding-detectors": [ /* encoding detector declarations */ ],
   "metadata-filters": [ /* metadata filter declarations */ ],
   "renderers": [ /* page renderer declarations */ ],
+  "content-enrichers": [ /* OCR engines etc., selected by name; see below */ ],
   "translator": { /* translator declaration */ },
   "content-handler-factory": { /* handler type for emitted content */ },
   "auto-detect-parser": { /* AutoDetectParser options */ },
@@ -125,6 +126,36 @@ Configuring a parser automatically excludes its default copy, so there is no dup
 `default-encoding-detector`, but it must not be mixed with explicit detector entries — see
 xref:configuration/encoding-detectors.adoc[Encoding Detectors].
 
+== The `content-enrichers` list (4.1.0+)
+
+Content enrichers are ordinary parsers that a container parser *invokes* on bytes it has already
+parsed — an OCR engine run on an embedded image or a rendered PDF page. Naming one here selects
+the engine explicitly instead of relying on which OCR module happens to be on the classpath:
+
+[source,json]
+----
+{
+  "content-enrichers": [
+    { "tesseract-ocr-parser": { "language": "eng" } }
+  ]
+}
+----
+
+An enricher advertises its *real* media types (`image/png`, ...) and does not compete with the
+parser registered for those types: `image-parser` still parses the image and calls the enricher.
+(The bundled OCR engines still advertise legacy `image/ocr-*` types; those are mapped to the
+real type, so naming them here just works.) *Every* enricher matching a media type runs, in the
+order listed — so an OCR engine followed by a VLM that tags images is two entries, both invoked
+per image. The list is authoritative: a media type no configured enricher matches gets no
+enrichment — never a classpath engine you did not name — and a named engine that reports no
+media types at startup (missing native binary, unreachable inference server) fails config load
+rather than going silently inert. Failures are best-effort: one enricher failing does not stop
+the others, and every failure is still reported through the parser's normal exception handling
+(timeouts abort the chain immediately). With no `content-enrichers` configured, behavior is
+unchanged — whichever OCR engine is on the classpath is used, exactly as before, and a WARN is
+logged at startup when several engines collide. Engine names: `tesseract-ocr-parser`, `tess4j-parser`, `openai-vlm-parser`,
+`claude-vlm-parser`, `gemini-vlm-parser`.
+
 == Windows file paths
 
 JSON treats the backslash as an escape character, so path options (`tesseractPath`,

docs/modules/ROOT/pages/configuration/metadata-filters.adoc

@@ -233,6 +233,11 @@ around.
 `parse-context` can carry a per-request `metadata-filters` list, which *replaces* the configured
 one for that request rather than adding to it.
 
+Since 4.1.0, the embedding filters also accept per-request config under their own name --
+e.g. `{"parse-context": {"openai-embedding-filter": {"skipEmbedding": true}}}` (likewise
+`jina-embedding-filter`) -- which is merged over the server-side config for that request.
+Endpoint fields (`baseUrl`, `apiKey`, `model`) cannot be changed per request.
+
 NOTE: In `CONTENT_ONLY` xref:pipes/parse-modes.adoc[parse mode], Tika applies an
 `include-field-metadata-filter` for `tk:content` and `tk:exception:container-exception` when you have
 configured no filter of your own. Only a filter that reaches the `parse-context` replaces it —

docs/modules/ROOT/pages/configuration/parsers/tess4j-parser.adoc

@@ -32,6 +32,13 @@ with a measured need for in-process OCR throughput *and* the expertise to run na
 safely.
 ====
 
+Component name: `tess4j-parser`. Adding `tika-parser-tess4j-module` to the classpath is a
+deliberate opt-in and is intended to make Tess4J the OCR engine — but when both engines are
+live, the winner is decided by registration order, which is not guaranteed. Since 4.1.0 a WARN
+at startup names any engine collision and the winner; to pin the engine deterministically, name
+it in the top-level `content-enrichers` list (`tess4j-parser` or `tesseract-ocr-parser`; see
+xref:configuration/index.adoc[Configuration]).
+
 `Tess4JParser` calls the Tesseract native library in-process via
 https://github.com/nguyenq/tess4j[Tess4J] and JNA instead of spawning a `tesseract` child process
 per image. That removes the per-file process-spawn overhead and can be significantly faster on

docs/modules/ROOT/pages/configuration/parsers/tesseract-ocr-parser.adoc

@@ -18,7 +18,23 @@
 = TesseractOCRParser Configuration
 
 Configuration options for `TesseractOCRParser`, which runs the `tesseract` command-line program in
-a separate process.
+a separate process. Component name: `tesseract-ocr-parser`.
+
+== Selecting the OCR engine (4.1.0+)
+
+With no configuration, Tesseract is the OCR engine whenever the `tesseract` binary is found.
+When more than one OCR engine is on the classpath (e.g. Tess4J or a VLM parser), a WARN at
+startup names the collision and the winner. To pin the engine explicitly, name it in the
+top-level `content-enrichers` list:
+
+[source,json]
+----
+{
+  "content-enrichers": [ { "tesseract-ocr-parser": { "language": "eng" } } ]
+}
+----
+
+See xref:configuration/index.adoc[Configuration] for how `content-enrichers` works.
 
 == Basic Configuration
 

docs/modules/ROOT/pages/configuration/parsers/vlm-parsers.adoc

@@ -22,27 +22,27 @@ to remote Vision-Language Model (VLM) endpoints. These parsers send images
 (or PDFs) to an external API and convert the model's markdown response into
 structured XHTML.
 
-Three implementations are provided out of the box. Only `openai-vlm-parser` is auto-loaded via SPI;
-the other two must be named explicitly in your configuration.
+Three implementations are provided out of the box. None is auto-loaded: each must be
+named explicitly in your configuration. (Changed in 4.1.0: `openai-vlm-parser` previously
+auto-registered via SPI.) To use a VLM as the OCR engine for embedded images and rendered
+PDF pages, name it in the `content-enrichers` list — see
+xref:configuration/index.adoc[Configuration].
 
-[cols="1,2,1,1"]
+[cols="1,2,1"]
 |===
-|Parser |Endpoint |Config key |Auto-loaded
+|Parser |Endpoint |Config key
 
 |`OpenAIVLMParser`
 |Any OpenAI-compatible chat completions endpoint (vLLM, Ollama, local FastAPI, OpenAI)
 |`openai-vlm-parser`
-|Yes
 
 |`ClaudeVLMParser`
 |Anthropic Messages API
 |`claude-vlm-parser`
-|No
 
 |`GeminiVLMParser`
 |Google Gemini `generateContent` API
 |`gemini-vlm-parser`
-|No
 |===
 
 All three handle the standard OCR image types (`image/ocr-png`, `image/ocr-jpeg`, ...).

tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java

@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.parser.enricher;
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.tika.mime.MediaType;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.Parser;
+
+/**
+ * Media-type-keyed registry of content enrichers: ordinary {@link Parser}s that a container
+ * parser <em>invokes</em> on bytes it has already parsed (OCR text for an image or a
+ * rendered PDF page), rather than being dispatched to by the composite parser. Configured
+ * as the top-level {@code "content-enrichers"} list, mirroring {@code "renderers"}.
+ * <p>
+ * Members advertise their <em>real</em> media types ({@code image/png}); legacy engines
+ * still advertising the {@code image/ocr-*} pseudo-types are keyed under the real type, so
+ * they are nameable here unmodified. An enricher does not compete with the parser
+ * registered for the same type: that parser still runs and calls the enricher.
+ *
+ * @since Apache Tika 4.1
+ */
+public class CompositeContentEnricher implements Serializable {
+
+    private static final long serialVersionUID = 1L;
+
+    private final Map<MediaType, List<Parser>> enricherMap;
+
+    public CompositeContentEnricher(List<Parser> enrichers) {
+        Map<MediaType, List<Parser>> tmp = new HashMap<>();
+        ParseContext empty = new ParseContext();
+        for (Parser enricher : enrichers) {
+            for (MediaType mediaType : enricher.getSupportedTypes(empty)) {
+                // legacy engines advertise image/ocr-*; key under the real type
+                MediaType keyType = stripLegacyOcrPrefix(mediaType.getBaseType());
+                List<Parser> forType = tmp.computeIfAbsent(keyType, k -> new ArrayList<>());
+                if (!forType.contains(enricher)) {
+                    forType.add(enricher);
+                }
+            }
+        }
+        tmp.replaceAll((k, v) -> Collections.unmodifiableList(v));
+        this.enricherMap = Collections.unmodifiableMap(tmp);
+    }
+
+    private static MediaType stripLegacyOcrPrefix(MediaType mediaType) {
+        String subtype = mediaType.getSubtype();
+        if (subtype.startsWith(LegacyDispatchEnricher.OCR_MEDIATYPE_PREFIX)) {
+            return new MediaType(mediaType.getType(),
+                    subtype.substring(LegacyDispatchEnricher.OCR_MEDIATYPE_PREFIX.length()));
+        }
+        return mediaType;
+    }
+
+    /**
+     * @return the enrichers for this media type in config order, empty when none;
+     *         parameters are ignored, alias normalization is the caller's job
+     */
+    public List<Parser> getEnrichers(MediaType mediaType) {
+        List<Parser> enrichers = enricherMap.get(mediaType.getBaseType());
+        return enrichers == null ? Collections.emptyList() : enrichers;
+    }
+
+    public Set<MediaType> getSupportedTypes() {
+        return enricherMap.keySet();
+    }
+}