TIKA-4809 stage 4

Author: tballison

docs/modules/ROOT/pages/migration-to-4x/migrating-tika-server-4x.adoc

@@ -176,6 +176,75 @@ The following `TikaServerConfig` options have been removed:
 * `taskPulseMillis` - No longer needed
 * `minimumTimeoutMillis` - No longer needed
 
+=== Configuration via HTTP Headers Removed
+
+4.x configures parsing through the tika-config file, not through request headers.
+The remaining per-request configuration headers have been removed and are now
+**silently ignored** if sent — the request succeeds, the header has no effect:
+
+|===
+|Removed header |Replacement
+
+|`writeLimit`
+|`parse-context.output-limits.writeLimit`
+
+|`throwOnWriteLimitReached`
+|`parse-context.output-limits.throwOnWriteLimit`
+
+|`maxEmbeddedResources`, `maxEmbeddedCount`
+|`parse-context.embedded-limits.maxCount`
+
+|`meta_*` (arbitrary metadata injection)
+|_no replacement_ — see below
+
+|`X-Tika-Handler`
+|Use an explicit handler path (`/tika/text`, `/rmeta/xml`, `/tika/json/html`, ...)
+|===
+
+The `X-Tika-OCR*` and `X-Tika-PDF*` header families were removed earlier in the
+4.x line. Parser configuration is now supplied as JSON.
+
+[source,json]
+----
+{
+  "parse-context": {
+    "output-limits": {
+      "writeLimit": 100000,
+      "throwOnWriteLimit": false
+    },
+    "embedded-limits": {
+      "maxCount": 10
+    }
+  }
+}
+----
+
+**Migration:** move these settings into your tika-config. For per-request values,
+POST to a `/config` endpoint (`/tika/config`, `/tika/config/json`, `/rmeta/config`,
+`/meta/config`) with the JSON above as the multipart `config` part; this requires
+`allowPerRequestConfig=true`.
+
+[IMPORTANT]
+====
+Two capabilities are genuinely gone, not relocated.
+
+**Per-request output bounds without `allowPerRequestConfig`.** With the headers
+removed and per-request config off by default, a caller can no longer bound the
+output of a single request; the limits are whatever the operator configured. If
+you relied on clients setting their own `writeLimit`, either enable
+`allowPerRequestConfig` or set a server-wide limit.
+
+**Client-supplied metadata (`meta_*`).** Headers prefixed `meta_` were copied
+into the returned metadata under the remainder of the header name, with no key
+restrictions — so a request could also overwrite keys Tika itself populates.
+There is no replacement on the push endpoints; attach provenance on your side of
+the call, or use `/pipes`, where the `FetchEmitTuple` carries metadata.
+====
+
+Transport headers are unaffected: `Content-Disposition`/`File-Name`,
+`Content-Type` and `Content-Length` still describe the payload and still
+influence detection.
+
 === Removed Features
 
 * **Fetcher-based streaming** - The `InputStreamFactory` pattern for fetching documents via HTTP headers (`fetcherName`, `fetchKey`) has been removed. All documents are now processed via temp files through the pipes infrastructure.

docs/modules/ROOT/pages/migration-to-4x/migrating-to-4x.adoc

@@ -45,7 +45,7 @@ If you have build scripts or container images that drop in just the jar, update
 In 3.x the default content handler produced XHTML/XML. In 4.x the default is **Markdown** everywhere:
 
 * `tika-app` outputs Markdown by default (was XHTML). Pass `-x`/`--xml`, `-h`/`--html`, or `-t`/`--text` to choose another format.
-* `tika-server` — the `/tika` and `/rmeta` endpoints return Markdown content by default (was XHTML/XML). Use an explicit handler path (`/tika/xml`, `/rmeta/xml`, ...) or the `X-Tika-Handler` header to choose another format.
+* `tika-server` — the `/tika` and `/rmeta` endpoints return Markdown content by default (was XHTML/XML). Use an explicit handler path (`/tika/xml`, `/rmeta/xml`, ...) to choose another format.
 * The async/pipes CLI emits Markdown by default (was plain text). Use `--handler x` (etc.) to choose another format.
 
 If you parse the extracted content programmatically and expect XHTML/XML, request it explicitly as shown above (TIKA-4663).

docs/modules/ROOT/pages/pipes/cpu-sizing.adoc

@@ -169,7 +169,7 @@ report `autoCap=user-set in forkedJvmArgs`.
 A reasonable starting point is **~2 GB of heap per forked worker** (passed via `-Xmx2g` in `forkedJvmArgs`). The number falls out of three independent constraints any of which can dominate:
 
 * **Worst-case PDF parsing.** A handful of pathological PDFs in any reasonably large corpus will allocate hundreds of MB of intermediate object data per document — large image streams, deeply nested form fields, big embedded fonts. Smaller heaps OOM on those documents; larger heaps just let GC clean up between docs.
-* **Embedded-document explosion.** A zip-bomb-shaped office document with thousands of embedded objects multiplies per-doc allocation by the embedding count. The `maxEmbeddedResources` setting caps the count, but each retained object still lives in the heap until the whole tree finishes parsing.
+* **Embedded-document explosion.** A zip-bomb-shaped office document with thousands of embedded objects multiplies per-doc allocation by the embedding count. The `parse-context.embedded-limits.maxCount` setting caps the count, but each retained object still lives in the heap until the whole tree finishes parsing.
 * **GC headroom.** G1GC behaves poorly above ~85% occupancy. A `-Xmx2g` worker comfortably handles documents that allocate up to ~1.5 GB of live data; below that you start trading throughput for memory.
 
 This is a default — not a tuning recommendation. To right-size for your specific corpus:

docs/modules/ROOT/pages/using-tika/server/index.adoc

@@ -143,17 +143,6 @@ the JSON variant, you can also nest a handler — `/tika/json/text`,
 envelope; that nested handler accepts the full set (`text`, `html`, `xml`,
 `md`, `markdown`, `body`, `ignore`).
 
-==== `X-Tika-Handler` header
-
-For the root `/tika` PUT endpoint you can also pick the handler with a header:
-
-[source,bash]
-----
-curl -T document.pdf -H "X-Tika-Handler: markdown" http://localhost:9998/tika
-----
-
-Accepted values: `text`, `html`, `xml`, `markdown` (or `md`), `body`, `ignore`. The default is `markdown`.
-
 === Recursive Metadata (`/rmeta`)
 
 Returns metadata for the container document and all embedded documents as a JSON

tika-server/tika-server-core/src/main/java/org/apache/tika/server/core/resource/RecursiveMetadataResource.java

@@ -17,7 +17,6 @@
 package org.apache.tika.server.core.resource;
 
 import static org.apache.tika.server.core.resource.TikaResource.fillMetadata;
-import static org.apache.tika.server.core.resource.TikaResource.getWriteLimit;
 import static org.apache.tika.server.core.resource.TikaResource.setupContentHandlerFactory;
 import static org.apache.tika.server.core.resource.TikaResource.setupContentHandlerFactoryIfNeeded;
 
@@ -38,7 +37,6 @@
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
-import org.apache.tika.config.EmbeddedLimits;
 import org.apache.tika.io.TikaInputStream;
 import org.apache.tika.metadata.Metadata;
 import org.apache.tika.parser.ParseContext;
@@ -65,41 +63,20 @@ public RecursiveMetadataResource(TikaResource tikaResource) {
      */
     public List<Metadata> parseMetadata(TikaInputStream tis, Metadata metadata,
                                                MultivaluedMap<String, String> httpHeaders,
-                                               ServerHandlerConfig handlerConfig)
+                                               String handlerTypeName)
             throws Exception {
 
         final ParseContext context = tikaResource.createParseContext();
 
         fillMetadata(null, metadata, httpHeaders);
         TikaResource.logRequest(LOG, "/rmeta", metadata);
 
-        // Set up handler factory in context using shared utility
-        setupContentHandlerFactory(context, handlerConfig.type().toString(), handlerConfig.writeLimit(),
-                handlerConfig.throwOnWriteLimitReached());
-
-        // Set up embedded limits if specified
-        if (handlerConfig.maxEmbeddedCount() >= 0) {
-            EmbeddedLimits limits = new EmbeddedLimits();
-            limits.setMaxCount(handlerConfig.maxEmbeddedCount());
-            context.set(EmbeddedLimits.class, limits);
-        }
+        setupContentHandlerFactory(context, handlerTypeName);
 
         // Filtering is done in child process, no need to filter again
         return tikaResource.parseWithPipes(tis, metadata, context, ParseMode.RMETA);
     }
 
-    static ServerHandlerConfig buildHandlerConfig(MultivaluedMap<String, String> httpHeaders, String handlerTypeName, ParseMode parseMode) {
-        int maxEmbeddedCount = -1;
-        // Support both old header name and new for backwards compatibility
-        if (httpHeaders.containsKey("maxEmbeddedResources")) {
-            maxEmbeddedCount = Integer.parseInt(httpHeaders.getFirst("maxEmbeddedResources"));
-        } else if (httpHeaders.containsKey("maxEmbeddedCount")) {
-            maxEmbeddedCount = Integer.parseInt(httpHeaders.getFirst("maxEmbeddedCount"));
-        }
-        return new ServerHandlerConfig(BasicContentHandlerFactory.parseHandlerType(handlerTypeName, DEFAULT_HANDLER_TYPE), parseMode,
-                getWriteLimit(httpHeaders), maxEmbeddedCount, TikaResource.getThrowOnWriteLimitReached(httpHeaders));
-    }
-
     /**
      * Returns an InputStream that can be deserialized as a list of
      * {@link Metadata} objects.
@@ -134,7 +111,7 @@ public Response getMetadataFromMultipart(Attachment att, @PathParam(HANDLER_TYPE
         try (TikaInputStream tis = TikaInputStream.get(att.getObject(InputStream.class))) {
             tis.getPath(); // Spool to temp file for pipes-based parsing
             List<Metadata> metadataList = parseMetadata(tis, Metadata.newInstance(context), att.getHeaders(),
-                    buildHandlerConfig(att.getHeaders(), handlerTypeName, ParseMode.RMETA));
+                    handlerTypeName);
             return Response.ok(new MetadataList(metadataList)).build();
         }
     }
@@ -159,18 +136,14 @@ public Response getMetadataWithConfig(
             TikaResource.logRequest(LOG, "/rmeta/config", metadata);
 
             return Response
-                    .ok(parseMetadataWithContext(tis, metadata, httpHeaders.getRequestHeaders(),
-                            buildHandlerConfig(httpHeaders.getRequestHeaders(), null, ParseMode.RMETA),
-                            context))
+                    .ok(parseMetadataWithContext(tis, metadata, null, context))
                     .build();
         }
     }
 
-    private MetadataList parseMetadataWithContext(TikaInputStream tis, Metadata metadata, MultivaluedMap<String, String> httpHeaders,
-                                                  ServerHandlerConfig handlerConfig, ParseContext context) throws Exception {
-        // Set up handler factory in context if not already set using shared utility
-        setupContentHandlerFactoryIfNeeded(context, handlerConfig.type().toString(),
-                handlerConfig.writeLimit(), handlerConfig.throwOnWriteLimitReached());
+    private MetadataList parseMetadataWithContext(TikaInputStream tis, Metadata metadata,
+                                                  String handlerTypeName, ParseContext context) throws Exception {
+        setupContentHandlerFactoryIfNeeded(context, handlerTypeName);
 
         // Filtering is done in child process, no need to filter again
         List<Metadata> metadataList = tikaResource.parseWithPipes(tis, metadata, context, ParseMode.RMETA);
@@ -210,7 +183,7 @@ public Response getMetadata(InputStream is, @Context HttpHeaders httpHeaders, @P
         try (TikaInputStream tis = TikaInputStream.get(is)) {
             tis.getPath(); // Spool to temp file for pipes-based parsing
             List<Metadata> metadataList = parseMetadata(tis, metadata, httpHeaders.getRequestHeaders(),
-                    buildHandlerConfig(httpHeaders.getRequestHeaders(), handlerTypeName, ParseMode.RMETA));
+                    handlerTypeName);
             return Response.ok(new MetadataList(metadataList)).build();
         }
     }