TIKA-4764 -- more granular selection of features in tika-server and tika-grpc (#2909)

Author: tballison

CHANGES.txt

@@ -18,6 +18,16 @@ Release 4.0.0-beta-1 - unreleased
      The extra jars are also forwarded onto forked pipes/server worker processes,
      so they are available where parsing actually happens (TIKA-4755).
 
+   * More granular, default-deny capability flags for tika-server and tika-grpc.
+     tika-server's enableUnsecureFeatures is split into allowPipes (gates the
+     /pipes and /async endpoints) and allowPerRequestConfig (gates the /config
+     endpoints and the multipart config part); the /status endpoint is no longer
+     gated and is enabled simply by listing it under endpoints. tika-grpc gains
+     the same allowPerRequestConfig flag plus allowComponentModifications (gates
+     runtime Save/Delete of fetchers and pipes iterators). All flags default to
+     false, so an out-of-the-box tika-grpc server no longer accepts per-request
+     configuration or runtime store mutations (TIKA-4764).
+
   OTHER CHANGES
 
    * Release artifacts are now channel-specific. Maven Central gets slim

docs/modules/ROOT/pages/advanced/integration-testing/run-uat-script.adoc

@@ -51,9 +51,9 @@ Coverage includes:
 * `/unpack/all` (embedded extraction; verifies the response is a valid zip)
 * `/language/stream`
 * `/meta/form`, `/rmeta/form` (multipart variants)
-* `enableUnsecureFeatures=false` gating: `/meta/config`, `/rmeta/config`,
-  `/tika/config` all return 403; and selecting the `/pipes`, `/async`, or `/status`
-  endpoints without `enableUnsecureFeatures` makes the server refuse to start
+* `allowPerRequestConfig=false` gating: `/meta/config`, `/rmeta/config`,
+  `/tika/config` all return 403; and `allowPipes=false` gating: selecting the
+  `/pipes` or `/async` endpoints without `allowPipes` makes the server refuse to start
 * `X-Tika-OCRskipOcr` header, `Content-Disposition` filename
 * 404 / 405 error handling
 

docs/modules/ROOT/pages/advanced/integration-testing/tika-server.adoc

@@ -212,39 +212,40 @@ curl -s -w "\nHTTP Status: %{http_code}\n" -X POST -F "file=@testPDF.pdf" http:/
 curl -s -w "\nHTTP Status: %{http_code}\n" -X POST -F "file=@testPDF.pdf" http://localhost:9998/unpack/config
 ----
 
-*Expected:* All return HTTP 403 with message: "Config endpoints are disabled. Set enableUnsecureFeatures=true in server config."
+*Expected:* All return HTTP 403 with message: "Config endpoints are disabled. Set allowPerRequestConfig=true in server config."
 
-=== Test 18b: `/pipes`, `/async`, `/status` Require enableUnsecureFeatures
+=== Test 18b: `/pipes` and `/async` Require allowPipes
 
 [source,bash]
 ----
-cat > tika-config-pipes-no-unsecure.json << 'EOF'
+cat > tika-config-pipes-no-flags.json << 'EOF'
 {
   "server": {"port": 9998, "endpoints": ["tika", "pipes"]},
   "pipes": {"numClients": 2},
   "plugin-roots": "/tmp/tika-server-test/plugins"
 }
 EOF
 
-java -jar tika-server-standard-4.0.0-SNAPSHOT.jar -c tika-config-pipes-no-unsecure.json
+java -jar tika-server-standard-4.0.0-SNAPSHOT.jar -c tika-config-pipes-no-flags.json
 ----
 
-*Expected:* The server refuses to start, failing with a `TikaConfigException` stating that the `pipes` endpoint requires `enableUnsecureFeatures` to be `true`. The same applies to `async` and `status`.
+*Expected:* The server refuses to start, failing with a `TikaConfigException` stating that the `pipes` endpoint requires `allowPipes` to be `true`. The same applies to `async`. (`/status` is no longer gated — it can be enabled simply by listing `status` under `endpoints`.)
 
-== Part 2: Tests with enableUnsecureFeatures
+== Part 2: Tests with allowPipes / allowPerRequestConfig
 
 Stop the default server and create a config file:
 
 [source,bash]
 ----
 pkill -f "tika-server-standard-4.0.0-SNAPSHOT.jar"
 
-cat > tika-config-unsecure.json << 'EOF'
+cat > tika-config-capabilities.json << 'EOF'
 {
   "server": {
     "port": 9998,
     "host": "localhost",
-    "enableUnsecureFeatures": true
+    "allowPipes": true,
+    "allowPerRequestConfig": true
   },
   "parsers": [
     {"default-parser": {}}
@@ -253,7 +254,7 @@ cat > tika-config-unsecure.json << 'EOF'
 }
 EOF
 
-java -jar tika-server-standard-4.0.0-SNAPSHOT.jar -c tika-config-unsecure.json &
+java -jar tika-server-standard-4.0.0-SNAPSHOT.jar -c tika-config-capabilities.json &
 sleep 10
 curl -s http://localhost:9998/version
 ----
@@ -382,7 +383,7 @@ rm -rf /tmp/tika-server-test
 
 The following endpoints were tested and verified working:
 
-=== Default Mode (enableUnsecureFeatures=false)
+=== Default Mode (allowPipes=false, allowPerRequestConfig=false)
 
 [cols="1,1,1", options="header"]
 |===
@@ -412,7 +413,7 @@ The following endpoints were tested and verified working:
 |`/unpack/config` |POST |BLOCKED (403) - Expected
 |===
 
-=== With enableUnsecureFeatures=true
+=== With allowPerRequestConfig=true
 
 [cols="1,1,1", options="header"]
 |===

docs/modules/ROOT/pages/advanced/robustness.adoc

@@ -97,7 +97,7 @@ tika-server restarts gracefully.
 
 tika-pipes::
 Available through programmatic use, tika-app `-a` option, or tika-server's `/async`
-and `/pipes` endpoints (the server endpoints require `enableUnsecureFeatures=true`).
+and `/pipes` endpoints (the server endpoints require `allowPipes=true`).
 
 == Security Testing and Prevention
 

docs/modules/ROOT/pages/configuration/index.adoc

@@ -74,7 +74,7 @@ used wherever a section is missing.
     "unpack-config": { /* embedded-byte extraction */ }
     /* other SelfConfiguring components by component name */
   },
-  "server": { /* tika-server options: enableUnsecureFeatures, cors, ... */ },
+  "server": { /* tika-server options: allowPipes, allowPerRequestConfig, cors, ... */ },
   "pipes": { /* Pipes process management: numClients, parseMode, ... */ },
   "fetchers": { /* named fetcher instances */ },
   "emitters": { /* named emitter instances */ },

docs/modules/ROOT/pages/migration-to-4x/migrating-tika-server-4x.adoc

@@ -132,17 +132,23 @@ The following `TikaServerConfig` options have been removed:
 
 * **Fetcher-based streaming** - The `InputStreamFactory` pattern for fetching documents via HTTP headers (`fetcherName`, `fetchKey`) has been removed. All documents are now processed via temp files through the pipes infrastructure.
 
-=== `/pipes`, `/async`, and `/status` Require `enableUnsecureFeatures`
+=== `/pipes` and `/async` Require `allowPipes`; Per-Request Config Requires `allowPerRequestConfig`
 
-Previously these endpoints were enabled simply by listing them under `server.endpoints`. They now *also* require `enableUnsecureFeatures` to be `true`; selecting any of `pipes`, `async`, or `status` without it causes the server to refuse to start with a clear error. `/pipes` and `/async` drive process-isolated batch parsing through your fetchers and emitters, and `/status` exposes server information, so this makes enabling them an explicit, deliberate opt-in.
+Previously these endpoints (and per-request parser configuration) were enabled simply by listing endpoints under `server.endpoints`. The capabilities are now split into two default-`false` flags in the `server` section:
 
-**Migration:** if your config selects `pipes`, `async`, or `status`, add `"enableUnsecureFeatures": true` to the `server` section:
+* `allowPipes` gates the `/pipes` and `/async` endpoints, which drive process-isolated batch parsing through your fetchers and emitters. Selecting either without `allowPipes` causes the server to refuse to start with a clear error.
+* `allowPerRequestConfig` gates per-request parser configuration: the `/config` family of endpoints and the multipart `config` part. When off, such requests are rejected with 403.
+
+`/status` is no longer gated: it exposes only aggregate counters, so it is enabled simply by listing `status` under `endpoints`.
+
+**Migration:** if your config selects `pipes` or `async`, add `"allowPipes": true`; if you rely on per-request config, add `"allowPerRequestConfig": true`:
 
 [source,json]
 ----
 {
   "server": {
-    "enableUnsecureFeatures": true,
+    "allowPipes": true,
+    "allowPerRequestConfig": true,
     "endpoints": ["tika", "rmeta", "pipes", "async", "status"]
   }
 }

docs/modules/ROOT/pages/pipes/index.adoc

@@ -29,7 +29,7 @@ While Tika Pipes has a programmatic Java API, it is best used through:
 
 * xref:using-tika/cli/index.adoc[tika-app] — batch processing from the command line
 * xref:using-tika/server/index.adoc[tika-server] — REST API with pipes-based robustness built in
-* xref:using-tika/grpc/index.adoc[tika-grpc] — gRPC API with pipes-based robustness built in
+* xref:using-tika/grpc/index.adoc[tika-grpc] — gRPC API with pipes-based robustness built in. More exposed by default than tika-server; run only on a trusted network (see xref:using-tika/grpc/index.adoc#_security[Security]).
 
 See xref:advanced/robustness.adoc[Robustness] for details on how Tika Pipes protects
 against problematic files.

docs/modules/ROOT/pages/pipes/timeouts.adoc

@@ -68,7 +68,7 @@ This can be combined with other parse-context settings:
 
 == Per-Request Overrides
 
-When using Tika Server with `enableUnsecureFeatures: true`, timeouts can be overridden per-request
+When using Tika Server with `allowPerRequestConfig: true`, timeouts can be overridden per-request
 by including `TimeoutLimits` in the `ParseContext` of a `FetchEmitTuple`:
 
 [source,java]

docs/modules/ROOT/pages/using-tika/grpc/index.adoc

@@ -29,6 +29,131 @@ register a fetcher (`SaveFetcher`) and then submit `FetchAndParseRequest`
 messages, each of which returns a `FetchAndParseReply` with extracted
 metadata and content.
 
+== Security
+
+[WARNING]
+====
+Treat tika-grpc as a privileged, trusted-network-only service — it is *more
+exposed by default than tika-server*. By default it has no transport security and
+no per-caller authorization, and its core `FetchAndParse` surface is always on:
+anyone who can reach the gRPC port can fetch and parse whatever the server's
+configured fetchers can reach.
+
+Its most dangerous capabilities — runtime mutation of the fetcher/iterator store
+(for example `SaveFetcher`, which can read files and load code on the server
+host) and per-request parse configuration — are *off by default*, behind the
+flags in <<_capability_flags>>, mirroring tika-server's default-deny capability
+flags (`allowPipes`, `allowPerRequestConfig`). Those flags are defense in depth,
+not a substitute for network controls and mutual TLS. Run tika-grpc only behind
+strict network controls.
+====
+
+=== Capability Flags
+
+Like tika-server, tika-grpc locks down its dangerous capabilities by default.
+The two flags below live in the `grpc` section of your tika-config and both
+default to `false`. Out of the box, the server only fetches and parses using the
+fetchers and pipes iterators declared in the config file, using the server's own
+parse configuration.
+
+[cols="1,3",options="header"]
+|===
+|Setting |Description
+
+|`allowPerRequestConfig`
+|When `true`, callers may attach per-request configuration to `FetchAndParse`
+requests (`parse_context_json` and `additional_fetch_config_json`), overriding
+the server's defaults for that request. Because this can reconfigure any pipeline
+component (fetcher, parser, timeouts, ...), it is off by default. When `false`, a
+request carrying either field is rejected with `PERMISSION_DENIED`.
+
+|`allowComponentModifications`
+|When `true`, callers may add, modify, or delete fetchers and pipes iterators at
+runtime (`SaveFetcher`, `DeleteFetcher`, `SavePipesIterator`,
+`DeletePipesIterator`). Because this changes what the server can reach for all
+subsequent requests (for example, adding a fetcher that escapes a configured base
+path), it is off by default. When `false`, those RPCs are rejected with
+`PERMISSION_DENIED`.
+|===
+
+Enable these only for trusted callers over a secured channel:
+
+[source,json]
+----
+{
+  "grpc": {
+    "allowPerRequestConfig": true,
+    "allowComponentModifications": true
+  }
+}
+----
+
+=== Transport Security (TLS)
+
+By default the gRPC server runs *without TLS*: connections are plaintext and
+unauthenticated. This includes the `apache/tika-grpc` Docker image, whose
+entrypoint does not pass `--secure`. Only run in this mode on a trusted network.
+
+Transport security is configured entirely through command-line flags (there is no
+JSON config for gRPC TLS), with three modes:
+
+*Insecure (default).* No `--secure` flag. Plaintext, no authentication.
+
+*Server (1-way) TLS.* Enable `-s`/`--secure` and supply the server certificate
+and key. The server authenticates to clients; clients are not authenticated.
+
+[source,bash]
+----
+java -jar tika-grpc-<version>.jar --secure \
+  --cert-chain server.pem --private-key server.key
+----
+
+Add `--private-key-password` if the private key is encrypted.
+
+*Mutual (2-way) TLS.* Additionally supply the trust collection (the CA used to
+verify client certificates) and require client authentication:
+
+[source,bash]
+----
+java -jar tika-grpc-<version>.jar --secure \
+  --cert-chain server.pem --private-key server.key \
+  --trust-cert-collection ca.pem --client-auth-required
+----
+
+Mutual TLS is opt-in: `--client-auth-required` is off by default, so it has no
+effect unless `--trust-cert-collection` is also given (a missing or non-existent
+trust-collection path is silently ignored). The default port is `50052`
+(`-p`/`--port`).
+
+When running the Docker image, append these flags to the container command — they
+are forwarded to the server — and mount the certificate files into the container.
+
+=== Kubernetes and Service Meshes
+
+Running tika-grpc in Kubernetes does not make it safe on its own. By default, pod
+networking is flat: any pod can reach any other pod's port, traffic is
+unencrypted, and there is no authentication or authorization between pods.
+Kubernetes gives you the tools to lock this down, but none of them are applied
+automatically. Two distinct controls are involved, and both matter:
+
+* *Transport security.* A service mesh (for example Istio or Linkerd) with sidecar
+  mTLS encrypts and authenticates pod-to-pod traffic. If the mesh provides this,
+  you can run tika-grpc without `--secure` and let the mesh handle transport
+  security in place of the TLS flags above.
+* *Reachability.* A `NetworkPolicy` that admits only your trusted client(s) to the
+  tika-grpc Service. This is the control that actually mitigates the exposure
+  described above: because tika-grpc has no per-caller authorization, the set of
+  pods that can reach the port is effectively the set of pods that can use its
+  enabled RPC surface.
+
+Mesh mTLS authenticates *who opened the connection*; it does not authorize *what
+that caller may do*, so an authenticated-but-untrusted pod can still invoke
+whatever RPC surface is enabled — at minimum `FetchAndParse` against your
+configured fetchers, and the runtime-mutation RPCs too if you have set
+`allowComponentModifications`. Restricting reachability with a `NetworkPolicy` is
+therefore required, not optional — running in Kubernetes without one leaves
+tika-grpc reachable by every pod in the cluster.
+
 == Per-Request `ParseContext`
 
 `FetchAndParseRequest.parse_context_json` lets the caller override the
@@ -46,6 +171,11 @@ parse-context component names; values are their JSON configs.
 See `META-INF/tika/parse-context.idx` (generated at build time from
 `@TikaComponent` annotations) for the available component names.
 
+NOTE: Per-request configuration is disabled by default. A request that sets
+`parse_context_json` (or `additional_fetch_config_json`) is rejected with
+`PERMISSION_DENIED` unless `allowPerRequestConfig` is enabled. See
+<<_capability_flags,Capability Flags>>.
+
 == Topics
 
 // Add links to specific topics as they are created

docs/modules/ROOT/pages/using-tika/index.adoc

@@ -36,6 +36,13 @@ and microservice architectures.
 
 xref:using-tika/grpc/index.adoc[gRPC]::
 Use Tika via gRPC protocol. Best for high-performance, cross-language communication.
++
+NOTE: tika-grpc is more exposed by default than tika-server — by default it has no
+transport security or per-caller authorization, and its core fetch-and-parse surface
+is always on. (Its most dangerous capabilities — runtime fetcher/iterator changes and
+per-request parse configuration — are off by default.) Run it only on a trusted,
+access-controlled network. See the
+xref:using-tika/grpc/index.adoc#_security[Security] section.
 
 == Which Should I Use?