Skip to content

Commit 6b53816

Browse files
tballisontballison
authored
Automatically add slices and/or log underprovisioned pipes configurations (#2793)
1 parent 4e0340b commit 6b53816

4 files changed

Lines changed: 238 additions & 2 deletions

File tree

‎docs/modules/ROOT/nav.adoc‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@
3030
** xref:pipes/parse-modes.adoc[Parse Modes]
3131
** xref:pipes/unpack-config.adoc[Extracting Embedded Bytes]
3232
** xref:pipes/timeouts.adoc[Timeouts]
33+
** xref:pipes/cpu-sizing.adoc[Forked-JVM CPU Sizing]
3334
* xref:configuration/index.adoc[Configuration]
3435
** xref:configuration/parsers/pdf-parser.adoc[PDF Parser]
3536
** xref:configuration/parsers/tesseract-ocr-parser.adoc[Tesseract OCR]

‎docs/modules/ROOT/pages/pipes/configuration.adoc‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,11 +42,11 @@ how many forked JVMs to run, timeouts, memory management, and parse behavior.
4242

4343
|`numClients`
4444
|`4`
45-
|Number of parallel forked JVM processes. Each processes one document at a time.
45+
|Number of parallel forked JVM processes. Each processes one document at a time. See xref:pipes/cpu-sizing.adoc[Forked-JVM CPU Sizing] for guidance on choosing this value relative to host CPU count.
4646

4747
|`forkedJvmArgs`
4848
|`[]`
49-
|JVM arguments for forked processes (e.g., `["-Xmx512m", "-Xms256m"]`).
49+
|JVM arguments for forked processes (e.g., `["-Xmx512m", "-Xms256m"]`). When `numClients > 1`, Tika auto-injects `-XX:ActiveProcessorCount` to right-size each fork's GC and JIT thread pools unless you provide your own; see xref:pipes/cpu-sizing.adoc[Forked-JVM CPU Sizing].
5050

5151
|`javaPath`
5252
|`java`

‎docs/modules/ROOT/pages/pipes/cpu-sizing.adoc‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
= Forked-JVM CPU Sizing
2+
3+
Tika Pipes runs multiple forked JVMs in per-client mode (one per `numClients`).
4+
Each JVM independently sizes its garbage collector, JIT compiler, and common
5+
`ForkJoinPool` based on the host CPU count. Without intervention, this causes
6+
thread-pool blowup at high `numClients`: e.g., 4 forks on a 16-core host
7+
default to ~16 GC threads × 4 = ~64 GC threads, all competing for the same 16
8+
cores.
9+
10+
To fix this, Tika Pipes auto-injects `-XX:ActiveProcessorCount` into each
11+
forked JVM's command line, sizing each fork's view of the CPU count to a fair
12+
slice of the host. This is on by default in per-client mode (`numClients > 1`)
13+
when the user has not already supplied `-XX:ActiveProcessorCount` in
14+
`forkedJvmArgs`.
15+
16+
== Mental model
17+
18+
----
19+
pod_cpus = parent_overhead (≈ 2) + numClients × per_fork_slice
20+
----
21+
22+
Where `per_fork_slice ≥ 2`:
23+
24+
* 1 CPU for the parser thread
25+
* 1 CPU for everything else the JVM does (GC concurrent worker, JIT,
26+
protocol heartbeat, socket I/O thread)
27+
28+
The parent JVM (the one running `tika-async-cli` / `tika-app -a`) is light
29+
on CPU — it just serializes requests, deserializes responses, and runs
30+
the heartbeat — but it must not be CPU-starved. A starved parent shows up
31+
as pathological tail latency on small operations like `socket.write()`,
32+
because the calling thread gets preempted between clock reads. We reserve
33+
2 cores for the parent by default.
34+
35+
== Formula
36+
37+
[source]
38+
----
39+
slice = (hostCores - PARENT_RESERVED_CORES) / numClients
40+
41+
PARENT_RESERVED_CORES = 2
42+
MIN_AUTO_CAP_SLICE = 2
43+
----
44+
45+
If `slice ≥ 2`, Tika injects `-XX:ActiveProcessorCount=<slice>` into each
46+
forked JVM. If `slice < 2`, the auto-cap is *skipped* and a `WARN` is
47+
logged advising the operator to lower `numClients`. Skipping is intentional:
48+
at `slice=1` the fork's only CPU is fully consumed by parsing, so its
49+
socket-reader thread cannot run and the parent's writes block on
50+
receiver-side back-pressure — measurably worse than no cap at all.
51+
52+
== Recommended sizing
53+
54+
For typical cloud-VM core counts:
55+
56+
[cols="1,1,1,3"]
57+
|===
58+
|hostCores |numClients |slice |Notes
59+
60+
|2 |1 |n/a |Tight; auto-cap not applied (single fork). Acceptable for low throughput.
61+
|4 |1 |n/a |Comfortable single-fork deployment.
62+
|4 |2 |1 → skipped |Auto-cap declines; consider `numClients=1`.
63+
|8 |1 |n/a |Lots of headroom; single-fork lifecycle isolation is fine.
64+
|8 |3 |2 |Sweet spot for medium pods.
65+
|16 |4 |3 |Sweet spot for 16-core hosts. Measured winner in benchmarks.
66+
|16 |6 |2 |Higher concurrency; tighter per-fork breathing room.
67+
|16 |8 |1 → skipped |Doesn't fit 16 cores. Keep at 4 or 6.
68+
|32 |8 |3 |Same shape as 16/4.
69+
|===
70+
71+
The general rule is: pick the largest `numClients` that satisfies
72+
`numClients × 2 + 2 ≤ hostCores`. Beyond that point, adding workers
73+
starts hurting throughput.
74+
75+
== Diagnostics
76+
77+
Every `PipesParser` startup emits a one-shot summary line on its main
78+
logger so operators can see what was decided:
79+
80+
[source]
81+
----
82+
INFO pipes-cpu-sizing: hostCores=16, numClients=4, parentReserved=2, autoCap=slice=3
83+
----
84+
85+
The `autoCap` field is one of:
86+
87+
* `slice=N` — the auto-cap fired; each fork sees N CPUs.
88+
* `skipped (slice<2)` — over-provisioned; operator should reduce `numClients`.
89+
* `n/a (single fork; not capped)` — `numClients=1`; fork sees the whole host.
90+
* `user-set in forkedJvmArgs` — operator set `-XX:ActiveProcessorCount` themselves.
91+
92+
Two `WARN`-level messages call out clearly-bad provisioning:
93+
94+
* `hostCores < 2` — the host has no room for the parser plus background JVM threads.
95+
* `numClients × 2 + 2 > hostCores` — the host is too small for the requested concurrency.
96+
97+
`grep pipes-cpu-sizing` on the parent's logs surfaces all sizing-related output.
98+
99+
== Disabling or overriding
100+
101+
If you want to manage `ActiveProcessorCount` yourself (e.g., to allocate a
102+
different slice based on workload knowledge), just include it in your config:
103+
104+
[source,json]
105+
----
106+
"pipes": {
107+
"numClients": 4,
108+
"forkedJvmArgs": ["-Xmx512m", "-XX:ActiveProcessorCount=4"]
109+
}
110+
----
111+
112+
When Tika sees an explicit `-XX:ActiveProcessorCount` in `forkedJvmArgs`, it
113+
respects your value and skips the auto-injection — the sizing summary will
114+
report `autoCap=user-set in forkedJvmArgs`.
115+
116+
== Container & cgroup behavior
117+
118+
The formula uses `Runtime.availableProcessors()` for the host CPU count,
119+
which on JDK 17+ honors cgroup CPU limits. So in Kubernetes:
120+
121+
* If a pod has `resources.limits.cpu` set, the JVM sees that limit and the
122+
formula sizes accordingly.
123+
* If a pod runs without an explicit `limits.cpu`, the JVM sees the *node's*
124+
full CPU count, which may not match what the pod can actually use. **Always
125+
set explicit CPU limits on pipes pods.**
126+
127+
== Shared-server mode
128+
129+
This document only covers per-client (forked-JVM) mode, which is the
130+
default. In shared-server mode (`useSharedServer=true`) all clients use a
131+
single forked JVM, so the multi-process thread-blowup problem doesn't
132+
apply and the auto-cap is not applied. See
133+
xref:pipes/shared-server-mode.adoc[Shared Server Mode] for that mode's
134+
trade-offs.

‎tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/PerClientServerManager.java‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,17 @@ public class PerClientServerManager implements ServerManager {
5151
private static final Logger LOG = LoggerFactory.getLogger(PerClientServerManager.class);
5252
private static final long WAIT_ON_DESTROY_MS = 10000;
5353
public static final int SOCKET_CONNECT_TIMEOUT_MS = 60000;
54+
/** Cores reserved for the parent JVM when auto-sizing forked JVMs'
55+
* -XX:ActiveProcessorCount. The parent has client-side serialization,
56+
* response deserialization, and heartbeat bookkeeping; if it's CPU-starved
57+
* small operations like socket flush show pathological tail latency. */
58+
private static final int PARENT_RESERVED_CORES = 2;
59+
/** Don't auto-cap below this many CPUs per fork. At cap=1 the fork's only
60+
* CPU is fully consumed by parsing, so its socket-reader thread can't run
61+
* and the parent's writes block on receiver-side back-pressure -- worse
62+
* than no cap at all. This guard matters for small k8s pods where the
63+
* formula could otherwise produce slice=1. */
64+
private static final int MIN_AUTO_CAP_SLICE = 2;
5465

5566
private final PipesConfig pipesConfig;
5667
private final Path tikaConfigPath;
@@ -67,6 +78,62 @@ public PerClientServerManager(PipesConfig pipesConfig, Path tikaConfigPath, int
6778
this.pipesConfig = pipesConfig;
6879
this.tikaConfigPath = tikaConfigPath;
6980
this.clientId = clientId;
81+
// Emit CPU-sizing diagnostics once per PipesParser (only on the first client).
82+
if (clientId == 0) {
83+
logCpuSizing();
84+
}
85+
}
86+
87+
/**
88+
* Emits a one-shot summary of how the auto-cap will behave for this PipesParser,
89+
* plus warnings for clearly-pathological provisioning. Grep for "pipes-cpu-sizing"
90+
* in logs to see the decision the JVM made.
91+
*/
92+
private void logCpuSizing() {
93+
int hostCores = Runtime.getRuntime().availableProcessors();
94+
int numClients = pipesConfig.getNumClients();
95+
boolean userSetCap = pipesConfig.getForkedJvmArgs().stream()
96+
.anyMatch(a -> a.startsWith("-XX:ActiveProcessorCount="));
97+
98+
// Hostile environment: fewer than 2 cores means the parser thread, GC, JIT,
99+
// and protocol heartbeat all share one CPU. Pipes will run but tail latency
100+
// will be poor regardless of numClients.
101+
if (hostCores < 2) {
102+
LOG.warn("pipes-cpu-sizing: hostCores={} is below the practical minimum. " +
103+
"Each fork JVM needs roughly 2 CPUs (1 for parsing, 1 for GC/JIT/" +
104+
"protocol heartbeat); on a single-CPU host these contend with each " +
105+
"other and performance will be poor.", hostCores);
106+
}
107+
108+
// Over-provisioned: numClients packed too tightly given the host's cores.
109+
// Triggers earlier than the slice<MIN guard so the user is warned even
110+
// when they explicitly set -XX:ActiveProcessorCount themselves.
111+
if (numClients > 1 && numClients * MIN_AUTO_CAP_SLICE + PARENT_RESERVED_CORES > hostCores) {
112+
int recommendedMax = Math.max(1,
113+
(hostCores - PARENT_RESERVED_CORES) / MIN_AUTO_CAP_SLICE);
114+
LOG.warn("pipes-cpu-sizing: numClients={} is over-provisioned for {}-core " +
115+
"host. Recommended max for this host: numClients={}. Forks need at " +
116+
"least {} CPUs each plus {} reserved for the parent JVM; otherwise " +
117+
"GC/JIT/protocol threads contend with parser threads across forks.",
118+
numClients, hostCores, recommendedMax,
119+
MIN_AUTO_CAP_SLICE, PARENT_RESERVED_CORES);
120+
}
121+
122+
// Always-on summary so ops can see what was decided. Grep for "pipes-cpu-sizing".
123+
String capDecision;
124+
if (userSetCap) {
125+
capDecision = "user-set in forkedJvmArgs";
126+
} else if (numClients <= 1) {
127+
capDecision = "n/a (single fork; not capped)";
128+
} else {
129+
int budget = Math.max(1, hostCores - PARENT_RESERVED_CORES);
130+
int slice = budget / numClients;
131+
capDecision = (slice >= MIN_AUTO_CAP_SLICE)
132+
? "slice=" + slice
133+
: "skipped (slice<" + MIN_AUTO_CAP_SLICE + ")";
134+
}
135+
LOG.info("pipes-cpu-sizing: hostCores={}, numClients={}, parentReserved={}, " +
136+
"autoCap={}", hostCores, numClients, PARENT_RESERVED_CORES, capDecision);
70137
}
71138

72139
@Override
@@ -305,6 +372,7 @@ private String[] getCommandline() throws IOException {
305372
boolean hasHeadless = false;
306373
boolean hasExitOnOOM = false;
307374
boolean hasLog4j = false;
375+
boolean hasActiveProcessorCount = false;
308376
String origGCString = null;
309377
String newGCLogString = null;
310378

@@ -321,12 +389,45 @@ private String[] getCommandline() throws IOException {
321389
if (arg.startsWith("-Dlog4j.configuration") || arg.startsWith("-Dlog4j2.configuration")) {
322390
hasLog4j = true;
323391
}
392+
if (arg.startsWith("-XX:ActiveProcessorCount=")) {
393+
hasActiveProcessorCount = true;
394+
}
324395
if (arg.startsWith("-Xloggc:")) {
325396
origGCString = arg;
326397
newGCLogString = arg.replace("${pipesClientId}", "id-" + clientId);
327398
}
328399
}
329400

401+
// If the user hasn't explicitly set -XX:ActiveProcessorCount, size each
402+
// forked JVM's view of CPUs to a fair slice of the host. Otherwise each
403+
// JVM defaults its GC, JIT, and common ForkJoinPool to "all cores", which
404+
// means N forked JVMs collectively spawn N x cores GC threads etc. and
405+
// fight each other. We also reserve PARENT_RESERVED_CORES so the parent
406+
// JVM (which serializes requests, deserializes responses, runs heartbeat
407+
// bookkeeping) isn't starved for CPU.
408+
// Skip the auto-cap when the computed slice would drop below
409+
// MIN_AUTO_CAP_SLICE -- below that, the fork can't keep its socket
410+
// reader responsive and back-pressures the parent.
411+
if (!hasActiveProcessorCount && pipesConfig.getNumClients() > 1) {
412+
int hostCores = Runtime.getRuntime().availableProcessors();
413+
int forkBudget = Math.max(1, hostCores - PARENT_RESERVED_CORES);
414+
int slice = forkBudget / pipesConfig.getNumClients();
415+
if (slice >= MIN_AUTO_CAP_SLICE) {
416+
configArgs.add("-XX:ActiveProcessorCount=" + slice);
417+
LOG.debug("clientId={}: auto-injected -XX:ActiveProcessorCount={} " +
418+
"(hostCores={}, parentReserved={}, numClients={})",
419+
clientId, slice, hostCores, PARENT_RESERVED_CORES,
420+
pipesConfig.getNumClients());
421+
} else {
422+
LOG.info("clientId={}: skipping -XX:ActiveProcessorCount auto-cap " +
423+
"(would yield slice={} < MIN_AUTO_CAP_SLICE={}; " +
424+
"hostCores={}, parentReserved={}, numClients={}). " +
425+
"Consider lowering numClients on this host.",
426+
clientId, slice, MIN_AUTO_CAP_SLICE, hostCores,
427+
PARENT_RESERVED_CORES, pipesConfig.getNumClients());
428+
}
429+
}
430+
330431
if (origGCString != null && newGCLogString != null) {
331432
configArgs.remove(origGCString);
332433
configArgs.add(newGCLogString);

0 commit comments

Comments
 (0)