update-4x-docs (#2802)

Author: tballison

docs/.gitignore

@@ -0,0 +1,3 @@
+# Generated by maven-antrun-plugin from antora-playbook.yml at build time.
+# Contains the current git-commit stamp injected for the docs home page.
+antora-playbook-stamped.yml

docs/build-docs.sh

@@ -1 +1 @@
-../../../../tika-parsers/tika-parsers-standard/tika-parsers-standard-package/src/test/resources/config-examples/migration-full-example.json
+../../../../tika-parsers/tika-parsers-standard/tika-parsers-standard-integration-tests/src/test/resources/config-examples/migration-full-example.json

docs/modules/ROOT/examples/migration-full-example.json

@@ -1 +1 @@
-../../../../tika-parsers/tika-parsers-standard/tika-parsers-standard-package/src/test/resources/config-examples/pdf-parser-basic.json
+../../../../tika-parsers/tika-parsers-standard/tika-parsers-standard-integration-tests/src/test/resources/config-examples/pdf-parser-basic.json

docs/modules/ROOT/examples/pdf-parser-basic.json

@@ -1 +1 @@
-../../../../tika-parsers/tika-parsers-standard/tika-parsers-standard-package/src/test/resources/config-examples/pdf-parser-full.json
+../../../../tika-parsers/tika-parsers-standard/tika-parsers-standard-integration-tests/src/test/resources/config-examples/pdf-parser-full.json

docs/modules/ROOT/examples/pdf-parser-full.json

@@ -1 +1 @@
-../../../../tika-parsers/tika-parsers-standard/tika-parsers-standard-package/src/test/resources/config-examples/tesseract-basic.json
+../../../../tika-parsers/tika-parsers-standard/tika-parsers-standard-integration-tests/src/test/resources/config-examples/tesseract-basic.json

docs/modules/ROOT/examples/tesseract-basic.json

@@ -1 +1 @@
-../../../../tika-parsers/tika-parsers-standard/tika-parsers-standard-package/src/test/resources/config-examples/tesseract-full.json
+../../../../tika-parsers/tika-parsers-standard/tika-parsers-standard-integration-tests/src/test/resources/config-examples/tesseract-full.json

docs/modules/ROOT/examples/tesseract-full.json

@@ -46,7 +46,6 @@
 ** xref:migration-to-4x/chunk-strategies.adoc[Chunk Strategies]
 ** xref:migration-to-4x/inference-handler-requirements.adoc[Inference Handler Requirements]
 * xref:advanced/index.adoc[Advanced]
-** xref:advanced/charset-detection-design.adoc[Charset Detection Pipeline]
 ** xref:advanced/language-detection.adoc[Language Detection]
 ** xref:advanced/language-detection-build.adoc[Building the Language Detector]
 ** xref:advanced/junk-detection.adoc[Text Quality Scoring (Junk Detection)]

docs/modules/ROOT/nav.adoc

@@ -510,4 +510,4 @@ with feature hashing.  The move to NB was driven by:
 == See also
 
 * Configuration: xref:../configuration/encoding-detectors.adoc[Configuring Encoding Detectors]
-* Language detection: xref:language-detection.adoc[Language Detection]
+* Language detection: xref:advanced/language-detection.adoc[Language Detection]

docs/modules/ROOT/pages/advanced/charset-detection-design.adoc

@@ -58,11 +58,11 @@ Source data lives in one directory per language (ISO 639 code), each containing
 up to two files:
 
 `sentences_wikipedia.txt`::
-  Line-numbered Wikipedia sentences: `{lineNum}{TAB}{text}`.
+  Line-numbered Wikipedia sentences: `\{lineNum}\{TAB}\{text}`.
   One sentence per line.
 
 `sentences_madlad.txt`::
-  Line-numbered MADLAD-400 documents: `{lineNum}{TAB}{text}`.
+  Line-numbered MADLAD-400 documents: `\{lineNum}\{TAB}\{text}`.
   Documents contain literal two-character `\n` escape sequences as
   sub-sentence separators.  The tool splits on these before processing.
 
@@ -108,16 +108,16 @@ per script:
 |===
 | File | Split | Purpose
 
-| `{script}.train.gz`
+| `\{script}.train.gz`
 | 80%
 | Bigram count accumulation in `TrainJunkModel`.
 
-| `{script}.dev.gz`
+| `\{script}.dev.gz`
 | 10%
 | Calibration (mu/sigma estimation) in `TrainJunkModel`.
   Also used for iterative evaluation during development.
 
-| `{script}.test.gz`
+| `\{script}.test.gz`
 | 10%
 | **Held out completely.**  Use only for final reported evaluation numbers.
   Never use to make model or threshold decisions.
@@ -146,7 +146,7 @@ Key options:
 
 | `--output-dir`
 | `~/datasets/madlad/junkdetect`
-| Where to write `{script}.train.gz`, `.dev.gz`, `.test.gz`, and `manifest.tsv`.
+| Where to write `\{script}.train.gz`, `.dev.gz`, `.test.gz`, and `manifest.tsv`.
 
 | `--total-budget-bytes`
 | `50000000`
@@ -178,7 +178,7 @@ then calibrates z-score statistics from the `.dev.gz` file.
 
 === Bigram table training
 
-[source]
+[source,subs="-attributes"]
 ----
 for each sentence in {script}.train.gz:
     utf8 = sentence.getBytes(UTF-8)
@@ -197,7 +197,7 @@ a small but nonzero probability for novel byte sequences.
 
 === Calibration
 
-For each sentence in `{script}.dev.gz`:
+For each sentence in `\{script}.dev.gz`:
 
 [source]
 ----