Skip to content

Commit 1ae3b4c

Browse files
dmkedmke
committed
docs: start on documentation
[skip ci]
1 parent d9aa4f2 commit 1ae3b4c

11 files changed

+582
-576
lines changed

‎README.md

+5-576
Large diffs are not rendered by default.

‎docs/cli-options.md

+60
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# CLI Options
2+
3+
Calling texd with options works in any [operation mode](./operation-modes.md);
4+
these commands are equivalent:
5+
6+
```console
7+
$ texd -h
8+
$ texd texlive/texlive:latest -h
9+
$ docker run --rm -t digineode/texd:latest -h
10+
```
11+
12+
- `--help`, `-h`
13+
14+
Prints a short option listing and exits.
15+
16+
- `--version`, `-v`
17+
18+
Prints version information and exits.
19+
20+
- `--listen-address=ADDR`, `-b ADDR` (Default: `:2201`)
21+
22+
Specifies host address (optional) and port number for the HTTP API to bind to. Valid values are,
23+
among others:
24+
25+
- `:2201` (bind to all addresses on port 2201)
26+
- `localhost:2201` (bind only to localhost on port 2201)
27+
- `[fe80::dead:c0ff:fe42:beef%eth0]:2201` (bind to a link-local IPv6 address on a specific
28+
interface)
29+
30+
- `--tex-engine=ENGINE`, `-X ENGINE` (Default: `xelatex`)
31+
32+
TeX engine used to compile documents. Can be overridden on a per-request basis (see HTTP API
33+
below). Supported engines are `xelatex`, `lualatex`, and `pdflatex`.
34+
35+
- `--compile-timeout=DURATION`, `-t DURATION` (Default: `1m`)
36+
37+
Maximum duration for a document rendering process before it is killed by texd. The value must be
38+
acceptable by Go's `ParseDuruation` function.
39+
40+
- `--parallel-jobs=NUM`, `-P NUM` (Default: number of cores)
41+
42+
Concurrency level. PDF rendering is inherently single threaded, so limiting the document
43+
processing to the number of cores is a good start.
44+
45+
- `--queue-wait=DURATION`, `-w DURATION` (Default: `10s`)
46+
47+
Time to wait in queue before aborting. When <= 0, clients will immediately receive a "full queue"
48+
response.
49+
50+
- `--job-directory=PATH`, `-D PATH` (Default: OS temp directory)
51+
52+
Place to put job sub directories in. The path must exist and it must be writable.
53+
54+
- `--pull` (Default: omitted)
55+
56+
Always pulls Docker images. By default, images are only pulled when they don't exist locally.
57+
58+
This has no effect when no image tags are given to the command line.
59+
60+
> Note: This option listing might be outdated. Run `texd --help` to get the up-to-date listing.

‎docs/future.md

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Future
2+
3+
One wishlist's item is asynchronous rendering: Consider rendering monthly invoices on the first
4+
of each month; depending on the amount of customers/contracts/invoice positions, this can easily
5+
mean you need to render a few thousand PDF documents.
6+
7+
Usually, the PDF generation is not time critical, i.e. they should finish in a reasonable amount of
8+
time (say, within the next 6h to ensure timely delivery to the customer via email). For this to
9+
work, the client could provide a callback URL to which texd sends the PDF via HTTP POST when
10+
the rendering is finished.
11+
12+
Of course, this will also increase complexity on both sides: The client must be network-reachable
13+
itself, an keep track of rendering request in order to associate the PDF to the correct invoice;
14+
texd on the other hand would need a priority queue (processing async documents only if no sync
15+
documents are enqueued), and it would need to store the callback URL somewhere.

‎docs/history.md

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# History
2+
3+
texd came to life because I've build dozens of Rails applications, which all needed to build PDF
4+
documents in one form or another (from recipies, to invoices, order confirmations, reports and
5+
technical documentation). Each server basically needed a local TeX installation (weighing in at
6+
several 100 MB, up to several GB). Compiling many LaTeX documents also became a bottleneck for
7+
applications running on otherwise modest hardware (or cloud VMs), as this process is also
8+
computationally expensive.
9+
10+
Over time I've considered using alternatives for PDF generation (Prawn, HexaPDF, gofpdf, SILE, iText
11+
PDF, to name but a few), and found that the quality of the rendered PDF is far inferior to the ones
12+
generated by LaTeX. Other times, the licensing costs are astronomical, or the library doesn't
13+
support some layouting feature, or the library in an early alpha stage or already abandoned...
14+
15+
I'll admit that writing TeX templates for commercial settings is a special kind of pain-inducing
16+
form of art. But looking back at using LaTeX for now over a decade, I still feel it's worth it.

‎docs/http-api/metrics.md

+29
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Metrics
2+
3+
For monitoring, texd provides a Prometheus endpoint at `/metrics`:
4+
5+
```console
6+
$ curl -i http://localhost:2201/metrics
7+
Content-Type: text/plain; version=0.0.4; charset=utf-8
8+
9+
...
10+
```
11+
12+
The metrics include Go runtime information, as well as texd specific metrics:
13+
14+
| Metric name | Type | Description |
15+
|:------------|:-----|:------------|
16+
| `texd_processed_total{status="success"}` | counter | Number of documents processed. |
17+
| `texd_processed_total{status="failure"}` | counter | Number of rendering errors, including timeouts. |
18+
| `texd_processed_total{status="rejected"}` | counter | Number of rejected requests, due to full job queue. |
19+
| `texd_processed_total{status="aborted"}` | counter | Number of aborted requests, usually due to timeouts. |
20+
| `texd_processing_duration_seconds` | histogram | Overview of processing time per document. |
21+
| `texd_input_file_size_bytes{type=?}` | histogram | Overview of input file sizes. Type is either "tex" (for .tex, .cls, .sty, and similar files), "asset" (for images and fonts), "data" (for CSV files), or "other" (for unknown files) |
22+
| `texd_output_file_size_bytes` | histogram | Overview of output file sizes. |
23+
| `texd_job_queue_length` | gauge | Length of rendering queue, i.e. how many documents are waiting for processing. |
24+
| `texd_job_queue_usage_ratio` | gauge | Queue capacity indicator (0.0 = empty, 1.0 = full). |
25+
| `texd_info{version="0.0.0", mode="local", ...}` | constant | Various version and configuration information. |
26+
27+
28+
Metrics related to processing also have an `engine=?` label indicating the TeX engine ("xelatex",
29+
"lualatex", or "pdflatex"), and an `image=?` label indicating the Docker image.

‎docs/http-api/render.md

+199
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,199 @@
1+
# Render a document
2+
3+
To create a PDF document from an input `.tex` file, send a HTTP POST to the `/render` endpoint.
4+
You may encode the payload as `multipart/form-data` or `application/x-www-form-encoded`, however
5+
the latter is not recommended.
6+
7+
Assuming, you have a `input.tex` in the current directory, you can issue the following command
8+
to send that file to your texd instance, and save the result in a file named `output.pdf`:
9+
10+
```console
11+
$ curl -X POST \
12+
-F "input.tex=<input.tex" \
13+
-o "output.pdf" \
14+
"http://localhost:2201/render"
15+
```
16+
17+
You can send multiple files (even in sub directories) as well:
18+
19+
```console
20+
$ curl -X POST \
21+
-F "cv.tex=<cv.tex" \
22+
-F "chapters/introduction.tex=<chapters/introduction.tex" \
23+
-F "logo.pdf=<logo.pdf" \
24+
-o "vita.pdf" \
25+
"http://localhost:2201/render?input=cv.tex"
26+
```
27+
28+
When sending multiple files, you should specify which one is the main input file (usually the one
29+
containing `\documentclass`), using the `input=` query parameter. If you omit this parameter, texd
30+
will try to guess the input file.
31+
32+
Please note that file names will be normalized, and files pointing outside the root directory
33+
will be discarded entirely (i.e. `../input.tex` is NOT a valid file name). You can't do this:
34+
35+
```console
36+
$ curl -X POST \
37+
-F "../input.tex=<input.tex" \
38+
-o "output.pdf" \
39+
"http://localhost:2201/render"
40+
```
41+
42+
However, this is perfectly fine:
43+
44+
```console
45+
$ curl -X POST \
46+
-F "input.tex=<../input.tex" \
47+
-o "output.pdf" \
48+
"http://localhost:2201/render"
49+
```
50+
51+
<details><summary>Guessing the input file (click to show details)</summary>
52+
53+
- only filenames starting with alphanumeric character and ending in `.tex` are considered
54+
(`foo.tex`, `00-intro.tex` will be considered, but not `_appendix.tex`, `figure.png`)
55+
- files in sub directories are ignored (e.g. `chapters/a.tex`)
56+
- if only one file in the root directory remains, it is taken as main input
57+
- otherwise search for a file containing a line starting with:
58+
- either `%!texd` at the beginning of the file
59+
- or `\documentclass` somewhere in the first KiB
60+
- if no match, consider (in order):
61+
- `input.tex`
62+
- `main.tex`
63+
- `document.tex`
64+
65+
</details>
66+
67+
If no main input file can be determined, texd will abort with an error.
68+
69+
## URL Parameters
70+
71+
- `input=<filename>` - instructs texd to skip guessing main input file and use the specified one.
72+
The filename must be present in the body.
73+
74+
- `engine=<value>` - specifies which TeX engine to run. Supported engines are:
75+
76+
- `xelatex` (default)
77+
- `lualatex`
78+
- `pdflatex`
79+
80+
Note that the default can be changed with a CLI option (e.g. `--tex-engine=lualatex`).
81+
82+
- `image=<imagename>` - selects Docker image for document processing.
83+
84+
This is only available in *ephemeral container* mode. The image name must match the ones listed
85+
in the texd command invocation, i.e. you can't select arbitrary images.
86+
87+
If you provide an unknown image name, you will receive a 404 Not Found response. In *local* and
88+
*CI service* mode, this parameter only logged, but will otherwise be ignored.
89+
90+
- `errors=<detail level>` - tries to retrieve the compilation log, in case of compilation errors.
91+
Acceptable detail levels are:
92+
93+
- *empty* (or `errors` completely absent), to return a JSON description (default)
94+
- `condensed`, to return only the TeX error message from the log file
95+
- `full`, to return the full log file as `text/plain` response
96+
97+
The "condensed" form extracts only the lines from the error log which start with a `!`. Due to
98+
the way TeX works, these lines may not paint the full picture, as TeX's log lines generally don't
99+
exceed a certain line length, and wrapped lines won't get another `!` prefix.
100+
101+
Note that this parameter changes the response content to a plain text file if you select `full`
102+
or `condensed`, and not a JSON response as in all other cases.
103+
104+
## Successful response
105+
106+
If compilation succeeds, you'll receive a status 200 OK, with content type `application/pdf`, and
107+
the PDF file as response body.
108+
109+
```http
110+
HTTP/1.1 200 OK
111+
Content-Type: application/pdf
112+
Content-Length: 1234
113+
114+
%PDF/1.5...
115+
```
116+
117+
## Failure responses
118+
119+
If the request was accepted, but could not complete due to errors, you will by default receive a 422
120+
Unprocessable Entity response with content type `application/json`, and an error description in
121+
JSON format:
122+
123+
```http
124+
HTTP/1.1 422 Unprocessable Entity
125+
Content-Type: application/json
126+
Content-Length: 154
127+
128+
{
129+
"error": "latexmk call failed with status 1",
130+
"category": "compilation",
131+
"output": "[truncated output log]"
132+
}
133+
```
134+
135+
The fields `error` and `category` represent a short error description and an error category,
136+
respectively.
137+
138+
Possible, known error categories are currently:
139+
140+
- *input* - one or more files are invalid (e.g. file was discarded after path normalization),
141+
or the main input file could not be determined.
142+
143+
- *compilation* - `latexmk` exited with an error (likely due to invalid or missing input files).
144+
145+
- *queue* - texd won't accept new render jobs, if its internal queue is at capacity. In this case
146+
wait for a few moments to give texd a chance to catch up and then try again.
147+
148+
- *reference* - texd could not find the provided reference store entries. The missing references
149+
are listed in the response; you need to repeat the request with those files included.
150+
151+
Additional fields, like `log` for compilation failures, might be present.
152+
153+
> Note: The JSON response is pretty-printed only for this README. Expect the actual response to
154+
> be minified.
155+
156+
If you set `errors=full`, you may receive a plain text file with the compilation log:
157+
158+
<details><summary>Show response (click to open)</summary>
159+
160+
```http
161+
HTTP/1.1 422 Unprocessable Entity
162+
Content-Type: text/plain
163+
Content-Length: 3156
164+
165+
This is XeTeX, Version 3.141592653-2.6-0.999993 (TeX Live 2021) (preloaded format=xelatex 2022.3.6) 12 MAR 2022 13:57
166+
entering extended mode
167+
restricted \write18 enabled.
168+
%&-line parsing enabled.
169+
... ommitting some lines ...
170+
! LaTeX Error: File `missing.tex' not found.
171+
172+
Type X to quit or <RETURN> to proceed,
173+
or enter new name. (Default extension: tex)
174+
175+
Enter file name:
176+
! Emergency stop.
177+
<read *>
178+
179+
l.3 \input{missing.tex}
180+
^^M
181+
*** (cannot \read from terminal in nonstop modes)
182+
```
183+
184+
</details>
185+
186+
For `errors=condensed`, you'll only receive the lines starting with `!` (with this prefix removed):
187+
188+
<details><summary>Show response (click to open)</summary>
189+
190+
```http
191+
HTTP/1.1 422 Unprocessable Entity
192+
Content-Type: text/plain
193+
Content-Length: 59
194+
195+
LaTeX Error: File `missing.tex' not found.
196+
Emergency stop.
197+
```
198+
199+
</details>

‎docs/http-api/status.md

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Status and Configuration
2+
3+
texd has a number of runtime configuration knobs and internal state variables, which may or may not
4+
of interest for API consumers. To receive a current snapshot, query `/status`:
5+
6+
```console
7+
$ curl -i http://localhost:2201/status
8+
HTTP/1.1 200 OK
9+
Content-Type: application/json; charset=utf-8
10+
Content-Length: 287
11+
12+
{
13+
"version": "0.0.0",
14+
"mode": "container",
15+
"images": ["texlive/texlive:latest"],
16+
"timeout": 60,
17+
"engines": ["xelatex","pdflatex","lualatex"],
18+
"default_engine": "xelatex",
19+
"queue": {
20+
"length": 0,
21+
"capacity": 16
22+
}
23+
}
24+
```

‎docs/http-api/web-ui.md

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Simple Web UI
2+
3+
You can try compiling TeX documents directly in your browser: Visit http://localhost:2201, and
4+
you'll be greeted with a very basic, but functional UI.
5+
6+
Please note, that this UI is *not* built to work in every browser. It intentionally does not
7+
use fancy build tools. It's just a simple HTML file, built by hand, using Bootstrap 5 for
8+
aesthetics and Vue 3 for interaction. Both Bootstrap and Vue are bundled with texd, so you won't
9+
need internet access for this to work.
10+
11+
If your browser does not support modern features like ES2022 proxies, `Object.entries`, `fetch`,
12+
and `<object type="application/pdf" />` elements, you're out of luck. (Maybe upgrade your browser?)
13+
Anyway, consider the UI only as demonstrator for the API.

‎docs/index.md

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# texd documentation
2+
3+
1. [Operation Modes](./operation-modes.md)
4+
2. [CLI Options](./cli-options.md)
5+
3. HTTP API
6+
1. [Render a document](./http-api/render.md)
7+
2. [Status and Configuration](./http-api/status.md)
8+
3. [Metrics](./http-api/metrics.md)
9+
4. [Simple Web UI](./http-api/web-ui.md)
10+
4. [Reference Store](./reference-store.md)
11+
5. [History](./history.md)
12+
6. [Future](./future.md)

0 commit comments

Comments
 (0)