Skip to content

Commit 05c241c

Browse files
committed
add pipeline guide to docs
1 parent 87dd5c7 commit 05c241c

2 files changed

Lines changed: 149 additions & 0 deletions

File tree

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
# Composing Tesseracts into pipelines
2+
3+
A single Tesseract packages one computation. Real work usually involves several: a mesher feeds a solver, an encoder feeds a model, a simulation feeds a post-processor. This page is about the step _after_ you've built your Tesseracts — calling them from your own code and chaining them into a larger workflow.
4+
5+
Tesseract Core is deliberately unopinionated about what you build on top. A project that uses Tesseracts might be a training loop, an optimization routine, a notebook, a web service, or a full application, with as much or as little logic of its own as you like. We won't prescribe that. What follows is the small amount of generic advice that applies regardless: _how to call and chain Tesseracts well_.
6+
7+
The [Design Patterns](design-patterns.md) page covers the complementary question of how to split a workflow into Tesseracts in the first place. Read that for the "how many, how granular" decisions; read this for "now how do I wire them together."
8+
9+
## Choosing how to call a Tesseract
10+
11+
Every Tesseract exposes the same three interfaces: a [CLI, a REST API, and a Python SDK](../using-tesseracts/use.md). For composing Tesseracts into a program, there are two approaches worth knowing, and which one you reach for depends on whether you're working inside an autodiff framework.
12+
13+
### If you're using JAX or PyTorch: use the framework bindings
14+
15+
This is the recommended default when your surrounding code already lives in JAX or PyTorch. **[Tesseract-JAX](https://github.com/pasteurlabs/tesseract-jax)** and **[Tesseract-Torch](https://github.com/pasteurlabs/tesseract-torch)** wrap a Tesseract so it behaves like a native operation in that framework — traceable, and above all _differentiable_, so gradients flow through the containerized computation and into the rest of your program.
16+
17+
Both expose the same one-function interface, `apply_tesseract(tesseract, inputs)`. With JAX:
18+
19+
```python
20+
import jax
21+
import jax.numpy as jnp
22+
from tesseract_core import Tesseract
23+
from tesseract_jax import apply_tesseract
24+
25+
t = Tesseract.from_image("vectoradd_jax")
26+
t.serve()
27+
28+
x = jnp.ones((1000,))
29+
y = jnp.ones((1000,))
30+
31+
def vector_sum(x, y):
32+
res = apply_tesseract(t, {"a": {"v": x}, "b": {"v": y}})
33+
return res["vector_add"]["result"].sum()
34+
35+
vector_sum(x, y) # call it like any JAX function
36+
jax.jit(vector_sum)(x, y) # ... jit it
37+
jax.grad(vector_sum)(x, y) # ... and differentiate through it
38+
39+
t.teardown() # stop the container when you're done
40+
```
41+
42+
The PyTorch equivalent integrates with autograd instead:
43+
44+
```python
45+
import torch
46+
from tesseract_core import Tesseract
47+
from tesseract_torch import apply_tesseract
48+
49+
t = Tesseract.from_image("vectoradd_torch")
50+
t.serve()
51+
52+
x = torch.ones(1000, requires_grad=True)
53+
y = torch.ones(1000)
54+
55+
res = apply_tesseract(t, {"a": {"v": x}, "b": {"v": y}})
56+
loss = res["vector_add"]["result"].sum()
57+
loss.backward() # gradients propagate back through the Tesseract
58+
print(x.grad)
59+
60+
t.teardown() # stop the container when you're done
61+
```
62+
63+
```{tip}
64+
These bindings are what makes Tesseracts *differentiable software components*. If your workflow is an optimization, calibration, or training problem, reaching for Tesseract-JAX or Tesseract-Torch usually means you get end-to-end gradients for free. See the [Differentiable Programming guide](../misc/differentiable-programming.md).
65+
```
66+
67+
### Otherwise: use the Python SDK
68+
69+
When you're not inside JAX or PyTorch (a plain Python script, a service, a batch job), the [`Tesseract`](#tesseract_core.Tesseract) SDK class is the general-purpose way to invoke a Tesseract:
70+
71+
```python
72+
import numpy as np
73+
from tesseract_core import Tesseract
74+
75+
with Tesseract.from_image("scaler") as scaler:
76+
result = scaler.apply({"vector": np.array([1.0, 2.0, 3.0]), "scale_factor": 2.0})
77+
78+
print(result["scaled_vector"]) # -> [2. 4. 6.]
79+
```
80+
81+
[`Tesseract.from_image`](#tesseract_core.Tesseract.from_image) references a built image by name; the `with` block starts the container and tears it down on exit. `apply` takes a dict of inputs and returns a dict of outputs — NumPy in, NumPy out, no manual serialization.
82+
83+
```{note}
84+
To call a Tesseract that already runs elsewhere (a shared service, a GPU node, a remote deployment), use [`Tesseract.from_url(...)`](#tesseract_core.Tesseract.from_url) instead of `from_image`. The calling code is otherwise identical, so a workflow developed locally moves to distributed execution without a rewrite.
85+
```
86+
87+
## Chaining Tesseracts
88+
89+
Chaining is deliberately unremarkable in all three cases: a Tesseract's outputs are ordinary values (a dict of arrays), so you feed them into the next call like any other Python data. There's no special pipeline object to learn.
90+
91+
With the SDK:
92+
93+
```python
94+
with Tesseract.from_image("scaler") as scaler, \
95+
Tesseract.from_image("normalizer") as normalizer:
96+
scaled = scaler.apply({"vector": vector, "scale_factor": 2.0})
97+
normalized = normalizer.apply({"vector": scaled["scaled_vector"]})
98+
```
99+
100+
And the same shape inside a differentiable JAX program — call `apply_tesseract` for each step and let the framework thread gradients through the whole chain:
101+
102+
```python
103+
def pipeline(vector):
104+
scaled = apply_tesseract(scaler, {"vector": vector, "scale_factor": 2.0})
105+
normalized = apply_tesseract(normalizer, {"vector": scaled["scaled_vector"]})
106+
return normalized["normalized_vector"]
107+
108+
jax.grad(lambda v: pipeline(v).sum())(vector) # gradient through both Tesseracts
109+
```
110+
111+
Because chaining is just data flow, ordinary control flow works too. Put a Tesseract call in a loop, behind a conditional, inside `jax.lax.scan`, or wherever your program needs it.
112+
113+
When a chain misbehaves, test each Tesseract in isolation first. A component you've verified on its own against a known input/output pair is a fixed point you can trust while debugging the workflow around it.
114+
115+
The one thing that makes or breaks chaining is at the _interface_, not the call site: a step composes cleanly with the next only if its output fields line up with the downstream input schema. Design for that.
116+
117+
```{seealso}
118+
[Designing good interfaces](design-patterns.md#designing-good-interfaces) — matching one Tesseract's `OutputSchema` to the next's `InputSchema` is what keeps chains readable. Design interfaces with the downstream consumer in mind.
119+
```
120+
121+
## Building a multi-Tesseract project
122+
123+
Once a project grows past a couple of Tesseracts, some structure pays off: a consistent place for each component, a way to share code between them, and a build-and-test loop that covers the whole set. Rather than assemble this by hand, the [`cookiecutter-tesseract`](https://github.com/pasteurlabs/cookiecutter-tesseract) template generates a ready-made project with all of it wired up:
124+
125+
```bash
126+
$ pip install cookiecutter
127+
$ cookiecutter github:pasteurlabs/cookiecutter-tesseract
128+
```
129+
130+
The generated project gives you an opinionated, working structure for a multi-Tesseract codebase, including:
131+
132+
- **A two-layer project layout** that separates the individual Tesseracts (each independently built and tested) from the application code that uses them.
133+
- **A one-command workflow** for scaffolding, building, and testing components, so you don't call the underlying tooling by hand.
134+
- **A shared-code package** that every component can depend on, so common helpers live in one place instead of being copied into each Tesseract.
135+
- **Per-component test fixtures** for checking each Tesseract in isolation, plus a place for tests of the application layer.
136+
- **Pre-build hooks** for components that need setup before their container builds (fetching weights, compiling an extension).
137+
- **Continuous integration** that builds and tests every component across supported Python versions, catching a broken interface the moment it stops matching its consumer.
138+
139+
Take the parts that fit your project and leave the rest. For anything beyond a handful of Tesseracts, starting from the template usually beats reinventing this plumbing.
140+
141+
## What's next
142+
143+
- [Design Patterns](design-patterns.md) — how to split a workflow into Tesseracts and design their interfaces.
144+
- [Interacting with Tesseracts](../using-tesseracts/use.md) — the full SDK, CLI, and REST interfaces.
145+
- [Differentiable Programming](../misc/differentiable-programming.md) — propagating gradients through a composed, multi-Tesseract program.
146+
- [Tesseract-JAX](https://github.com/pasteurlabs/tesseract-jax) and [Tesseract-Torch](https://github.com/pasteurlabs/tesseract-torch) — the framework bindings.
147+
- [Performance](../misc/performance.md) — minimizing container and data-transfer overhead in chained workflows.
148+
- Questions? Ask on the [Tesseract User Forums](https://si-tesseract.discourse.group/).

docs/content/introduction/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,7 @@ Changelog <https://github.com/pasteurlabs/tesseract-core/releases>
8585
8686
../creating-tesseracts/create.md
8787
../creating-tesseracts/design-patterns.md
88+
../creating-tesseracts/pipelines.md
8889
../creating-tesseracts/advanced.md
8990
../creating-tesseracts/llm-assistance.md
9091
../creating-tesseracts/deploy.md

0 commit comments

Comments
 (0)