doc: documentation mega issue sprint

AGENTS.md

@@ -0,0 +1,28 @@
+# Notes for AI Agents
+
+This file contains counter-intuitive aspects of the Tesseract codebase that AI agents should know.
+
+## Environment setup
+
+- **Use `uv` for dependency management.** Install with `uv pip install -e .[dev]`. This is faster and more reliable than plain pip.
+- **Run `pre-commit install`** after cloning to set up git hooks.
+
+## Testing
+
+- **Prefer end-to-end tests over unit tests.** Tests that build and run real Tesseracts catch more bugs than mocked unit tests.
+- **Avoid mocks.** If you need complex mocking, write an end-to-end test instead.
+- **Don't test implementation details.** Tests should verify behavior, not internal structure.
+- **Be mindful of slow tests.** End-to-end tests are slow. Check if an existing test can be extended before adding a new one.
+- **Don't add mocks for Docker.** Tests that need Docker should be marked as end-to-end tests and skipped in fast test runs.
+- **Rarely test exceptions.** Only test exception handling when control flow is complex or the error message is critical for UX. Don't write tests that just verify an exception is raised.
+- **Never skip or disable tests without asking.** If a test is failing and you want to skip it, ask the user first. Don't add `@pytest.skip`, `@pytest.mark.xfail`, or comment out tests without explicit approval.
+
+## Code style
+
+- **Follow existing patterns.** Look at similar code in the codebase and match its style. Don't introduce new patterns without good reason.
+- **Use pre-commit.** Run `pre-commit run --all-files` before committing. Hooks include Ruff for linting/formatting.
+- **Follow conventional commits.** PR titles must follow the format: `type[(scope)]: description` (e.g., `feat(sdk): add new feature`).
+
+## Architecture
+
+- **The runtime is separate from the CLI.** `tesseract_core.runtime` runs inside containers; `tesseract_core.sdk` and CLI run on the host.

CONTRIBUTING.md

@@ -88,6 +88,50 @@ $ pytest --skip-endtoend
 $ pytest --always-run-endtoend tests/endtoend_tests
 ```
 
+#### Testing philosophy
+
+We follow these principles when writing tests:
+
+- **Prefer end-to-end tests over unit tests** — Tests that exercise real
+  Tesseract builds and invocations catch more bugs than isolated unit tests.
+  When in doubt, write an end-to-end test.
+
+- **Avoid mocks where feasible** — Mocks can hide real integration issues.
+  If a test requires complex mocking to work, consider whether an end-to-end
+  test would be more valuable.
+
+- **Don't test implementation details** — Tests should verify behavior, not
+  internal structure. If refactoring breaks your test but not the actual
+  functionality, the test was too tightly coupled.
+
+- **Be mindful of slow tests** — End-to-end tests that build Tesseracts are
+  slow. Before adding a new one, check if an existing test can be extended,
+  or if a faster unit test would suffice for your specific case.
+
+### Linting and code quality
+
+We use [pre-commit](https://pre-commit.com/) to run linters and formatters
+automatically before each commit. The hooks are configured in
+`.pre-commit-config.yaml` and include:
+
+- **Ruff** — Fast Python linter and formatter (replaces flake8, isort, black)
+- **Various file checks** — Trailing whitespace, YAML validation, etc.
+
+To run all pre-commit hooks manually on all files:
+
+```console
+$ pre-commit run --all-files
+```
+
+To run a specific hook:
+
+```console
+$ pre-commit run ruff --all-files
+```
+
+Pre-commit also handles automatic dependency updates via Dependabot-style
+version bumps. These are configured to run periodically in CI.
+
 ### GitHub workflow
 
 This project uses Git for version control and follows a GitHub workflow. To

README.md

@@ -3,73 +3,106 @@
   <img alt="" src="https://github.com/pasteurlabs/tesseract-core/blob/main/docs/static/logo-light.png" width="128" align="right">
 </picture>
 
-### Tesseract Core
+### Tesseract
 
-Universal, autodiff-native software components for Simulation Intelligence. :package:
+Universal, autodiff-native software components for Simulation Intelligence
 
 [Read the docs](https://docs.pasteurlabs.ai/projects/tesseract-core/latest/) |
 [Report an issue](https://github.com/pasteurlabs/tesseract-core/issues) |
-[Talk to the community](https://si-tesseract.discourse.group/) |
+[Community forum](https://si-tesseract.discourse.group/) |
 [Contribute](https://github.com/pasteurlabs/tesseract-core/blob/main/CONTRIBUTING.md)
 
 ---
 
 [![DOI](https://joss.theoj.org/papers/10.21105/joss.08385/status.svg)](https://doi.org/10.21105/joss.08385)
+[![SciPy](https://img.shields.io/badge/SciPy-2025-blue)](https://proceedings.scipy.org/articles/kvfm5762)
 
-**Tesseract Core** bundles:
+## The problem
 
-1. Tools to define, create, and run Tesseracts, via the `tesseract` CLI and `tesseract_core` Python API.
-2. The Tesseract Runtime, a lightweight, high-performance execution environment for Tesseracts.
+Real-world scientific workflows span multiple tools, languages, and computing environments. You might have a mesh generator in C++, a solver in Julia, and post-processing in Python. Getting these to work together is painful. Getting gradients to flow through them for optimization? Nearly impossible.
 
-## What is a Tesseract?
+Existing autodiff frameworks work great within a single codebase, but fall short when your pipeline crosses framework boundaries or includes legacy/commercial tools.
 
-Tesseracts are components that expose experimental, research-grade software to the world. They are self-contained, self-documenting, and self-executing, via command line and HTTP. They are designed to be easy to create, easy to use, and easy to share, including in a production environment. This repository contains all you need to define your own and execute them.
+## The solution
 
-Tesseracts provide built-in support for [differentiable programming](https://docs.pasteurlabs.ai/projects/tesseract-core/latest/content/introduction/differentiable-programming.html) by propagating gradient information at the level of individual components, making it easy to build complex, diverse software pipelines that can be optimized end-to-end.
+Tesseract packages scientific software into **self-contained, portable components** that:
+
+- **Run anywhere** — Local machines, cloud, HPC clusters. Same container, same results.
+- **Expose clean interfaces** — CLI, REST API, and Python SDK. No more deciphering undocumented scripts.
+- **Propagate gradients** — Each component can expose derivatives, enabling end-to-end optimization across heterogeneous pipelines.
+- **Self-document** — Schemas, types, and API docs are generated automatically.
+
+## Who is this for?
+
+- **Researchers** building differentiable physics simulations or probabilistic models who need to combine tools from different ecosystems
+- **R&D engineers** packaging research code for use by others, without spending weeks on DevOps
+- **Platform engineers** deploying scientific workloads at scale with consistent interfaces and dependency isolation
+
+## Example: Shape optimization across tools
+
+The [rocket fin optimization case study](https://si-tesseract.discourse.group/t/parametric-shape-optimization-of-rocket-fins-with-ansys-spaceclaim-pyansys-and-tesseract/109) combines three Tesseracts:
+
+```
+[SpaceClaim geometry] → [Mesh + SDF] → [PyMAPDL FEA solver]
+         ↑                                      |
+         └──────── gradients flow back ─────────┘
+```
+
+Each component uses a different differentiation strategy (analytic adjoints, finite differences, JAX autodiff), yet they compose into a single optimizable pipeline. Result: 24% improvement in structural stiffness.
 
 ## Quick start
 
 > [!NOTE]
-> Before proceeding, make sure you have a [working installation of Docker](https://docs.docker.com/engine/install/) and a modern Python installation (Python 3.10+); if you prefer Docker Desktop for your platform, see [our extended installation instructions](https://docs.pasteurlabs.ai/projects/tesseract-core/latest/content/introduction/installation.html#basic-installation).
-
-1. Install Tesseract Core:
+> Requires [Docker](https://docs.docker.com/engine/install/) and Python 3.10+.
 
-   ```bash
-   $ pip install tesseract-core
-   ```
+```bash
+$ pip install tesseract-core
 
-2. Build an example Tesseract:
+# Clone and build an example
+$ git clone https://github.com/pasteurlabs/tesseract-core
+$ tesseract build tesseract-core/examples/vectoradd
 
-   ```bash
-   $ git clone https://github.com/pasteurlabs/tesseract-core
-   $ tesseract build tesseract-core/examples/vectoradd
-   ```
+# Run it
+$ tesseract run vectoradd apply '{"inputs": {"a": [1, 2], "b": [3, 4]}}'
+# → {"result": [4.0, 6.0], ...}
 
-3. Display its API documentation:
+# Compute the Jacobian
+$ tesseract run vectoradd jacobian '{"inputs": {"a": [1, 2], "b": [3, 4]}, "jac_inputs": ["a"], "jac_outputs": ["result"]}'
 
-   ```bash
-   $ tesseract apidoc vectoradd
-   ```
+# See auto-generated API docs
+$ tesseract apidoc vectoradd
+```
 
 <p align="center">
 <img src="https://github.com/pasteurlabs/tesseract-core/blob/main/docs/img/apidoc-screenshot.png" width="600">
 </p>
 
-4. Run the Tesseract:
+## Core features
+
+| Feature               | Description                                                                   |
+| --------------------- | ----------------------------------------------------------------------------- |
+| **Containerized**     | Docker-based packaging ensures reproducibility and dependency isolation       |
+| **Multi-interface**   | CLI, REST API, and Python SDK for the same component                          |
+| **Differentiable**    | First-class support for Jacobians, JVPs, and VJPs across component boundaries |
+| **Schema-validated**  | Pydantic models define explicit input/output contracts                        |
+| **Language-agnostic** | Wrap Python, Julia, C++, or any executable behind a thin Python API           |
+
+## Ecosystem
+
+- **[tesseract-core](https://github.com/pasteurlabs/tesseract-core)** — CLI, Python API, and runtime (this repo)
+- **[tesseract-jax](https://github.com/pasteurlabs/tesseract-jax)** — JAX integration for automatic gradient generation
+- **[tesseract-streamlit](https://github.com/pasteurlabs/tesseract-streamlit)** — Auto-generate interactive web apps from Tesseracts
 
-   ```bash
-   $ tesseract run vectoradd apply '{"inputs": {"a": [1], "b": [2]}}'
-   {"result":{"object_type":"array","shape":[1],"dtype":"float64","data":{"buffer":[3.0],"encoding":"json"}}}⏎
-   ```
+## Learn more
 
-> [!TIP]
-> Now you're ready to dive into the [documentation](https://docs.pasteurlabs.ai/projects/tesseract-core/latest/) for more information on
-> [installation](https://docs.pasteurlabs.ai/projects/tesseract-core/latest/content/introduction/installation.html),
-> [creating Tesseracts](https://docs.pasteurlabs.ai/projects/tesseract-core/latest/content/creating-tesseracts/create.html), and
-> [invoking them](https://docs.pasteurlabs.ai/projects/tesseract-core/latest/content/using-tesseracts/use.html).
+- [Documentation](https://docs.pasteurlabs.ai/projects/tesseract-core/latest/)
+- [Creating your first Tesseract](https://docs.pasteurlabs.ai/projects/tesseract-core/latest/content/creating-tesseracts/create.html)
+- [Differentiable programming guide](https://docs.pasteurlabs.ai/projects/tesseract-core/latest/content/introduction/differentiable-programming.html)
+- [Design patterns](https://docs.pasteurlabs.ai/projects/tesseract-core/latest/content/creating-tesseracts/design-patterns.html)
+- [Example gallery](https://docs.pasteurlabs.ai/projects/tesseract-core/latest/content/examples/example_gallery.html)
 
 ## License
 
-Tesseract Core is licensed under the [Apache License 2.0](https://github.com/pasteurlabs/tesseract-core/blob/main/LICENSE) and is free to use, modify, and distribute (under the terms of the license).
+Apache License 2.0. Free to use, modify, and distribute.
 
-Tesseract is a registered trademark of Pasteur Labs, Inc. and may not be used without permission.
+Tesseract is a registered trademark of Pasteur Labs, Inc.

docs/content/creating-tesseracts/design-patterns.md

@@ -0,0 +1,162 @@
+# Tesseract Design Patterns
+
+This page provides guidance on common questions around Tesseract design: what should (and shouldn't) be a Tesseract, and how to structure your workflow.
+
+## When to use a Tesseract
+
+Tesseracts are most useful when you need to:
+
+- **Share software across teams** — Package your code so others can use it without understanding implementation details
+- **Combine heterogeneous software** — Integrate components written in different languages/frameworks into a unified pipeline
+- **Deploy to diverse hardware** — Run the same component on different machines (local, cloud, HPC) without modification
+- **Enable gradient-based optimization** — Expose derivatives for use in optimization or calibration workflows
+- **Ensure reproducibility** — Capture dependencies and environment in a container for consistent execution
+
+## When NOT to use a Tesseract
+
+Tesseracts add overhead that isn't always justified:
+
+- **Single-user, single-environment workflows** — If you're the only one running the code on a single machine, the containerization overhead may not be worth it
+- **Sub-second latency requirements** — Tesseracts are designed for compute kernels that run for at least several seconds; for very fast operations, the container and HTTP overhead becomes significant
+- **Tightly coupled iterations** — If your inner loop calls a function millions of times, that function shouldn't be a Tesseract; instead, wrap the entire loop
+- **Simple scripts** — A quick data transformation script that won't be reused doesn't need Tesseract packaging
+
+## How granular should Tesseracts be?
+
+One of the most common questions is: _"Should I make one big Tesseract or many small ones?"_
+
+### Prefer fewer, coarser Tesseracts when:
+
+- Operations are tightly coupled and always run together
+- Data transfer between steps would be expensive (e.g., large meshes or tensors)
+- The combined operation is what users actually want to call
+- You need maximum performance (fewer container invocations)
+
+### Prefer more, finer Tesseracts when:
+
+- Components have different hardware requirements (e.g., one needs GPU, one doesn't)
+- Components have conflicting dependencies
+- Different team members own different parts
+- You want to swap out implementations (e.g., different solvers for the same interface)
+- Components are reusable across multiple workflows
+
+### A practical rule of thumb
+
+Think about the **natural unit of work** that makes sense to share. A Tesseract should wrap functionality that:
+
+1. Has a clear, well-defined interface (inputs and outputs)
+2. Represents a meaningful computation (not just a utility function)
+3. Could reasonably be owned and maintained by one person or team
+4. Takes at least a few seconds to run (to amortize container overhead)
+
+## Example: Simulation workflow
+
+Consider a CFD simulation workflow with these steps:
+
+1. Generate mesh from CAD geometry
+2. Run CFD solver
+3. Post-process results
+4. Visualize output
+
+**Option A: One Tesseract**
+
+```
+CAD → [Mesh + Solve + Post-process + Visualize] → Report
+```
+
+Pros: Simple to deploy, no intermediate data transfer
+Cons: Can't swap solver, can't run meshing on CPU while solving on GPU
+
+**Option B: Four Tesseracts**
+
+```
+CAD → [Mesh] → [Solve] → [Post-process] → [Visualize] → Report
+```
+
+Pros: Maximum flexibility, clear ownership
+Cons: Data transfer overhead, more complex orchestration
+
+**Option C: Two Tesseracts (recommended)**
+
+```
+CAD → [Mesh] → [Solve + Post-process + Visualize] → Report
+```
+
+Pros: Separates geometry (often CPU-bound, different expertise) from simulation (often GPU-bound, different team), minimal data transfer for tightly coupled steps
+
+The right choice depends on your team structure, hardware constraints, and reuse patterns. When in doubt, start with fewer Tesseracts and split them later if needed — it's easier to break apart than to combine.
+
+## Designing good interfaces
+
+### Keep schemas focused
+
+Your `InputSchema` and `OutputSchema` should contain only what's needed for the computation. Avoid:
+
+- Configuration that rarely changes (put it in the Tesseract itself or make it a build-time option)
+- Metadata that's not used in computation
+- Redundant fields that can be derived from others
+
+### Use meaningful types
+
+```python
+# Less clear
+class InputSchema(BaseModel):
+    data: Array[(None, None), Float64]  # What is this?
+
+# More clear
+class InputSchema(BaseModel):
+    mesh_coordinates: Array[(None, 3), Float64] = Field(
+        description="Node coordinates of the mesh (N nodes x 3 dimensions)"
+    )
+```
+
+### Design for composability
+
+If your Tesseract might be chained with others, design interfaces that make this natural:
+
+```python
+# Mesh generator output
+class MeshOutput(BaseModel):
+    nodes: Array[(None, 3), Float64]
+    elements: Array[(None, 4), Int32]
+
+# Solver input (matches mesh output)
+class SolverInput(BaseModel):
+    nodes: Array[(None, 3), Float64]
+    elements: Array[(None, 4), Int32]
+    boundary_conditions: BoundaryConditions
+```
+
+## Common anti-patterns
+
+### The "kitchen sink" Tesseract
+
+Don't create a single Tesseract that does everything your project needs. This defeats the purpose of modularity and makes it hard to maintain or reuse.
+
+### The "micro-Tesseract"
+
+Don't wrap trivial operations like `add(a, b)` as Tesseracts. The overhead isn't justified, and such operations should just be regular functions in your pipeline code.
+
+### Ignoring the single-entrypoint design
+
+Tesseracts have one `apply` function. If you need multiple entrypoints, create multiple Tesseracts. Don't try to work around this with mode flags:
+
+```python
+# Anti-pattern: mode switching
+class InputSchema(BaseModel):
+    mode: Literal["train", "predict", "evaluate"]
+    ...
+
+# Better: separate Tesseracts
+# - model-trainer (for training)
+# - model-predictor (for inference)
+# - model-evaluator (for evaluation)
+```
+
+### Stateful operations
+
+Tesseracts are designed to be stateless and context-free. Each call to `apply` should be independent. If you need state:
+
+- Pass it explicitly in the input schema
+- Store it in external storage (files, databases) and reference it by path
+- Reconsider whether a Tesseract is the right abstraction