Skip to content

Commit 102eb20

Browse files
committed
reworked infra role to work with cardano-up
1 parent 999ee58 commit 102eb20

38 files changed

Lines changed: 2867 additions & 34 deletions

docs/ADDING_A_TOOL.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -174,6 +174,33 @@ dev:
174174

175175
Write the keys idempotently (replace in place rather than appending) so repeated `just dev` runs don't accumulate duplicate lines. Off-chain/devnet consumers read these and never need to know which tool wrote them.
176176

177+
**Infrastructure providers are added as data, not a new template.** The infra role is backed by a single shared `cardano-up` driver (`templates/_infra/cardano-up`), and all selected providers aggregate into one `infra/` component. To add a provider (e.g. `dolos`), you write only a `registry/tools/<provider>.toml` — no template:
178+
179+
```toml
180+
[tool]
181+
id = "dolos"
182+
name = "Dolos"
183+
description = ""
184+
website = "https://…"
185+
languages = [] # infra providers have no user-facing language
186+
system_deps = ["docker", "cardano-up"]
187+
detect = [] # infra is scanned by a driver marker, not per-tool
188+
189+
[roles.infrastructure]
190+
template = "_infra/cardano-up" # always the shared driver
191+
192+
[infra]
193+
cardano_up_package = "dolos" # the `cardano-up install` package id
194+
# Map cardano-up's `context env` outputs → contract .env keys. A mapping for an
195+
# existing key (e.g. NODE_SOCKET_PATH) overrides the default at generation time.
196+
env = [{ from = "DOLOS_SOCKET_PATH", to = "NODE_SOCKET_PATH" }]
197+
```
198+
199+
If a mapping targets a contract `.env` key that isn't seeded yet (a brand-new
200+
connection var), promote it: add a `contract::ENV_*` constant and a seeded
201+
`KEY=` line in `templates/_base/env.jinja` so every project always carries it.
202+
See `docs/proposals/infra-via-cardano-up.md` for the full model.
203+
177204
**Devnet tools** provision a local throwaway chain. They should read both the blueprint and the `.env` if they are present, but must work if neither exists, and write the connection vars above during `dev` — that is how off-chain components reach the devnet, and it composes with any off-chain tool without per-pair code.
178205

179206
**Formal-methods tools** have no extra contract beyond the four Justfile targets.

docs/ARCHITECTURE.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -145,7 +145,7 @@ pub struct FileEntry { dest: PathBuf, source: TemplateSource, render: bool }
145145
pub enum TemplateSource { Base(String), Role(String), Optional(String), Inline(Vec<u8>) }
146146
```
147147

148-
`TemplateContext` is `Serialize` and is the entire surface templates can see. It carries `has_*` booleans per role, an `Option<RoleContext>` per single-tool role, `infra_tools: Vec<RoleContext>`, the contract constants (`blueprint_path`, `env_vars`), and Nix info. `render` is derived from the `.jinja` extension.
148+
`TemplateContext` is `Serialize` and is the entire surface templates can see. It carries `has_*` booleans per role, an `Option<RoleContext>` per single-tool role, `infra_tools: Vec<InfraToolContext>` plus `infra_context_name` and the resolved `infra_env` (the aggregated infra component, TECH_SPEC §4.6), the contract constants (`blueprint_path`, `env_vars`), and Nix info. `render` is derived from the `.jinja` extension.
149149

150150
---
151151

@@ -225,7 +225,7 @@ Walks `selection.assignments`, resolves each tool against the registry, and buil
225225
Produces the ordered `FilePlan`:
226226
1. **Base layer** (always): `Justfile`, `README.md`, `.gitignore`, `.env`.
227227
2. **Blueprint dir**: `blueprint/.gitkeep`, emitted whenever the selection includes any blueprint-producing-or-consuming role: i.e., any role **except** infrastructure (equivalently: present unless the project is infrastructure-only).
228-
3. **Role layers**: for each assignment, read the template's `manifest.toml` and add its files. Infrastructure tools each nest under `infra/<tool_id>/`.
228+
3. **Role layers**: for each assignment, read the template's `manifest.toml` and add its files. **Infrastructure is special — it aggregates**: all selected infra tools share one driver template (`_infra/cardano-up`) emitted **once** at `infra/`, rendered over the full set (`TemplateContext.infra_tools`). This is because the infra engine (`cardano-up`) manages the whole stack as a single unit, not per service. Every other role is one tool → one directory. See `docs/proposals/infra-via-cardano-up.md`.
229229
4. **Optional layer**: `flake.nix` + `.envrc` when `nix` is set.
230230

231231
No I/O: only embedded assets are read. `render` is set from the `.jinja` extension.

docs/ROADMAP.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ Deliverables below are tracked as checklists (`[ ]` = not yet done).
3333
**The tool: all five roles present, four building green, formal-methods preview:**
3434
- [ ] **On-chain:** Aiken; make the template genuinely `build`+`test` green (blueprint at canonical path).
3535
- [ ] **Off-chain:** MeshJS + Tx3; both generate and build.
36-
- [ ] **Infrastructure:** no tool ships yet — the role is present in the vocabulary but unfilled (a real deployable service such as Kupo+Ogmios or a node provider is a follow-up). Yaci DevKit fills the **devnet** role instead (it is a dev/test kit, never deployed).
36+
- [x] **Infrastructure:** filled via `cardano-up`**Kupo, Ogmios, Dolos, Tx Submit API, Cardano Node, Cardano Node API, and Dingo** ship as selectable providers that aggregate into a single `infra/` component (one cardano-up context per project). Adding further providers is pure data (a registry TOML, no template). Uses `cardano-up`'s released `--context` flag (blinklabs-io#294; see `docs/proposals/infra-via-cardano-up.md`). Yaci DevKit remains in the **devnet** role (it is a dev/test kit, never deployed).
3737
- [ ] **Devnet:** Yaci DevKit (local devnet — its `dev` starts a Blockfrost-compatible devnet and writes the standard `.env` connection vars, so off-chain connects to it automatically; `test` runs an integration smoke test).
3838
- [ ] **Formal-methods:** preview; visible in the registry/UI as "coming soon"; the Blaster placeholder is not a build-green deliverable yet (made real at DX.05).
3939

docs/TECH_SPEC.md

Lines changed: 29 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -123,19 +123,35 @@ id = "aiken" # required, unique across registry, kebab
123123
name = "Aiken" # required, human display
124124
description = "" # required, newcomer-facing
125125
website = "https://…" # required
126-
languages = ["aiken"] # required, ≥1
126+
languages = ["aiken"] # required, ≥1 — except infra tools, which may use [] (no user-facing language)
127127
system_deps = ["aiken"] # required (may be []); abstract dep ids → registry/deps.toml (§9)
128128
nix_packages = ["aiken"] # optional (default []); nixpkgs attrs for the dev shell
129129

130130
[roles.on-chain] # ≥1 [roles.<kebab>] block; key validated against Role
131131
template = "aiken/on-chain" # required; path under templates/
132132
```
133133

134+
Tools filling the **infrastructure** role additionally require an `[infra]` table
135+
(validated at load: `[roles.infrastructure]` present ⇒ `[infra]` required, else
136+
`RegistryError::InfraConfigMissing`). It declares the `cardano-up` package and the
137+
output→`.env`-key mappings the aggregated driver writes (see the infra-via-cardano-up proposal):
138+
139+
```toml
140+
[roles.infrastructure]
141+
template = "_infra/cardano-up" # the shared driver template (all infra tools use this)
142+
143+
[infra]
144+
cardano_up_package = "kupo" # package id passed to `cardano-up install`
145+
env = [{ from = "KUPO_URL", to = "INDEXER_URL" }] # cardano-up output → contract .env key
146+
```
147+
134148
`system_deps` is **per-tool, flat** (§9.1): it applies whenever the tool is selected for any role.
135149

136150
```rust
137151
RoleConfig { template }
138-
ToolDef { id, name, description, website, languages, nix_packages, roles: HashMap<Role, RoleConfig> }
152+
EnvMapping { from, to }
153+
InfraConfig { cardano_up_package, env: Vec<EnvMapping> }
154+
ToolDef { id, name, description, website, languages, nix_packages, detect, roles: HashMap<Role, RoleConfig>, infra: Option<InfraConfig> }
139155
```
140156

141157
Load-time validation (`registry/loader.rs`), all fatal:
@@ -206,7 +222,7 @@ MiniJinja environment (planned config):
206222

207223
### 4.4 Path safety & destinations
208224

209-
- `dest` is resolved **relative to the role dir** (`on-chain/`, `off-chain/`, `test/`, `formal-methods/`); for infrastructure, relative to `infra/<tool_id>/`.
225+
- `dest` is resolved **relative to the role dir** (`on-chain/`, `off-chain/`, `test/`, `formal-methods/`); for infrastructure, relative to `infra/` (the aggregated component — no per-tool subdir, §6.1).
210226
- `dest` MUST be relative and MUST NOT contain `..` or a leading `/` (no escaping the project root). Enforced + tested. (Manifests are first-party today, but the check is cheap insurance and required if templates ever become third-party.)
211227
- Base/optional layer dests are fixed (§6).
212228

@@ -228,10 +244,13 @@ struct TemplateContext {
228244

229245
on_chain: Option<RoleContext>,
230246
off_chain: Option<RoleContext>,
231-
infra_tools: Vec<RoleContext>, // 0..n, canonical order (§11)
247+
infra_tools: Vec<InfraToolContext>, // 0..n, canonical order (§11); aggregated infra component
232248
devnet: Option<RoleContext>,
233249
formal_methods: Option<RoleContext>,
234250

251+
infra_context_name: String, // cardano-up context the infra driver targets (= project_name)
252+
infra_env: Vec<EnvMapping>, // resolved, key-unique .env emissions for infra (proposal §5.4)
253+
235254
blueprint_path: String, // "blueprint/plutus.json" (contract constant)
236255
env_vars: <ordered map>, // see §6.3; iterated in sorted-key order
237256

@@ -240,6 +259,8 @@ struct TemplateContext {
240259
}
241260

242261
struct RoleContext { tool_id, tool_name, language, dir } // language = tool.languages[0]
262+
struct InfraToolContext { tool_id, tool_name, cardano_up_package, env: Vec<EnvMapping> }
263+
struct EnvMapping { from, to } // cardano-up output var → contract .env key
243264
```
244265

245266
This struct is the contract. Adding a field is additive; renaming/removing is a breaking template-API change.
@@ -262,7 +283,7 @@ Determinism note: any consumer that emits tools/roles must sort (§11), since `b
262283

263284
1. **Base layer** (always): `Justfile`, `README.md`, `.gitignore`, `.env`.
264285
2. **Blueprint dir**: `blueprint/.gitkeep`, **if any non-infrastructure role is present** (§6.2). Source is `TemplateSource::Inline(empty)`.
265-
3. **Role layers**: assignments processed in **`Role::ALL` order** (not flag order). For each, read the template manifest and append its files (rendered per §4.2). Infra tools nested under `infra/<tool_id>/`, **tools sorted by `tool_id`** (§11).
286+
3. **Role layers**: assignments processed in **`Role::ALL` order** (not flag order). For each, read the template manifest and append its files (rendered per §4.2). **Infrastructure aggregates**: all selected infra tools share one driver template (`_infra/cardano-up`), emitted **once** at `infra/` on the first infra assignment (the rest are contiguous after the canonical sort and skipped); they are still sorted by `tool_id` for the rendered `infra_tools`/`infra_env` order (§11). All infra tools must resolve to the same template path, else `ScaffoldError::InfraTemplateMismatch`. See `docs/proposals/infra-via-cardano-up.md`.
266287
4. **Optional layer**: if `nix`, `flake.nix` (rendered) + `.envrc` (`Inline "use flake\n"`).
267288

268289
`--dry-run` returns this `FilePlan` (no rendering, no I/O).
@@ -311,7 +332,7 @@ The top level aggregates only the tasks that **terminate and compose**:
311332

312333
### 7.2 No top-level `dev`
313334

314-
There is **no top-level `dev` target**. Long-running / interactive tasks (watch modes, local devnets, REPLs) do not aggregate into one foreground command — that is exactly why a multi-service launcher is awkward — so they are **per-component**: the developer runs `just -f <role>/Justfile dev` (or `just -f infra/<tool>/Justfile dev`) directly, documented in the README.
335+
There is **no top-level `dev` target**. Long-running / interactive tasks (watch modes, local devnets, REPLs) do not aggregate into one foreground command — that is exactly why a multi-service launcher is awkward — so they are **per-component**: the developer runs `just -f <role>/Justfile dev` (or `just -f infra/Justfile dev` for the aggregated infra stack) directly, documented in the README.
315336

316337
`dev` is **optional per component** (§7): a tool provides it only when it has a genuine watch/daemon/devnet mode. Because the top level never aggregates `dev`, a component without one costs nothing — and we don't ship no-op `dev` targets just to fill the slot.
317338

@@ -507,6 +528,8 @@ The standalone `cardano-init doctor` takes **no flags describing the project**:
507528
3. A tool matches if **any** of its `detect` signatures matches. Exactly one match ⇒ the component is identified; zero (or an ambiguous multiple) ⇒ the directory is reported as **unrecognized** (renamed, modified, or a foreign project). A renamed *directory* simply isn't found, so that role is absent.
508529
4. The required set is `{just}` ∪ the `system_deps` of every identified tool (§9.1), fed to the resolver (§9.4).
509530

531+
**Infrastructure is the exception.** The aggregated `infra/` component has no per-tool subdirs (it's the single cardano-up driver), so it is *not* matched against per-tool `detect` signatures. Instead the scan recognizes it by a driver marker — `infra/Justfile` referencing `cardano-up` — and reports a synthetic `cardano-up` component (`doctor::INFRA_DRIVER_ID`). Its contribution to the required set is the **union of all registered infra tools' `system_deps`** (`{docker, cardano-up}`), data-driven from the registry. So infra tools carry `detect = []`.
532+
510533
**Detect signatures (`detect` in `registry/tools/<tool>.toml`).** A list; each entry is either:
511534
- a **bare path** (relative to the role dir) — matches if the file exists; or
512535
- a **table** `{ file = "<path>", contains = "<substring>" }` — matches if the file exists *and* its text contains the substring.

registry/deps.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,8 +53,8 @@ install = [{ npm = "@aiken-lang/aikup" }, { curl = "https://install.aiken-lang.o
5353

5454
[cardano-up]
5555
binaries = ["cardano-up"]
56-
docs = "https://github.com/cardano-foundation/cardano-up"
57-
install = [{ npm = "@cardano-foundation/cardano-up" }, { nix = "cardano-up" }]
56+
docs = "https://github.com/blinklabs-io/cardano-up"
57+
install = [{ go = "github.com/blinklabs-io/cardano-up/cmd/cardano-up@latest" }]
5858

5959
[rustup]
6060
binaries = ["rustup"]
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
[tool]
2+
id = "cardano-node-api"
3+
name = "Cardano Node API"
4+
description = """\
5+
A multi-protocol API in front of a local Cardano node, exposing its functions \
6+
over gRPC (UTxO RPC) and REST. Useful when off-chain code prefers a modern API \
7+
to the raw node protocols. Run as infrastructure via cardano-up (which pulls \
8+
in a Cardano node automatically). Its gRPC endpoint is published to the \
9+
project .env as CARDANO_NODE_API_URL, and the node socket as NODE_SOCKET_PATH."""
10+
website = "https://github.com/blinklabs-io/cardano-node-api"
11+
languages = []
12+
system_deps = ["docker", "cardano-up"]
13+
nix_packages = []
14+
15+
[roles.infrastructure]
16+
template = "_infra/cardano-up"
17+
18+
[infra]
19+
cardano_up_package = "cardano-node-api"
20+
env = [{ from = "CARDANO_NODE_API_GRPC", to = "CARDANO_NODE_API_URL" }]

registry/tools/cardano-node.toml

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
[tool]
2+
id = "cardano-node"
3+
name = "Cardano Node"
4+
description = """\
5+
The reference Cardano node. Choose this when you just want a synced local node \
6+
and its UNIX socket — no indexer or query layer — for tools that talk to the \
7+
node directly. Run as infrastructure via cardano-up. The socket path is \
8+
published to the project .env as NODE_SOCKET_PATH. Pair it with an off-chain \
9+
tool that uses the socket, or add an indexer/query provider (kupo, ogmios) for \
10+
a fuller stack."""
11+
website = "https://github.com/IntersectMBO/cardano-node"
12+
languages = []
13+
system_deps = ["docker", "cardano-up"]
14+
nix_packages = []
15+
16+
[roles.infrastructure]
17+
template = "_infra/cardano-up"
18+
19+
[infra]
20+
# Only a node socket. The driver's base default already maps
21+
# CARDANO_NODE_SOCKET_PATH → NODE_SOCKET_PATH, so no explicit mapping is needed.
22+
cardano_up_package = "cardano-node"
23+
env = []

registry/tools/dingo.toml

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
[tool]
2+
id = "dingo"
3+
name = "Dingo"
4+
description = """\
5+
A Cardano node implementation from Blink Labs with batteries included: it \
6+
syncs the chain and serves a Blockfrost-compatible HTTP API alongside its own \
7+
node socket — a single all-in-one process. Run as infrastructure via \
8+
cardano-up (it does not run a separate cardano-node). Its Blockfrost-compatible \
9+
API is published to the project .env as INDEXER_URL and its socket as \
10+
NODE_SOCKET_PATH, so off-chain components connect to it automatically."""
11+
website = "https://github.com/blinklabs-io/dingo"
12+
languages = []
13+
system_deps = ["docker", "cardano-up"]
14+
nix_packages = []
15+
16+
[roles.infrastructure]
17+
template = "_infra/cardano-up"
18+
19+
[infra]
20+
cardano_up_package = "dingo"
21+
env = [
22+
{ from = "DINGO_BLOCKFROST", to = "INDEXER_URL" },
23+
{ from = "DINGO_SOCKET_PATH", to = "NODE_SOCKET_PATH" },
24+
]

registry/tools/dolos.toml

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
[tool]
2+
id = "dolos"
3+
name = "Dolos"
4+
description = """\
5+
A lightweight Cardano data node. Dolos syncs the chain and serves it over a \
6+
UTxO RPC (gRPC) interface — a single process that replaces a full node plus \
7+
separate indexer for many off-chain needs. Run as infrastructure via \
8+
cardano-up. It provides its own node socket (it does not run a separate \
9+
cardano-node), published to the project .env as NODE_SOCKET_PATH, and its gRPC \
10+
endpoint as DOLOS_GRPC_URL."""
11+
website = "https://github.com/txpipe/dolos"
12+
languages = []
13+
system_deps = ["docker", "cardano-up"]
14+
nix_packages = []
15+
16+
[roles.infrastructure]
17+
template = "_infra/cardano-up"
18+
19+
[infra]
20+
cardano_up_package = "dolos"
21+
env = [
22+
{ from = "DOLOS_GRPC", to = "DOLOS_GRPC_URL" },
23+
{ from = "DOLOS_SOCKET_PATH", to = "NODE_SOCKET_PATH" },
24+
]

registry/tools/kupo.toml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
[tool]
2+
id = "kupo"
3+
name = "Kupo"
4+
description = """\
5+
A fast, lightweight chain-index for Cardano. Kupo watches the chain for \
6+
addresses, policy ids, or transaction outputs you care about and serves them \
7+
over a simple HTTP API — ideal as the indexer behind an off-chain app. Run as \
8+
infrastructure via cardano-up, which also pulls in a Cardano node \
9+
automatically. Its endpoint is published to the project .env as INDEXER_URL, \
10+
so off-chain components connect to it without extra wiring."""
11+
website = "https://github.com/CardanoSolutions/kupo"
12+
languages = []
13+
system_deps = ["docker", "cardano-up"]
14+
nix_packages = []
15+
16+
[roles.infrastructure]
17+
template = "_infra/cardano-up"
18+
19+
[infra]
20+
cardano_up_package = "kupo"
21+
env = [{ from = "KUPO_URL", to = "INDEXER_URL" }]

0 commit comments

Comments
 (0)