Skip to content
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
0b5fc78
test(engine/sink): add unit tests for strict-relative chokepoint
bnimit Jun 1, 2026
67cf8ce
feat(engine/sink)!: require relative paths in sink output
bnimit Jun 1, 2026
bae7f83
chore: gitignore .local-specs/
bnimit Jun 1, 2026
7008ff5
chore(engine/fixtures): migrate workflow YAML to relative sink paths
bnimit Jun 1, 2026
51428d8
test(engine): integration tests for relative sink output path
bnimit Jun 1, 2026
9c901cc
docs(engine): update Sandbox & sink writes section for relative paths
bnimit Jun 1, 2026
f496e3f
docs: changelog entry for relative sink-output paths
bnimit Jun 1, 2026
bbb08bf
chore(engine): bump workspace version to 0.0.376
bnimit Jun 1, 2026
483c5b9
style: cargo fmt
bnimit Jun 1, 2026
74555a6
fix(engine/fixtures): address PR #2123 review feedback
bnimit Jun 2, 2026
3900491
chore(engine/examples): regenerate plateau workflow YAMLs
bnimit Jun 2, 2026
681a53d
chore: re-trigger CI after PR review fixes
bnimit Jun 2, 2026
11643b9
merge: main into chore/sink-relative-paths
bnimit Jun 2, 2026
e27955e
fix(engine/sink): route cesium3dtiles + mvt buffer keys through SinkO…
bnimit Jun 2, 2026
a454c76
fix(engine/test): plateau-tiles-test uses run_with_sandbox_root
bnimit Jun 2, 2026
34af8c0
style: cargo fmt
bnimit Jun 2, 2026
cf16250
refactor(engine/sink): extract ensure_relative_path helper
bnimit Jun 2, 2026
cef62f7
refactor(engine/sink): make SinkOutput::from_resolved_uri pub(crate)
bnimit Jun 2, 2026
f786803
refactor(engine/sink): unify SinkOutput around new(sandbox_root, path…
bnimit Jun 3, 2026
a6aa0bb
docs(engine): update Sandbox & sink writes section for unified API
bnimit Jun 3, 2026
77253ad
style: cargo fmt
bnimit Jun 3, 2026
4fad127
refactor(engine/sink): remove ensure_valid_relative_path
bnimit Jun 3, 2026
554b8e7
Merge branch 'main' into chore/sink-relative-paths
bnimit Jun 4, 2026
8a52598
chore(engine): bump workspace version to 0.0.378
bnimit Jun 4, 2026
390ac37
chore(engine/fixtures): migrate PLATEAU6 quality-check fixtures to re…
bnimit Jun 4, 2026
b38f176
docs(engine): clarify per-sink portability across storage backends
bnimit Jun 4, 2026
74a742a
docs(engine): remove Sandbox & sink writes section from AGENTS.md
bnimit Jun 4, 2026
ec1f0f2
error log
asrcpq Jun 4, 2026
4d8d4db
log before null fallback
asrcpq Jun 4, 2026
eeba0e2
fix(engine/sink): make sandbox-root prefix-strip a hard error
bnimit Jun 4, 2026
0cb8832
Revert "log before null fallback"
asrcpq Jun 4, 2026
feab4f0
Revert "error log"
asrcpq Jun 4, 2026
2a6dc72
Merge branch 'main' into chore/sink-relative-paths
bnimit Jun 5, 2026
80cfbd2
chore(engine/sink): drop CHANGELOG entry and .local-specs gitignore line
bnimit Jun 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@
reearth-flow-websocket/
reearth-flow-websocket-rs/
target/
.local-specs/
22 changes: 22 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,28 @@

All notable changes to this project will be documented in this file.

## Unreleased

### Breaking changes

- **engine/sink**: Sink `output` parameters now require a **relative path**.
The engine joins the path against the per-job artifact directory
(`sandbox_root`) internally; workflow authors no longer prefix with
`Url(env["workerArtifactPath"]) / ...`.

**Migration**: Replace `output: { type: flowExpr, value: 'Url(env["workerArtifactPath"]) / "x"' }`
with `output: { type: string, value: "x" }` (static) or
`output: { type: flowExpr, value: '"x_" + attributes["k"]' }` (dynamic).
The equivalent `file::join_path(env.get("workerArtifactPath"), "x")` form
also migrates to just `"x"`.

Workflows that still use the old absolute-URI form fail at runtime with
an error message that names `workerArtifactPath` — search your logs for
that keyword to locate workflows that need migration.

All built-in plateau example fixtures (87 files) have been migrated as
part of this release.

## 0.1.0-alpha.17 - 2026-05-26

### Web
Expand Down
46 changes: 33 additions & 13 deletions engine/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,25 +61,45 @@ Use the `add-action` skill for a full step-by-step guide including i18n workflow

### Sandbox & sink writes

Every executor context carries a `sandbox_root: Uri` field that bounds where
sink actions are allowed to write. The chokepoint is
`reearth_flow_action_sink::SinkOutput::from_path`, which validates the
destination URI against `ctx.sandbox_root` via `sandbox::ensure_under` before
acquiring a storage handle.
Sinks accept a **relative path** for their `output` parameter. The engine
joins the path against `ctx.sandbox_root` (the per-job artifact directory)
and validates the result via `sandbox::ensure_under`. The chokepoint is
`reearth_flow_action_sink::SinkOutput::from_path`.

Workflow authors write:

```yaml
output:
type: string
value: "out.gpkg" # or "group/a.geojson", with attribute concat for dynamic names
```

The engine joins this against whatever `sandbox_root` the worker/CLI was
launched with — `file:///var/jobs/abc/`, `gs://bucket/jobs/abc/`,
`ram:///jobs/abc/`, etc. The same workflow YAML is portable across storage
backends.
Comment thread
bnimit marked this conversation as resolved.
Outdated

`SinkOutput::from_path` rejects:
- Empty strings, leading/trailing whitespace, literal `.` / `..`
- Absolute URIs (`scheme://...`) — error message names `workerArtifactPath`
to direct customers to the migration
- Leading `/` (ambiguous) or leading `~` (home expansion not supported)
- Paths that resolve to the artifact directory itself after normalization
- Paths that escape the sandbox via `..` (caught by `ensure_under`)

**All sink-side writes MUST go through `SinkOutput`.** Calling
`Storage::put_sync` (or any other raw I/O like `std::fs::write`) from a sink
or sink-adjacent code path skips the check and reintroduces unbounded
writes. If a new sink format or sidecar write is needed, route it through
`SinkOutput::from_path` / `SinkOutput::join` / `SinkOutput::write`. Reviewers
should flag any direct `put_sync` / `std::fs` calls in sink code as
regressions.
`Storage::put_sync` (or any other raw I/O like `std::fs::write`) from a
sink or sink-adjacent code path skips the check and reintroduces
unbounded writes. If a new sink format or sidecar write is needed, route
it through `SinkOutput::from_path` / `SinkOutput::join` /
`SinkOutput::write`. Reviewers should flag any direct `put_sync` /
`std::fs` calls in sink code as regressions.

Production entrypoints (`Runner::run_with_sandbox_root`,
`AsyncRunner::run_with_sandbox_root`) reject the `file:///` sentinel so a
misconfigured `workerArtifactPath` cannot silently disable the sandbox.
`Runner::run` (legacy / tests) intentionally uses that sentinel and bypasses
the guard.
`Runner::run` (legacy / tests) intentionally uses that sentinel and
bypasses the guard.

## Key Constraints

Expand Down
50 changes: 25 additions & 25 deletions engine/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion engine/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ homepage = "https://github.com/reearth/reearth-flow"
license = "MIT OR Apache-2.0"
repository = "https://github.com/reearth/reearth-flow"
rust-version = "1.93.1" # Remember to update clippy.toml as well
version = "0.0.375"
version = "0.0.376"

[profile.dev]
opt-level = 0
Expand Down
22 changes: 15 additions & 7 deletions engine/runtime/action-processor/src/feature/writer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ mod citygml;
mod csv;
mod json;

use std::{collections::HashMap, str::FromStr, sync::Arc};
use std::{collections::HashMap, sync::Arc};

use indexmap::IndexMap;
use reearth_flow_common::{csv::Delimiter, uri::Uri};
Expand Down Expand Up @@ -255,8 +255,15 @@ impl Processor for FeatureWriter {
let path = scope
.eval_ast::<String>(&output)
.map_err(|e| FeatureProcessorError::FeatureWriterFactory(format!("{e:?}")))?;
let output = Uri::from_str(path.as_str())?;
let buffer = self.buffer.entry(output).or_default();
// Use SinkOutput::from_path so the path is validated as strict-relative
// and joined against sandbox_root, rather than against CWD.
let sink_out = reearth_flow_action_sink::SinkOutput::from_path(
&NodeContext::from(ctx.clone()),
path.as_str(),
)
.map_err(|e| FeatureProcessorError::FeatureWriterFactory(format!("{e}")))?;
let output_uri = sink_out.uri().clone();
let buffer = self.buffer.entry(output_uri).or_default();
Comment thread
bnimit marked this conversation as resolved.
Outdated
buffer.push(ctx.feature);
Ok(())
}
Expand All @@ -278,12 +285,13 @@ impl Processor for FeatureWriter {
),
])
.into();
// Enforce sandbox: CSV/TSV/JSON/CityGML all write directly to
// `output`; without this check `FeatureWriter` would be an
// out-of-sandbox escape hatch alongside the sink writers.
// Defense-in-depth: ctx.sandbox_root was validated at buffer-insertion
// time in process() via SinkOutput::from_path, but ctx here may be a
// different NodeContext at flush time. Re-verify before each write to
// keep the sandbox gate co-located with put_sync.
reearth_flow_action_sink::ensure_under(&ctx.sandbox_root, output).map_err(|e| {
FeatureProcessorError::FeatureWriter(format!(
"output {output} rejected by sandbox: {e}"
"sink output {output} rejected by sandbox: {e}"
))
})?;
match self.params {
Expand Down
Loading
Loading