Skip to content

Commit 849ccd7

Browse files
fix(fs): narrow create_once's duplicate arm and single-source the staging prefixes
Four corrections from independent review of the fold, each a real gap rather than a wording fix: 1. `create_once` evaluated `written.and_then(|()| hard_link(…))`, so its `Err(_) if path.is_file() => Ok(false)` arm also caught failures from the staged WRITE. An `ENOSPC`/`EIO` on the staged bytes was then reported as "this key was already published" whenever the target happened to exist — bytes that were never written are nobody's publication. Only the hardlink may answer that question now. The absorbed callers propagated a failed write unconditionally, so this restores their contract. No test can force the edge in-process (there is no write-fault injection in this tree), so it is review-pinned. 2. `replace` removed the staged file on a REFUSED creation, which meant st2 unlinked whatever an agent had planted at the staging path instead of merely refusing to follow it — and asymmetric with `create_once`, which returns before its cleanup. The creation now returns before the cleanup scope in both. 3. Folding the staging name into a module made each caller's prefix a bare string argument, and the assertion that pinned `.status.tmp-` went with `tmp_name`. The walkers that skip staged files by prefix now read the writer's own const — `status::TMP_STAGING_PREFIX` for the six in `catalog`, `catalog_transaction` and `agent_publish`, `message::TMP_STAGING_PREFIX` for the four in `message`, and `harness_context::is_legacy_staging_name` derives its prefix from the `TMP_PREFIX` the writer passes — so a walker can no longer drift from the writer. Each module also asserts its const's VALUE, which is the part a shared const cannot protect: renaming both sides together would leave every already-staged file on the fleet unrecognized. 4. `reserved_message_temporary_symlinks_are_never_followed` predicted staging names from one probe of a counter that is now process-global, so a sibling test advancing it past the planted window would have turned the test intermittent. It now plants, re-probes, and only proceeds once the very next name is one it has blocked. Also corrects the HC-T08 sentence added by the fold: bus message files are state-plane records too and stay at the writing process's umask, so "like every other state-plane record" was wrong. agent-identity: dev3.direct.omp.43sz6ujq agent-persona: generalist agent-supervisor: unavailable agent-tool: OMP agent-tool-version: 18.1.7 agent-runtime: OMP 18.1.7 tooling-profile: dotfiles@39a19af
1 parent 375bac9 commit 849ccd7

8 files changed

Lines changed: 103 additions & 48 deletions

File tree

docs/vrs/08-harness-context/requirements.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -103,11 +103,13 @@ record.
103103
other purposes, buys nothing a named include entry does not — bounded by
104104
st2-side test that pins the names it expects (HC-R05), and by the fact that no
105105
correctness property here depends on the transport at all: everything works
106-
with no replication, and remote visibility is what is lost. The pair is
107-
published mode `0600`, like every other state-plane record, so a transport
108-
that reads the catalog as a different unprivileged uid needs that decision
109-
taken deliberately rather than inherited from a default; no such transport
110-
runs today, which is what leaves the wire-cost half of `DQ-C1` unmeasured.
106+
with no replication, and remote visibility is what is lost. Both records are
107+
published mode `0600` — as is every record st2's shared publication primitive
108+
writes, though not the bus message files, which stay at the writing process's
109+
umask — so a transport reading the catalog as a different unprivileged uid
110+
needs that decision taken deliberately rather than inherited from a default.
111+
No such transport runs today, which is also what leaves the wire-cost half of
112+
`DQ-C1` unmeasured.
111113

112114
## Requirements
113115

src/agent_publish.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -676,7 +676,7 @@ fn copy_filtered_catalog(
676676
name_text.as_ref(),
677677
"resources" | "archive" | "inbox" | "status"
678678
)
679-
|| declaration_parent && name_text.starts_with(".status.tmp-"))
679+
|| declaration_parent && name_text.starts_with(crate::status::TMP_STAGING_PREFIX))
680680
{
681681
continue;
682682
}

src/catalog.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -641,7 +641,7 @@ pub(crate) fn validate_catalog_relative_profile_module_path(relative: &Path) ->
641641
matches!(
642642
*name,
643643
".workspace" | "resources" | "archive" | "inbox" | "status"
644-
) || name.starts_with(".status.tmp-")
644+
) || name.starts_with(crate::status::TMP_STAGING_PREFIX)
645645
});
646646
let reserved_template_subtree = first == "_templates"
647647
&& components.iter().skip(1).any(|name| {
@@ -657,7 +657,7 @@ pub(crate) fn validate_catalog_relative_profile_module_path(relative: &Path) ->
657657
| "archive"
658658
| "inbox"
659659
| "status"
660-
) || name.starts_with(".status.tmp-")
660+
) || name.starts_with(crate::status::TMP_STAGING_PREFIX)
661661
});
662662
anyhow::ensure!(
663663
!(reserved_control || reserved_root || reserved_agent_state || reserved_template_subtree),

src/catalog_transaction.rs

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2033,7 +2033,7 @@ fn collect_bundle_files(
20332033
| ".harness-state.seq"
20342034
| ".harness-state.lock"
20352035
| ".harness-context.lock"
2036-
) || name_text.starts_with(".status.tmp-");
2036+
) || name_text.starts_with(crate::status::TMP_STAGING_PREFIX);
20372037
if first.is_some() && relative_to_bundle.components().count() == 1 && state {
20382038
if source == ProjectionSource::Prepared {
20392039
anyhow::bail!(
@@ -2244,7 +2244,7 @@ fn reject_state_children(identity_path: &Path) -> Result<()> {
22442244
let name = entry.file_name();
22452245
let name = name.to_str().context("identity path is not UTF-8")?;
22462246
if matches!(name, "resources" | "archive" | "inbox" | "status")
2247-
|| name.starts_with(".status.tmp-")
2247+
|| name.starts_with(crate::status::TMP_STAGING_PREFIX)
22482248
{
22492249
anyhow::bail!(
22502250
"prepared catalog contains state-plane path: {}",
@@ -2934,7 +2934,7 @@ fn validate_declaration_leaf_path(path: &str) -> Result<()> {
29342934
!matches!(
29352935
components[3],
29362936
".workspace" | "resources" | "archive" | "inbox" | "status"
2937-
) && !components[3].starts_with(".status.tmp-"),
2937+
) && !components[3].starts_with(crate::status::TMP_STAGING_PREFIX),
29382938
"catalog apply marker contains a workspace or state-plane path"
29392939
);
29402940
}

src/fsatomic.rs

Lines changed: 15 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -95,18 +95,20 @@ pub(crate) fn replace(
9595
) -> io::Result<()> {
9696
let parent = parent_of(path)?;
9797
let staged = prepare(parent, &staging)?;
98-
let landed = (|| -> io::Result<()> {
99-
let mut file = create_staging(&staged)?;
98+
// A refused creation returns BEFORE the cleanup scope: the thing occupying a
99+
// `{prefix}.tmp-{pid}-{counter}` path is not ours, and refusing to follow it must not turn
100+
// into unlinking it.
101+
let mut file = create_staging(&staged)?;
102+
let staged_bytes = (|| -> io::Result<()> {
100103
file.write_all(bytes)?;
101104
if durability == Durability::FsyncFileAndDir {
102105
file.sync_all()?;
103106
}
104-
drop(file);
105-
fs::rename(&staged, path)
107+
Ok(())
106108
})();
107-
if let Err(error) = landed {
108-
// Best-effort: the staging name is unique per write, so a leftover is inert rather than a
109-
// path a later write could collide with.
109+
drop(file);
110+
if let Err(error) = staged_bytes.and_then(|()| fs::rename(&staged, path)) {
111+
// Best-effort, and only for the file this call created.
110112
let _ = fs::remove_file(&staged);
111113
return Err(error);
112114
}
@@ -127,15 +129,18 @@ pub(crate) fn create_once(path: &Path, bytes: &[u8], staging: Staging<'_>) -> io
127129
let parent = parent_of(path)?;
128130
let staged = prepare(parent, &staging)?;
129131
let mut file = create_staging(&staged)?;
130-
let written = file.write_all(bytes);
132+
let staged_bytes = file.write_all(bytes);
131133
drop(file);
132-
let created = match written.and_then(|()| fs::hard_link(&staged, path)) {
134+
// A failed staged write is never a duplicate: `Ok(false)` claims somebody else published
135+
// these bytes, and bytes that were never written are nobody's publication. Only the hardlink
136+
// may answer that question.
137+
let created = staged_bytes.and_then(|()| match fs::hard_link(&staged, path) {
133138
Ok(()) => Ok(true),
134139
// `hard_link` reports `AlreadyExists` for a taken name, but a target that is already a
135140
// regular file is the same answer whatever the error says.
136141
Err(_) if path.is_file() => Ok(false),
137142
Err(error) => Err(error),
138-
};
143+
});
139144
let _ = fs::remove_file(&staged);
140145
created
141146
}

src/harness_context.rs

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -392,8 +392,14 @@ pub(crate) fn is_legacy_harness_context_staging_file(
392392
Ok(true)
393393
}
394394

395+
/// The matcher and the writer must agree on the prefix, so both read [`TMP_PREFIX`]: this name is
396+
/// the one INVARIANTS row 29 says current-catalog identity walkers overlook, and a writer that
397+
/// drifted from this matcher would leave a staged file the walkers no longer recognize.
395398
fn is_legacy_staging_name(name: &str) -> bool {
396-
let Some(suffix) = name.strip_prefix(".harness-context.tmp-") else {
399+
let Some(suffix) = name
400+
.strip_prefix(TMP_PREFIX)
401+
.and_then(|rest| rest.strip_prefix(".tmp-"))
402+
else {
397403
return false;
398404
};
399405
let Some((pid, counter)) = suffix.split_once('-') else {

src/message.rs

Lines changed: 47 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -399,10 +399,14 @@ pub fn materialize_message_once(
399399
result
400400
}
401401

402-
/// The staging-name prefix for everything this module stages. Four sent-record and inbox walkers
403-
/// match `.message.tmp-` by prefix, so the name is a contract with them, not a local detail.
402+
/// The staging-name prefix for everything this module stages.
404403
const TMP_PREFIX: &str = ".message";
405404

405+
/// The full staged-name prefix four inbox and sent-record walkers skip by prefix, spelled ONCE so
406+
/// a walker cannot drift from the writer. Tied to [`TMP_PREFIX`] by
407+
/// `the_staging_prefix_is_the_one_the_inbox_walkers_skip`.
408+
const TMP_STAGING_PREFIX: &str = ".message.tmp-";
409+
406410
fn tmp_name() -> String {
407411
crate::fsatomic::staging_name(TMP_PREFIX)
408412
}
@@ -711,7 +715,7 @@ fn read_sent_records(directory: &Path) -> anyhow::Result<Vec<SentRecord>> {
711715
let Some(name) = name.to_str() else {
712716
anyhow::bail!("sent record filename is not UTF-8");
713717
};
714-
if name.starts_with(".message.tmp-") {
718+
if name.starts_with(TMP_STAGING_PREFIX) {
715719
continue;
716720
}
717721
anyhow::ensure!(name.ends_with(".json"), "unexpected sent record entry");
@@ -744,7 +748,7 @@ fn read_pending_records(directory: &Path) -> anyhow::Result<Vec<SentRecord>> {
744748
let Some(name) = name.to_str() else {
745749
anyhow::bail!("pending sent record filename is not UTF-8");
746750
};
747-
if name.starts_with(".message.tmp-") {
751+
if name.starts_with(TMP_STAGING_PREFIX) {
748752
continue;
749753
}
750754
let digest = name
@@ -804,7 +808,7 @@ fn read_sent_commits(directory: &Path) -> anyhow::Result<BTreeMap<String, SentCo
804808
let Some(name) = name.to_str() else {
805809
anyhow::bail!("sent commit filename is not UTF-8");
806810
};
807-
if name.starts_with(".message.tmp-") {
811+
if name.starts_with(TMP_STAGING_PREFIX) {
808812
continue;
809813
}
810814
let digest = name
@@ -845,7 +849,7 @@ fn read_sent_keys(directory: &Path) -> anyhow::Result<BTreeMap<String, SentKey>>
845849
let Some(name) = name.to_str() else {
846850
anyhow::bail!("sent key filename is not UTF-8");
847851
};
848-
if name.starts_with(".message.tmp-") {
852+
if name.starts_with(TMP_STAGING_PREFIX) {
849853
continue;
850854
}
851855
let digest = name
@@ -2561,6 +2565,18 @@ fn remove_inbox_duplicate(source: &Path, filename: &str) -> anyhow::Result<()> {
25612565
mod tests {
25622566
use super::*;
25632567

2568+
/// The staged name is a contract with four walkers that skip it by prefix (`inbox_dir` and the
2569+
/// three sent-record scans), all of which now match [`TMP_STAGING_PREFIX`] so none can drift
2570+
/// from the writer. What a shared const cannot catch is both sides being renamed together,
2571+
/// which would leave every already-staged file on the fleet unrecognized — hence the value
2572+
/// assertion.
2573+
#[test]
2574+
fn the_staging_prefix_is_the_one_the_inbox_walkers_skip() {
2575+
assert_eq!(TMP_PREFIX, ".message");
2576+
assert_eq!(TMP_STAGING_PREFIX, format!("{TMP_PREFIX}.tmp-"));
2577+
assert!(!is_message_filename(&tmp_name()), "a staged name must never look like a message");
2578+
}
2579+
25642580
/// [`atomic_create_file`]'s create-once contract. It is a hardlink, not a rename, and that is
25652581
/// the whole point: the first publication wins, a second reports `false` instead of replacing
25662582
/// the winner's bytes, and neither leaves a staged sibling behind for the four `.message.tmp-`
@@ -2588,7 +2604,7 @@ mod tests {
25882604
let residue = fs::read_dir(path.parent().unwrap())
25892605
.unwrap()
25902606
.map(|entry| entry.unwrap().file_name().to_string_lossy().into_owned())
2591-
.filter(|name| name.starts_with(".message.tmp-"))
2607+
.filter(|name| name.starts_with(TMP_STAGING_PREFIX))
25922608
.collect::<Vec<_>>();
25932609
assert!(residue.is_empty(), "staging residue left behind: {residue:?}");
25942610
}
@@ -2606,7 +2622,7 @@ mod tests {
26062622
let residue = fs::read_dir(tmp.path())
26072623
.unwrap()
26082624
.map(|entry| entry.unwrap().file_name().to_string_lossy().into_owned())
2609-
.filter(|name| name.starts_with(".message.tmp-"))
2625+
.filter(|name| name.starts_with(TMP_STAGING_PREFIX))
26102626
.collect::<Vec<_>>();
26112627
assert!(residue.is_empty(), "staging residue left behind: {residue:?}");
26122628
}
@@ -2764,23 +2780,29 @@ mod tests {
27642780
fs::create_dir_all(&inbox).unwrap();
27652781
let victim = tmp.path().join("victim");
27662782
fs::write(&victim, "must remain unchanged").unwrap();
2767-
// The counter now lives in `fsatomic`, so the next names are predicted from a probe
2768-
// rather than read off a module-local static: one call consumes `start`, so the writes
2769-
// this test blocks are the 4096 after it.
2770-
let probe = tmp_name();
2771-
let start = probe
2772-
.rsplit_once('-')
2773-
.and_then(|(_, counter)| counter.parse::<u64>().ok())
2774-
.expect("the staging grammar ends in the counter");
2775-
for counter in start + 1..start + 1 + 4096 {
2776-
symlink(
2777-
&victim,
2778-
inbox.join(format!(
2779-
"{TMP_PREFIX}.tmp-{}-{counter}",
2780-
std::process::id()
2781-
)),
2782-
)
2783-
.unwrap();
2783+
// The counter lives in `fsatomic` and is shared with every other staging site, so a
2784+
// sibling test running in parallel advances it too. Plant a window, then PROBE again: the
2785+
// loop only exits once the very next name is one this test has already blocked, so the
2786+
// assertion below cannot become "the call happened to pick a free name".
2787+
let counter_of = |name: &str| {
2788+
name.rsplit_once('-')
2789+
.and_then(|(_, counter)| counter.parse::<u64>().ok())
2790+
.expect("the staging grammar ends in the counter")
2791+
};
2792+
let mut planted_through = 0;
2793+
loop {
2794+
let probe = counter_of(&tmp_name());
2795+
if probe < planted_through {
2796+
break;
2797+
}
2798+
for counter in probe + 1..=probe + 512 {
2799+
symlink(
2800+
&victim,
2801+
inbox.join(format!("{TMP_PREFIX}.tmp-{}-{counter}", std::process::id())),
2802+
)
2803+
.unwrap();
2804+
}
2805+
planted_through = probe + 512;
27842806
}
27852807

27862808
let error = materialize_message_once(&inbox, "1784649988123-symlnk.md", "must not escape")

src/status.rs

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -284,6 +284,14 @@ fn write_record(path: &Path, state: State, written_at_ms: u64) -> anyhow::Result
284284
)
285285
}
286286

287+
/// The staging-name prefix this module hands [`crate::fsatomic`].
288+
const TMP_PREFIX: &str = ".status";
289+
290+
/// The full staged-name prefix six catalog and publication walkers match, spelled ONCE here so a
291+
/// walker can never drift from the writer. Tied to [`TMP_PREFIX`] by
292+
/// `the_staging_prefix_is_the_one_the_catalog_walkers_match`.
293+
pub(crate) const TMP_STAGING_PREFIX: &str = ".status.tmp-";
294+
287295
/// Atomic write: a staged sibling + rename, so a concurrent reader sees either the old bytes or
288296
/// the new bytes, never a partial file.
289297
///
@@ -294,7 +302,7 @@ fn write_atomic(path: &Path, content: &str) -> anyhow::Result<()> {
294302
crate::fsatomic::replace(
295303
path,
296304
content.as_bytes(),
297-
crate::fsatomic::Staging::new(".status"),
305+
crate::fsatomic::Staging::new(TMP_PREFIX),
298306
crate::fsatomic::Durability::Rename,
299307
)?;
300308
Ok(())
@@ -305,6 +313,18 @@ mod tests {
305313
use super::*;
306314
use std::time::{Duration as Dur, SystemTime};
307315

316+
/// The staged name is a contract with six walkers that skip it by prefix, in three other
317+
/// modules: `catalog.rs`, `catalog_transaction.rs` and `agent_publish.rs` all match
318+
/// [`TMP_STAGING_PREFIX`], so a changed writer prefix cannot desynchronize them. What a const
319+
/// cannot catch is the prefix being renamed on BOTH sides at once — a staged status file would
320+
/// then still be skipped locally, but the fleet's existing records and any other reader of the
321+
/// old name would not. That is what this assertion is for.
322+
#[test]
323+
fn the_staging_prefix_is_the_one_the_catalog_walkers_match() {
324+
assert_eq!(TMP_PREFIX, ".status");
325+
assert_eq!(TMP_STAGING_PREFIX, format!("{TMP_PREFIX}.tmp-"));
326+
}
327+
308328
/// [`write_atomic`]'s publication contract: the target ends up carrying the complete new
309329
/// bytes, no staged sibling survives, and the record is owner-only.
310330
///

0 commit comments

Comments
 (0)