Skip to content

Commit fff659e

Browse files
authored
sync: key the scan cache locally, and replicate the executable bit (#47)
Two changes with one theme: fabric should agree with git about what a catalog is, and a local cache must never take its key from a replicated structure. THE CACHE. scan_folder reused a recorded hash when size and mtime matched what the MANIFEST held. The manifest crosses the wire, so a local caching decision was made from a value another machine chose. Two contending entries of equal size could collide on size plus mtime, the cache reported content the file did not hold, and the versions leapfrogged forever. That was the permanent three-node divergence, and removing mtime stamping only removed the mechanism that manufactured the collision, not the conflation that believed it. A separate scan_cache now records what THIS machine observed on its own disk, and never crosses the wire. It is deliberately NOT merged into the observed receipt: that receipt decides whether a missing path becomes a tombstone, and a tombstone is sent to peers, so loading a performance concern onto it would repeat the original mistake mirrored. Two structures, two jobs. It is filled only from real disk reads, never from a requested value, so a filesystem that truncates a timestamp cannot make it miss forever. Absent in an older state file, where serde default gives an empty one that warms on the first scan. THE EXECUTABLE BIT. Fabric tracked content and nothing else. Git tracks content, the executable bit and symlinks, and deliberately not mtime. The synced catalog holds fabric binaries, and they arrived without the bit and could not be run until somebody chmod-ed them by hand. The bit is now recorded and applied before the atomic rename, so a file is never briefly visible with the wrong mode. FileMeta.mtime is demoted rather than removed. It is documented as informational and is never applied. Removing it would break parsing on any peer not yet upgraded, because the field has no serde default; adding a field with a default is safe in a way removing one is not. Symlinks are still skipped, but they now say so and say that git tracks them. Silence was the problem. ONE CAUSE, TWO SYMPTOMS, NAMED IN ONE PLACE. local_write returns early when the content hash is unchanged, which is what makes applying a peer's content echo-free. So no change that alters zero bytes can propagate. An invisible heartbeat and an invisible chmod are the same defect, not two. Both symptoms are pinned by tests that assert they still do NOT propagate, so nobody fixes one and assumes the other followed, and the chmod case is recorded as a DIVERGENCE FROM GIT rather than an implementation note. Verified by sabotage, not only by green: dropping mtime_nanos from the cache key fails the sub-second test AND the three-node stress; never applying the mode fails the executable test. 211 lib tests green, plus lifecycle, provisioning and the real-iroh sync slice.
1 parent 560f83c commit fff659e

5 files changed

Lines changed: 701 additions & 128 deletions

File tree

CHANGELOG.md

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,66 @@ EXPERIMENTAL, so on-disk formats and the CLI may change without notice.
88

99
### Added
1010

11+
- **The executable bit is replicated.** Fabric now syncs the attributes git
12+
syncs. A file arrives executable if it was executable at its origin, applied
13+
before the atomic rename so it is never briefly visible with the wrong mode.
14+
15+
The live case: the synced catalog holds fabric binaries, and they arrived
16+
without the bit and could not be run until somebody chmod-ed them by hand.
17+
18+
**Divergence from git:** a `chmod` on an ALREADY SYNCED file does not
19+
propagate. Git propagates one, because a mode change rewrites the tree object
20+
and is a real commit. Fabric does not, for the reason below. A NEW file
21+
carries its mode correctly, which is the case that actually bites.
22+
23+
- **Symlinks are skipped out loud.** A skipped symlink now says so, and says
24+
that git tracks symlinks and fabric does not yet. Previously it was skipped in
25+
silence. Fabric still does not sync them: a symlink is a different kind of
26+
manifest entry rather than a file with a flag.
27+
28+
### Known limitation
29+
30+
- **Fabric cannot propagate a metadata-only change. One cause, two symptoms.**
31+
`SyncNode::local_write` returns early when the content hash is unchanged, and
32+
that early return is what makes applying a peer's content echo-free. So any
33+
change that alters no bytes never advances a logical version, and a change
34+
that does not advance a version never crosses the wire.
35+
36+
1. *A heartbeat is invisible.* Rewriting the same bytes with a new mtime does
37+
not propagate, so a replica keeps the older timestamp. A consumer must not
38+
read a replica's mtime as an activity signal.
39+
2. *A chmod is invisible.* `chmod +x` on an already-synced file changes no
40+
bytes, so the new mode does not propagate.
41+
42+
Both are the same defect. Fixing either symptom alone does not touch the
43+
cause. Closing it needs a local metadata-only change to advance a version
44+
while a received one stays inert, which puts an asymmetry into the exact
45+
mechanism that prevents infinite echo — a core engine change.
46+
47+
### Changed
48+
49+
- **The scan cache is keyed on local disk facts, not on the replicated
50+
manifest.** The cache reused a recorded hash when size and mtime matched what
51+
the *manifest* held — but the manifest crosses the wire, so a local caching
52+
decision was made from a value another machine chose. Two contending entries
53+
of equal size could collide on size plus mtime, and the cache then reported
54+
content the file did not hold. That was the permanent three-node divergence.
55+
56+
A separate, never-transmitted `scan_cache` now records what this machine
57+
observed on its own disk. It is deliberately not merged into the `observed`
58+
receipt: that receipt decides whether a missing path becomes a tombstone, and
59+
a tombstone crosses the wire, so loading a performance concern onto it would
60+
repeat the original mistake mirrored. The cache is filled only from real disk
61+
reads, never from a requested value, so a filesystem that truncates cannot
62+
make it miss forever. Absent in an older state file, where it warms on the
63+
first scan.
64+
65+
- **`FileMeta.mtime` is documented as informational and is never applied.**
66+
Fabric records a modification time, sends it, and does not write it to a
67+
materialized file, because git does not track mtime. The field is retained
68+
rather than removed: it has no serde default, so removing it would break
69+
parsing on any peer that has not upgraded.
70+
1171
- **`fabric status` reports per-path probe latency.** The daemon had measured a
1272
round trip and a path class on every liveness probe since the connection
1373
telemetry landed, and there was no way to read it but to parse

README.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -850,6 +850,37 @@ stop the daemon, and it does not affect the built-in `fabric shell` ALPN.
850850

851851
## File Sync
852852

853+
### What fabric syncs, and where it differs from git
854+
855+
A catalog should be carriable by either fabric or git, so the two must agree
856+
about what a catalog *is*. Fabric syncs the attributes git tracks:
857+
858+
| | git | fabric |
859+
| --- | --- | --- |
860+
| file content | yes | yes |
861+
| executable bit | yes | yes |
862+
| symlinks | yes | **no** — skipped, and logged when skipped |
863+
| modification time | no | recorded but never applied |
864+
| other permission bits | no | no |
865+
866+
Two differences are worth knowing before you rely on either transport.
867+
868+
**A `chmod` on an already-synced file does not propagate.** Git propagates one:
869+
a mode change rewrites the tree object and is a real commit. Fabric does not,
870+
because a chmod alters no bytes — see the limitation below. A **new** file
871+
carries its mode correctly.
872+
873+
**A same-content rewrite does not propagate at all.** Rewriting a file with
874+
identical bytes and a new timestamp changes nothing fabric will send, so a
875+
replica keeps its older mtime. **Do not read a replica's mtime as an activity
876+
signal** — if you need a heartbeat, put the time in the file's bytes.
877+
878+
Both are the same limitation: fabric cannot propagate a metadata-only change,
879+
because a change that alters no bytes never advances a logical version, and that
880+
early return is what keeps applying a peer's content free of echo loops.
881+
882+
883+
853884
`fabric sync` keeps a folder converged with trusted peers. A declarative config
854885
file lists sync *entries*; the running daemon watches each folder and syncs
855886
changes to peers near-instantly over fabric's own transport. A tool or a human

0 commit comments

Comments
 (0)